CN112395051B - Data fusion implementation method and device of observability system - Google Patents
Data fusion implementation method and device of observability system Download PDFInfo
- Publication number
- CN112395051B CN112395051B CN202011397029.XA CN202011397029A CN112395051B CN 112395051 B CN112395051 B CN 112395051B CN 202011397029 A CN202011397029 A CN 202011397029A CN 112395051 B CN112395051 B CN 112395051B
- Authority
- CN
- China
- Prior art keywords
- data
- network
- intercepting
- call data
- interception point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/251—Fusion techniques of input or preprocessed data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45591—Monitoring or debugging support
Abstract
The application relates to a method and a device for realizing data fusion of an observability system, wherein the method comprises the following steps: receiving external variables, wherein the external variables comprise a sampling mode for intercepting network traffic, a sampling proportion for intercepting the network traffic and an interception point set by a user; when the interceptor is successfully matched with the preset interception point or the interception point, intercepting system call data of a kernel space of the business system and function call data of a user space by adopting eBPF (enhanced binary band pass filter); outputting the system call data and the function call data in a log form, or forwarding the system call data and the function call data to a subsequent server by adopting eBPF. The data acquisition efficiency can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for implementing data fusion of an observability system.
Background
The implementation of the observability scheme relies on three aspects of data: trace/log/metrics (indexes), the usage scenarios of which are mainly in the field of APM and observability. At present, the main implementation method in the field of APM is to add relevant monitoring code segments by automatic code insertion of java agents or manual modification of business system codes. In the observability field, comprehensive service system operation state monitoring needs to be performed through three types of data, namely APM, Log and Metric, and usually the data come from independent monitoring acquisition agents or even independent monitoring management systems.
The existing data acquisition mode needs data integration processing, so that uniform observation and analysis are carried out on trace/log/metrics data of different sources and different formats, key information is extracted, and the processing mode for constructing service call chain data is too complex, and has certain difficulty in realizing real-time monitoring.
Disclosure of Invention
In order to improve the efficiency of data acquisition, the application provides a method and a device for realizing data fusion of an observability system.
In a first aspect, the present application provides a data fusion implementation method for an observability system, which is applied to an interceptor deployed in a business system, where a preset interception point is preconfigured in the interceptor, and the method includes:
receiving external variables, wherein the external variables comprise a sampling mode for intercepting network traffic, a sampling proportion for intercepting the network traffic and an interception point set by a user;
when the interceptor is successfully matched with the preset interception point or the interception point, intercepting system call data of a kernel space of the business system and function call data of a user space by adopting eBPF (enhanced binary band pass filter);
outputting the system call data and the function call data in a log form, or forwarding the system call data and the function call data to a subsequent server by adopting eBPF.
Preferably, the intercepting, by the eBPF, system call data of a kernel space of the business system includes:
intercepting network traffic, and constructing a traffic topological relation of the service system according to quintuple information of the network traffic;
counting system resource occupation indexes of processes related to the network traffic;
modifying network sending flow and network receiving flow in the network flow;
and carrying out corresponding entity conversion on the processes of the service system and the operating system layer.
Preferably, the modifying network transmission traffic in the network traffic includes:
judging whether call chain information exists in the network sending flow, wherein the call chain information comprises a unique identifier of a service request and an identifier of a current unit;
if not, completing the calling chain information in the network sending flow;
the modifying network receiving traffic in the network traffic includes:
in the network receiving flow, the identifier of the last basic working unit is modified into the upstream identifier of the current basic working unit.
Preferably, the intercepting, by the eBPF, the function call data of the user space of the service system includes:
collecting a calling path, an entry parameter, an exit parameter and time consumption of a function corresponding to the interception point, and generating service calling chain trace data by combining an identifier in network receiving flow of the current basic working unit;
and modifying the parameters of the interception points according to specified conditions.
Preferably, the method further comprises the following steps: and intercepting a kernel system calling sequence triggered by a process related to network traffic when the matching of the interceptor and the preset interception point or the interception point fails.
Preferably, the method further comprises the following steps: and when the language used by the service system is a non-compiled language, adding a layer of dtrace interception point conversion.
In a second aspect, the present application provides an apparatus for implementing data fusion of an observability system, which is applied to an interceptor deployed in a business system, where a preset interception point is preconfigured in the interceptor, the apparatus includes:
the system comprises a data receiving module, a data processing module and a data processing module, wherein the data receiving module is used for receiving external variables, and the external variables comprise a sampling mode for intercepting network traffic, a sampling proportion for intercepting the network traffic and an interception point set by a user;
the data interception module is used for intercepting system call data of a kernel space of the business system and function call data of a user space by adopting eBPF (enhanced binary band pass filter) when the interceptor is successfully matched with the preset interception point or the interception point;
and the data processing module is used for outputting the system call data and the function call data in a log form, or forwarding the system call data and the function call data to a subsequent server by adopting eBPF.
Preferably, the data interception module is further configured to: and intercepting a kernel system calling sequence triggered by a process related to network traffic when the matching of the interceptor and the preset interception point or the interception point fails.
In a third aspect, the present application provides an electronic device comprising a memory having a computer program stored thereon and a processor that, when executing the computer program, implements the method according to any of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to any one of the first aspect.
In the data acquisition method and device for the observability data fusion system provided by the embodiment of the application, firstly, an open interface provided by an interceptor is used for receiving external variables, then when the interceptor is successfully matched with a preset interception point or interception point, the eBPF is used for intercepting system call data of a kernel space of a business system and function call data of a user space, and finally the system call data and the function call data are output in a log form, or the eBPF is used for forwarding the system call data and the function call data to a subsequent service end, because the eBPF is a virtual machine which runs in a kernel and is executed in the kernel in a local code form and speed, the eBPF can be combined with a Trace system of the kernel to provide almost infinite observability, the efficiency of data acquisition and observability scheme construction is greatly improved, and the troubleshooting efficiency is further improved, time and labor cost are saved, and a systematic productization observability scheme is finally completed.
Drawings
FIG. 1 illustrates a block diagram of the hardware architecture of an exemplary computer terminal to which embodiments of the present application can be applied;
FIG. 2 shows a schematic flow diagram of a data fusion implementation method of an observability system according to an embodiment of the application;
FIG. 3 shows a block diagram of a data fusion implementation apparatus of an observability system according to an embodiment of the present application;
fig. 4 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 shows a hardware configuration block diagram of an exemplary computer terminal to which embodiments of the present application can be applied. As shown in fig. 1, computer terminal 100 may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions.
In addition, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is merely illustrative and is not intended to limit the structure of the electronic device. For example, computer terminal 100 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module, or be married in whole or in part to any of the other elements in the computer terminal 100. As contemplated in the disclosed embodiments, the data processing circuit acts as a processor control (e.g., selection of a variable resistance termination path to interface with).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data acquisition devices corresponding to the method for acquiring data in the embodiment of the present application, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the method for acquiring data of the application program. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, internal memory, or other non-volatile solid-state memory. In some embodiments, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 100 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 100. In one embodiment, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one embodiment, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with the user interface of the computer terminal 100.
It should be noted here that in some alternative embodiments, the computer device shown in fig. 1 described above may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements.
It should be noted that FIG. 1 is only one example of a specific example and is intended to illustrate the types of components that may be present in the computer device described above.
Fig. 2 shows a schematic flow diagram of a data fusion implementation method of an observability system according to an embodiment of the present application. In some embodiments, the method may be performed by, or implemented as, an interceptor deployed at the business system end. Referring to fig. 2, the data acquisition method includes the steps of:
step 210, receiving an external variable.
In this embodiment, the external variable may be provided by an external environment, and the external environment may be an environment variable, a configuration file, an interface call, or the like. External variables include, but are not limited to, the sampling mode of network traffic interception, the sampling proportion of network traffic interception, and the interception point set by the user. In one example, the interception points may be func1 and func2 in the main file.
In some embodiments, the external variable may be received by an interceptor deployed at the business system, in particular, the interceptor provides an open interface through which to receive variable settings from the external environment.
Of course, the interception point can be obtained not only by receiving external variables, but also by configuring a preset interception point in the interceptor in advance, for common main stream open source middleware, service framework, etc., a series of packaged interception points can be configured in the interceptor in advance, for example, librdkafka, libcur, boost, hiberlite, log4cpp, etc. (which may include a main stream common library of various C/C + +/golang/rust compiled languages).
In some embodiments, a layer of dtrace intercept point translation is added when the language used by the business system is a non-compiled language. For example, when the language used by the business system is Java, the Java application needs to provide the interception point through-XX: + ExtendedDTraceProbes of libjvm.
It should be noted that, when the interceptor is used to call the system call data of the kernel space of the service system and the function call data of the user space, it is necessary to first determine whether the interceptor is matched with a preset interception point or an interception point set by a user. Whether the interceptor is matched with a preset interception point or an interception point set by a user can be judged by adopting the existing technology, and the details are not repeated here.
When the interceptor is successfully matched with the preset interception point or the interception point, executing step 220; when the interceptor fails to match the predetermined interception point or the interception point, step 230 is performed.
Step 220, intercepting system call data of a kernel space and function call data of a user space of the business system by using eBPF.
In some embodiments, intercepting system call data of a kernel space of a business system using eBPF comprises: intercepting network flow, and constructing a flow topological relation of a service system according to quintuple information of the network flow; counting system resource occupation indexes of processes related to network traffic, such as IO, CPU, MEM and the like of a nginx system and a mysql system; modifying network sending flow and network receiving flow in the network flow; and carrying out corresponding entity conversion on the processes of the service system and the operating system layer.
In this embodiment, modifying the network transmission traffic in the network traffic includes: judging whether call chain information exists in network sending flow, wherein the call chain information comprises a unique identifier of a service request and an identifier of a current unit; if not, the call chain information is completed in the network sending flow.
In this embodiment, modifying the network reception traffic in the network traffic includes: in the network receiving flow, the identifier of the last basic working unit is modified into the upstream identifier of the current basic working unit.
In some embodiments, intercepting, with the eBPF, function call data of a user space of a business system includes: collecting a calling path, an entering parameter, an exiting parameter and time consumption of a function corresponding to an interception point, and generating service calling chain trace data by combining an identifier in network receiving flow of a current basic working unit; and modifying the parameters of the interception points according to specified conditions.
In this embodiment, the specified condition may be to determine whether to inject a log output by judging the ret state, the excepting, the variable value, and the like of the interception point, or to adjust the content of the existing log output, so as to be used for dynamically adjusting the log output of the non-service logic.
Step 230, intercepting a kernel system call sequence triggered by a process related to the network traffic.
In this embodiment, the kernel system call sequence triggered by the intercepted network traffic related process may be generated by a subsequent server in a sequence clustering machine learning manner to generate a main system call sequence classification for use in system layer performance analysis.
And 240, outputting the system call data and the function call data in a log form, or forwarding the system call data and the function call data to a subsequent server by adopting eBPF.
In this embodiment, the external variable is received through the open interface provided by the interceptor, and then when the interceptor is successfully matched with the preset interception point or the interception point, intercepting system call data of a kernel space and function call data of a user space of a business system by adopting eBPF (enhanced business process platform), finally outputting the system call data and the function call data in a log form, or the eBPF is adopted to forward the system call data and the function call data to the subsequent server, since the eBPF is a virtual machine running in the kernel, the kernel is executed in the form of local code and at the speed, and can be combined with a Trace system of the kernel to provide almost infinite observability, greatly improve the efficiency of data acquisition and observability scheme construction, therefore, the obstacle removing efficiency is further improved, the time and the labor cost are saved, and the systematic productization observability scheme is finally completed.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are exemplary embodiments and that the acts and modules referred to are not necessarily required in this application.
The above is a description of method embodiments, and the embodiments of the present application are further described below by way of apparatus embodiments.
Fig. 3 shows a block diagram of a data fusion implementation apparatus of an observability system according to an embodiment of the present application. In some embodiments, the data collection device may be included in, or implemented as, an interceptor. Referring to fig. 3, the data acquisition apparatus includes:
the data receiving module 310 is configured to receive external variables, where the external variables include a sampling mode for intercepting the network traffic, a sampling ratio for intercepting the network traffic, and an interception point set by a user.
And the data interception module 320 is configured to intercept system call data of a kernel space and function call data of a user space of the business system by using the eBPF when the interceptor is successfully matched with the preset interception point or the interception point.
And the data processing module 330 is configured to output the system call data and the function call data in a log form, or forward the system call data and the function call data to a subsequent server by using the eBPF.
In some embodiments, the data interception module 320 is further configured to: and intercepting a kernel system calling sequence triggered by a process related to the network traffic when the matching of the interceptor and a preset interception point or the interception point fails.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the described module may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
Fig. 4 shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present application.
As shown in fig. 4, the electronic apparatus includes a Central Processing Unit (CPU)401 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for system operation are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary.
In particular, according to embodiments of the present application, the process described above with reference to the flowchart fig. 2 may be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a machine-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409, and/or installed from the removable medium 411. The above-described functions defined in the system of the present application are executed when the computer program is executed by a Central Processing Unit (CPU) 401.
It should be noted that the computer readable medium shown in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, and may be described as: a processor includes a data receiving unit, a data intercepting unit, and a data processing unit. Where the names of these units or modules do not in some cases constitute a limitation of the unit or module itself, for example, the data receiving unit may also be described as a "unit for receiving external variables".
As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may be separate and not incorporated into the electronic device. The computer readable storage medium stores one or more programs that, when executed by one or more processors, perform the data fusion implementation methods described in the observability systems of the present application.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the application referred to in the present application is not limited to the embodiments with a particular combination of the above-mentioned features, but also encompasses other embodiments with any combination of the above-mentioned features or their equivalents without departing from the spirit of the application. For example, the above features may be replaced with (but not limited to) features having similar functions as those described in this application.
Claims (8)
1. A data fusion implementation method of an observability system is applied to an interceptor deployed in a business system, and is characterized in that a preset interception point is pre-configured in the interceptor, and the method comprises the following steps:
receiving external variables, wherein the external variables comprise a sampling mode for intercepting network traffic, a sampling proportion for intercepting the network traffic and an interception point set by a user;
when the interceptor is successfully matched with a preset interception point or an interception point, intercepting system call data of a kernel space of the business system and function call data of a user space by adopting eBPF;
outputting the system call data and the function call data in a log form, or forwarding the system call data and the function call data to a subsequent server by adopting eBPF;
the intercepting of the system call data of the kernel space of the business system by adopting the eBPF comprises the following steps:
intercepting network traffic, and constructing a traffic topological relation of the service system according to quintuple information of the network traffic;
counting system resource occupation indexes of processes related to the network traffic;
modifying network sending flow and network receiving flow in the network flow;
carrying out corresponding entity conversion on the processes of the business system and the operating system layer;
wherein the modifying network transmission traffic in the network traffic comprises:
judging whether call chain information exists in the network sending flow, wherein the call chain information comprises a unique identifier of a service request and an identifier of a current unit;
if not, completing the calling chain information in the network sending flow;
the modifying network received traffic in the network traffic comprises:
in the network receiving flow, the identifier of the last basic working unit is modified into the upstream identifier of the current basic working unit.
2. The method of claim 1, wherein intercepting, with the eBPF, function call data of a user space of the business system comprises:
collecting a calling path, an entering parameter, an exiting parameter and time consumption of a function corresponding to an interception point, and generating service calling chain trace data by combining an identifier in network receiving flow of a current basic working unit;
and modifying the parameters of the interception points according to specified conditions.
3. The method of claim 1, further comprising:
and intercepting a kernel system calling sequence triggered by a process related to network traffic when the matching of the interceptor and a preset interception point or the interception point fails.
4. The method of claim 1, further comprising:
and when the language used by the service system is a non-compiled language, adding a layer of dtrace interception point conversion.
5. A data fusion implementation device of an observability system is applied to an interceptor deployed in a business system, and is characterized in that a preset interception point is preconfigured in the interceptor, and the device comprises:
the system comprises a data receiving module, a data processing module and a data processing module, wherein the data receiving module is used for receiving external variables, and the external variables comprise a sampling mode for intercepting network traffic, a sampling proportion for intercepting the network traffic and an interception point set by a user;
the data interception module is used for intercepting system call data of a kernel space and function call data of a user space of the business system by adopting eBPF (enhanced binary phase band pass filter) when the interceptor is successfully matched with a preset interception point or interception point;
the data processing module is used for outputting the system call data and the function call data in a log form, or forwarding the system call data and the function call data to a subsequent server by adopting eBPF;
wherein the data processing module is specifically configured to:
intercepting network traffic, and constructing a traffic topological relation of the service system according to quintuple information of the network traffic;
counting system resource occupation indexes of processes related to the network traffic;
modifying network sending flow and network receiving flow in the network flow;
carrying out corresponding entity conversion on the processes of the business system and the operating system layer;
wherein the data processing module is further specifically configured to:
judging whether call chain information exists in the network sending flow, wherein the call chain information comprises a unique identifier of a service request and an identifier of a current unit;
if not, completing the calling chain information in the network sending flow;
the data processing module is further specifically configured to:
in the network receiving flow, the identifier of the last basic working unit is modified into the upstream identifier of the current basic working unit.
6. The apparatus of claim 5, wherein the data interception module is further configured to: and intercepting a kernel system calling sequence triggered by a process related to network traffic when the matching of the interceptor and a preset interception point or the interception point fails.
7. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the processor, when executing the computer program, implements the method of any of claims 1-4.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011397029.XA CN112395051B (en) | 2020-12-04 | 2020-12-04 | Data fusion implementation method and device of observability system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011397029.XA CN112395051B (en) | 2020-12-04 | 2020-12-04 | Data fusion implementation method and device of observability system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112395051A CN112395051A (en) | 2021-02-23 |
| CN112395051B true CN112395051B (en) | 2021-10-22 |
Family
ID=74605761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011397029.XA Active CN112395051B (en) | 2020-12-04 | 2020-12-04 | Data fusion implementation method and device of observability system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112395051B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115967467A (en) * | 2021-10-09 | 2023-04-14 | 中兴通讯股份有限公司 | Data processing method, device and computer readable storage medium |
| CN114039875B (en) * | 2021-10-30 | 2023-09-01 | 北京网聚云联科技有限公司 | Data acquisition method, device and system based on eBPF technology |
| CN114036569A (en) * | 2021-11-25 | 2022-02-11 | 江苏安超云软件有限公司 | eBPF-based data secure transmission method and device |
| CN116074130B (en) * | 2023-04-07 | 2023-06-20 | 中国工商银行股份有限公司 | System protection method, device, equipment and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175161A (en) * | 2019-04-25 | 2019-08-27 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium of record log |
| CN110213198A (en) * | 2018-02-28 | 2019-09-06 | 中标软件有限公司 | The monitoring method and system of network flow |
| CN110618912A (en) * | 2019-08-29 | 2019-12-27 | 凡普数字技术有限公司 | Method and device for monitoring equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10623372B2 (en) * | 2017-12-06 | 2020-04-14 | Nicira, Inc. | Load balancing IPsec tunnel processing with extended Berkeley packet filter (eBPF) |
-
2020
- 2020-12-04 CN CN202011397029.XA patent/CN112395051B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110213198A (en) * | 2018-02-28 | 2019-09-06 | 中标软件有限公司 | The monitoring method and system of network flow |
| CN110175161A (en) * | 2019-04-25 | 2019-08-27 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium of record log |
| CN110618912A (en) * | 2019-08-29 | 2019-12-27 | 凡普数字技术有限公司 | Method and device for monitoring equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| "eBPF技术简介";LFAPAC;《https://my.oschina.net/LFAPAC/blog/4520879 》;20200821;第1、4-5页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112395051A (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112395051B (en) | Data fusion implementation method and device of observability system | |
| CN103346974B (en) | Controlling method of service process and network device | |
| CN111177617A (en) | Web direct operation and maintenance method and device based on operation and maintenance management system and electronic equipment | |
| CN111177112A (en) | Database blocking method and device based on operation and maintenance management system and electronic equipment | |
| CN110390493B (en) | Task management method and device, storage medium and electronic equipment | |
| CN113158180B (en) | Method, device, equipment and readable storage medium for constructing automobile network security threat scene | |
| CN111813622A (en) | Equipment monitoring method and device | |
| CN112905645A (en) | Bank data processing method and device, electronic equipment and storage medium | |
| CN109508193B (en) | Application deployment and operation method, device, terminal equipment and medium | |
| CN111639059A (en) | Log information storage and positioning method, electronic equipment and storage medium | |
| CN114285774B (en) | Flow recording method and device, electronic equipment and storage medium | |
| CN111949862B (en) | Method and device for managing business task flow and electronic equipment | |
| CN115328506A (en) | Application package generation method, loading method, device, medium and equipment | |
| CN115061907A (en) | Test method and test system, electronic device, and storage medium | |
| CN110471718B (en) | Task processing method and device | |
| CN113141236A (en) | Message processing method and device | |
| CN108536482B (en) | Application shielding method, device, server and storage medium | |
| CN108288135B (en) | System compatibility method and device, computer readable storage medium and electronic equipment | |
| CN110908763A (en) | Acceptance method, device, equipment and storage medium | |
| CN117312761B (en) | Method and device for calculating data fragment processing time | |
| CN115309612B (en) | Method and device for monitoring data | |
| CN115543588B (en) | Electromagnetic perception task execution method, system and terminal based on script arrangement | |
| CN118041417A (en) | Link tracking method, device, equipment and readable storage medium | |
| CN113760694A (en) | Method and device for calculating code coverage rate | |
| CN113849398A (en) | Software tool kit testing method, system, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |