CN117056387A - Container log query method, related equipment and storage medium - Google Patents

Container log query method, related equipment and storage medium Download PDF

Info

Publication number
CN117056387A
CN117056387A CN202311083647.0A CN202311083647A CN117056387A CN 117056387 A CN117056387 A CN 117056387A CN 202311083647 A CN202311083647 A CN 202311083647A CN 117056387 A CN117056387 A CN 117056387A
Authority
CN
China
Prior art keywords
target
log
ebpf
container
component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311083647.0A
Other languages
Chinese (zh)
Inventor
请求不公布姓名
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Real AI Technology Co Ltd
Original Assignee
Beijing Real AI Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Real AI Technology Co Ltd filed Critical Beijing Real AI Technology Co Ltd
Priority to CN202311083647.0A priority Critical patent/CN117056387A/en
Publication of CN117056387A publication Critical patent/CN117056387A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2457Query processing with adaptation to user needs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45591Monitoring or debugging support
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The embodiment of the application discloses a container log query method, related equipment and a storage medium. The method is applied to a K8s system, the K8s system comprises a cluster gateway and a plurality of physical machines, service components are respectively deployed in the user space of each physical machine, eBPF components and eBPF cache spaces are respectively deployed in the kernel space of each physical machine, and the method comprises the following steps: the target physical machine obtains a log query request; the target service component determines a log path corresponding to the request and sends the log path and target time period information in the request to the target eBPF component; the target eBPF component acquires a target container log according to the log path and the target time period information, and stores the log into a target eBPF cache space; the target service component acquires the log from the target eBPF cache space and sends the log to the front-end equipment through the cluster gateway. By implementing the method provided by the embodiment of the application, the container log query can be performed on the premise of not wasting storage resources and not affecting the cluster stability.

Description

Container log query method, related equipment and storage medium
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a container log query method, related equipment and a storage medium.
Background
Kubernetes (K8 s for short) is a container orchestration engine of Google open source, which supports automated deployment, large-scale scalability, application containerization management, when an application is deployed in a K8s cluster, multiple instances of the application are typically deployed to load balance application requests, and in a distributed environment, the application in the K8s cluster will run on multiple physical machines.
The collection of K8s cluster container logs is crucial to effective management, monitoring and investigation, the container logs provide key information about the running state, performance and abnormal behavior of application programs, help operation and maintenance and development personnel to know the running condition of systems and applications and timely find and solve the problems, in K8s clusters, the logs are stored on a local file system of a physical machine by default, and because the application programs in the K8s clusters can run on a plurality of physical machines, the manual collection and analysis of the logs are very complex and inefficient
For this reason, the prior art provides an ELK (short for search server elastic search, log collection framework log stack, and visualization platform Kibana combination)/EFK (short for log collection framework elastic search, log collection framework Fluentd, and visualization platform Kibana combination) log collection scheme and a client-go (an official programmatic interactive client library written in Golang language)/kubecl logs (one of K8s command line tools) log collection scheme to implement automated container log collection; however, the ELK/EFK log collection scheme is used for collecting the container log, all logs in each node need to be backed up and stored in a dedicated storage space of the ELK/EFK in advance, and then the logs are extracted from the storage space for display, so that a large amount of disk space is needed for storing log data by using the ELK/EFK log collection scheme, and computer resources (memory, disk and the like) are wasted; the client-go/kubcl logs log collection scheme is used for collecting log containers, and the K8s apiserver is required to call kubcelet and CRI cluster components in a cluster to obtain log data of corresponding containers and then display, so that although the client-go/kubcl logs collection scheme requires less resources, when K8s clusters are larger, the K8s apiserver, kubcelet and interfaces (Container Runtime Interface, CRI) of the containers are significantly weighted when the log query requests are larger and the log quantity is larger, and the cluster stability is affected.
There is a need for a method for performing container log queries without wasting storage resources and affecting cluster stability.
Disclosure of Invention
The embodiment of the application provides a container log query method, related equipment and a storage medium, which can perform container log query on the premise of not wasting storage resources and not affecting cluster stability.
In a first aspect, an embodiment of the present application provides a container log query method, where the method is applied to a K8s system, where the K8s system includes a cluster gateway and a plurality of physical machines, the plurality of physical machines includes a target physical machine, service components are deployed in user spaces of the physical machines, and eBPF components and eBPF cache spaces are deployed in kernel spaces of the physical machines, respectively, where the method includes:
the target physical machine receives a log query request from front-end equipment from the cluster gateway, wherein the log query request carries a target container identifier and target period information;
the target service component in the target physical machine determines a target container log path according to the target container identifier and a preset first corresponding relation, and sends the target container log path and the target period information to a target eBPF component, wherein the first corresponding relation comprises a corresponding relation between the container identifier and the log path, and the target eBPF component is the eBPF component in the target physical machine;
The target eBPF component acquires a target container log from a target log file according to the target container log path and the target period information, and stores the target container log into a target eBPF cache space in the target physical machine, wherein the container log of each container in the target physical machine is stored in the target log file;
and the target service component acquires the target container log from the target eBPF cache space and sends the target container log to the front-end equipment through the cluster gateway.
In a second aspect, an embodiment of the present application further provides a K8s system, where the K8s system includes a trunking gateway and a plurality of physical machines, where the plurality of physical machines includes a target physical machine, service components are deployed in user spaces of the physical machines, and eBPF components and eBPF cache spaces are deployed in kernel spaces of the physical machines, respectively, where:
the target physical machine is used for receiving a log query request from the front-end equipment from the cluster gateway, wherein the log query request carries a target container identifier and target period information;
the target service component in the target physical machine is configured to determine a target container log path according to the target container identifier and a preset first correspondence, and send the target container log path and the target period information to a target eBPF component, where the first correspondence includes a correspondence between the container identifier and the log path, and the target eBPF component is an eBPF component in the target physical machine;
The target eBPF component is used for acquiring a target container log from a target log file according to the target container log path and the target time period information, storing the target container log into a target eBPF cache space in the target physical machine, and storing the container log of each container in the target physical machine in the target log file;
the target service component is further configured to obtain the target container log from the target eBPF cache space, and send the target container log to the front-end device through the trunking gateway.
In some embodiments, when executing the step of receiving the log query request from the front-end device from the cluster gateway, the target physical machine is specifically configured to:
receiving the log query request sent by the front-end equipment through the cluster gateway; determining a target physical machine identifier through the cluster gateway according to the target container identifier and a preset second corresponding relation, wherein the second corresponding relation comprises a corresponding relation between the container identifier and the physical machine identifier; the log query request is sent to the target physical machine through the cluster gateway according to the target physical machine identifier; or alternatively, the first and second heat exchangers may be,
Receiving the log query request sent by the front-end equipment through the cluster gateway; and sending the log query request to the target physical machine through the cluster gateway according to the target physical machine identifier carried in the log query request.
In some embodiments, before the step of determining the target container log path according to the target container identifier and the preset first correspondence is performed by the target service component in the target physical machine:
the target service component is further configured to decode the log query request according to a preset target decoding rule, and obtain the target container identifier and the target period information;
at this time, after the target service component in the target physical machine acquires the target container log from the target eBPF cache space, and before the step of sending, by the cluster gateway, the target container log to the front-end device:
the target server component is further configured to perform an encoding operation on the target container log according to a preset target encoding rule, obtain a target encoded log, and use the target encoded log as the target container log.
In some embodiments, the target eBPF component is specifically configured to, when executing the step of obtaining the target container log from the target log file according to the target container log path and the target period information:
determining a target container log subfile from the target log file according to the target container log path; and acquiring the target container log corresponding to the target time period information from the target container log subfile.
In some embodiments, the target eBPF component is mounted on a kernel function of the target physical machine; before the target service component performs the step of sending the target container log path and the target period information to a target eBPF component:
the target service component is further used for starting the target eBPF component by calling the kernel function so as to operate the target eBPF component;
the target eBPF component is further configured to obtain a target container log from a target log file according to the target container log path and the target period information, and store the target container log into a target eBPF cache space in the target physical machine, where the method further includes:
And the target service component is further used for closing the target eBPF component by calling the kernel function, so that the target eBPF component stops running.
In some embodiments, a target eBPF verifier and a target kernel virtual machine are deployed in kernel space of the target physical machine; before the target service component performs the step of starting the target eBPF component by calling the kernel function:
the target service component is further configured to send a target eBPF bytecode preset in the user space to the target eBPF verifier;
the target eBPF verifier is further used for performing verification processing on the target eBPF byte codes to obtain verification results, wherein the verification processing comprises infinite loop verification processing, crash code verification processing and running time verification processing;
if the result is that the verification passes, the target eBPF verifier is further configured to send the target eBPF byte code to the target kernel virtual machine;
and the target eBPF compiler in the target kernel virtual machine is also used for compiling the target eBPF byte code to obtain the target eBPF component, and mounting the target eBPF component on a kernel function of the target physical machine.
In some embodiments, before performing the step of receiving the log query request from the front-end device in the cluster gateway, the target physical machine is further configured to:
and acquiring a go program package, and constructing the target service component in a user space according to the go program package.
In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the method when executing the computer program.
In a fourth aspect, embodiments of the present application also provide a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, implement the above-described method.
In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a transceiver coupled to a terminal, for executing the technical solution provided in the first aspect of the embodiment of the present application.
In a sixth aspect, an embodiment of the present application provides a chip system, where the chip system includes a processor, configured to support a terminal to implement a function related to the first aspect, for example, generate or process information related to a container log query method provided in the first aspect. In one possible design, the above chip system further includes a memory for holding program instructions and data necessary for the terminal. The chip system may be formed of a chip or may include a chip and other discrete devices.
In a seventh aspect, an embodiment of the present application provides a computer program product including instructions, where the computer program product when executed on a computer causes the computer to execute the container log query method provided in the first aspect, and also achieves the beneficial effects of the container log query method provided in the first aspect.
Compared with the prior art, in the scheme provided by the embodiment of the application, in the first aspect, because the eBPF component and the eBPF cache space are deployed in the kernel space of the physical machine in the embodiment, the target container log corresponding to the log query request can be directly obtained through the eBPF component, the target container log is stored in the target eBPF cache space, the service component in the user space can obtain the target container log from the target eBPF cache space, and the target container log is returned to the front-end equipment, so that the query of the container log file of the front-end equipment is completed, and compared with the ELK/EFK log acquisition scheme in the prior art, the scheme directly obtains the required log from the log file through the eBPF component and the eBPF cache space without an additional disk space, so that the storage resource can be saved by carrying out the query of the container log through the scheme; in the second aspect, the solution can directly realize the inquiry of the container log through the eBPF component, the eBPF cache space and the service component deployed in the user space, and does not need to call the cluster component, so that the burden of the cluster component is not increased by the log inquiry performed by the solution, and the occurrence of unstable cluster caused by the log inquiry performed by calling the cluster component is avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a container log query method provided by an embodiment of the present application;
FIG. 2 is a flow chart of a method for querying a container log according to an embodiment of the present application;
FIG. 3 is a schematic block diagram of a K8s system provided by an embodiment of the present application;
fig. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a server according to an embodiment of the present application.
Detailed Description
The terms first, second and the like in the description and in the claims of embodiments of the application and in the above-described figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those explicitly listed but may include other steps or modules not expressly listed or inherent to such process, method, article, or apparatus, such that the partitioning of modules by embodiments of the application is only one logical partitioning, may be implemented with additional partitioning, such as a plurality of modules may be combined or integrated in another system, or some features may be omitted, or not implemented, and further, such that the coupling or direct coupling or communication connection between modules may be via some interfaces, indirect coupling or communication connection between modules may be electrical or otherwise similar, none of which are limited in embodiments of the application. The modules or sub-modules described as separate components may or may not be physically separate, may or may not be physical modules, or may be distributed in a plurality of circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purposes of the embodiment of the present application.
The embodiment of the application provides a container log query method, related equipment and a storage medium, wherein an execution subject of the container log query method can be a K8s system or computer equipment integrated with the K8s system, the K8s system can be realized in a hardware or software mode, and the computer equipment can be a terminal or a server.
When the computer device is a server, the server may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligence platforms, and the like.
When the computer device is a terminal, the terminal may include: smart phones, tablet computers, notebook computers, desktop computers, smart televisions, smart speakers, personal digital assistants (hereinafter abbreviated as PDA, english: personal Digital Assistant), desktop computers, smart watches, and the like, which carry multimedia data processing functions (e.g., video data playing functions, music data playing functions), but are not limited thereto.
The scheme of the embodiment of the application can be realized based on an artificial intelligence technology, and particularly relates to the technical field of computer vision in the artificial intelligence technology and the fields of cloud computing, cloud storage, databases and the like in the cloud technology, and the technical fields are respectively described below.
Artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.
The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
Computer Vision (CV) is a science of studying how to "look" a machine, and more specifically, to replace human eyes with a camera and a Computer to perform machine Vision such as recognition, tracking and measurement on a target, and further perform graphic processing to make the Computer process into an image more suitable for human eyes to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data. Computer vision techniques typically include image processing, model robustness detection, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D techniques, virtual reality, augmented reality, synchronous positioning, map construction, etc., as well as common model robustness detection, fingerprint recognition, etc., biometric techniques.
With research and advancement of artificial intelligence technology, research and application of artificial intelligence technology is being developed in various fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned, automatic driving, unmanned aerial vehicles, robots, smart medical treatment, smart customer service, etc., and it is believed that with the development of technology, artificial intelligence technology will be applied in more fields and with increasing importance value.
The scheme of the embodiment of the application can be realized based on cloud technology, and particularly relates to the technical fields of cloud computing, cloud storage, databases and the like in the cloud technology, and the technical fields are respectively described below.
Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. Cloud technology (Cloud technology) is based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by Cloud computing business models, and can form a resource pool, so that the Cloud computing business model is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a significant amount of computing, storage resources, such as video websites, image-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing. According to the embodiment of the application, the identification result can be stored through the cloud technology.
Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside. In the embodiment of the application, the information such as network configuration and the like can be stored in the storage system, so that the server can conveniently call the information.
At present, the storage method of the storage system is as follows: when creating logical volumes, each logical volume is allocated a physical storage space, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage space of the logical volume, and the file system records storage position information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage position information of each object.
The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided into stripes in advance according to the set of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID, redundant Array of Independent Disk), and a logical volume can be understood as a stripe, whereby physical storage space is allocated for the logical volume.
The Database (Database), which can be considered as an electronic filing cabinet, is a place for storing electronic files, and users can perform operations such as adding, inquiring, updating, deleting and the like on the data in the files. A "database" is a collection of data stored together in a manner that can be shared with multiple users, with as little redundancy as possible, independent of the application.
The database management system (Database Management System, abbreviated as DBMS) is a computer software system designed for managing databases, and generally has basic functions of storage, interception, security, backup and the like. The database management system may classify according to the database model it supports, e.g., relational, XML (Extensible Markup Language ); or by the type of computer supported, e.g., server cluster, mobile phone; or by the query language used, e.g., SQL (structured query language ), XQuery; or by performance impact emphasis, such as maximum scale, maximum speed of operation; or other classification schemes. Regardless of the manner of classification used, some DBMSs are able to support multiple query languages across categories, for example, simultaneously. In the embodiment of the application, the identification result can be stored in the database management system, so that the server can conveniently call.
It should be specifically noted that, the service terminal according to the embodiments of the present application may be a device that provides voice and/or data connectivity to the service terminal, a handheld device with a wireless connection function, or other processing device connected to a wireless modem. Such as mobile telephones (or "cellular" telephones) and computers with mobile terminals, which can be portable, pocket, hand-held, computer-built-in or car-mounted mobile devices, for example, which exchange voice and/or data with radio access networks. For example, personal communication services (English full name: personal Communication Service, english short name: PCS) telephones, cordless telephones, session Initiation Protocol (SIP) phones, wireless local loop (Wireless Local Loop, english short name: WLL) stations, personal digital assistants (English full name: personal Digital Assistant, english short name: PDA) and the like.
Referring to fig. 1, fig. 1 is a schematic application scenario diagram of a container log query method according to an embodiment of the present application. The container log query method is applied to a K8s system in fig. 1, the K8s system includes a cluster gateway 10 and a plurality of physical machines, the plurality of physical machines include target physical machines, service components are respectively deployed in user spaces of the physical machines, primary packet filters (extended Berkeley Packet Filter, eBPF) components and eBPF cache spaces (eBPF Maps) with eBPF extensions are respectively deployed in kernel spaces of the physical machines, and in the embodiment of the application, a front-end device queries a container log in the target physical machine for illustration, and the front-end device queries a log container in other physical machines in the K8s system similar to a flow of querying a log container in the target physical machine.
As shown in fig. 1, a target service component is deployed in a user space of a target physical machine, and a target eBPF component and a target eBPF cache space are deployed in a kernel space. When a user needs to inquire a log of a certain container in a K8s system through front-end equipment, sending a log inquiry request to the K8s system through the front-end equipment, wherein the log inquiry request carries a target container identifier and target period information, after the cluster gateway 10 in the K8s system receives the log inquiry request, routing the request to a target physical machine according to the target container identifier in the log inquiry request, and then determining a target container log path by a target service component according to the target container identifier and a preset first corresponding relation, and sending the target container log path and the target period information to a target eBPF component, wherein the first corresponding relation comprises a corresponding relation between the container identifier and the log path; the target eBPF component acquires a target container log from a target log file according to a target container log path and target period information, and stores the target container log into a target eBPF cache space in the target physical machine; finally, the target service component acquires the target container log from the target eBPF cache space, and sends the target container log to the front-end equipment through the cluster gateway 10, so that a user inquires the required container log through the front-end equipment.
The eBPF cache space in the embodiment allows eBPF components in the kernel space and service components in the user space to read and write data in the eBPF cache space.
Fig. 2 is a flow chart of a container log query method according to an embodiment of the present application. As shown in fig. 2, the method includes the following steps 101-108.
101. The cluster gateway receives a log query request sent by front-end equipment, wherein the log query request carries a target container identifier and target period information.
In some embodiments, when a user needs to query a log in the K8s system, a log query request is sent to the K8s system through a front-end device, and at this time, a cluster gateway in the K8s system receives the log query request.
Before step 101 is performed, deployment of the target service component needs to be completed in advance, and in some embodiments, deployment may be performed by: and the target physical machine acquires a go program package and constructs the target service component in a user space according to the go program package.
In this embodiment, the performance of the go program is better than that of the c program, so that the target service component is constructed by using the go program package, and the performance of the target service component is also better.
In some embodiments, the user may scroll the query log at the head-end equipment, and each time it scrolls, a log query request corresponding to the scrolled page may be generated.
102. And the cluster gateway routes the log query request to a corresponding target physical machine.
In this embodiment, after receiving the log query request, the trunking gateway routes the log query request to the corresponding target physical machine.
In some embodiments, a second correspondence is preset in the trunking gateway, where the second correspondence includes a correspondence between a container identifier and a physical machine identifier, and at this time, after the trunking gateway receives a log query request, a target physical machine identifier is determined according to the target container identifier and the preset second correspondence, and then the log query request is sent to the target physical machine according to the target physical machine identifier, that is, the log query request is sent to a target physical machine corresponding to the target physical machine identifier.
Therefore, in this embodiment, since the second correspondence is maintained in the trunking gateway, the trunking gateway can automatically determine the target physical machine identifier corresponding to the target container identifier according to the second correspondence, without manually designating the target physical machine identifier by the user, thereby reducing the workload of the user for querying the log.
In other embodiments, the log query request carries a target physical machine identifier, where the target physical machine identifier may be obtained by a front-end device or may be specified by a target user, and at this time, after receiving the log query request, the cluster gateway may send the log query request to a corresponding target physical machine directly according to the target physical machine identifier carried in the log query request.
Therefore, in this embodiment, the trunking gateway may directly route the request according to the target physical machine identifier in the log query request, without further judging the physical machine identifier corresponding to the request by the trunking gateway, thereby reducing the workload of the trunking gateway and accelerating the efficiency of forwarding the request.
103. And determining a target container log path by the target service component in the target physical machine according to the target container identifier and a preset first corresponding relation.
The first corresponding relation comprises a corresponding relation between a container identifier and a log path, and the target container log path is a path corresponding to the log of the target container in the target log file.
In this embodiment, when the cluster gateway routes the log query request to a corresponding target physical machine, the target physical machine receives the log query request through a target service component in the user space, then obtains a target container identifier in the log query request, and then determines a target container log path corresponding to the target container identifier according to a first corresponding relationship.
In some embodiments, the data structure exchanged between the service component and the front-end device in the K8s system is agreed, specifically, a Protocol buffer bidirectional flow RPC Protocol is used as a communication Protocol between the front-end device and the service component in the physical machine, and a target decoding rule and a target encoding rule can be set for the front-end device and the service component respectively based on the Protocol.
The Protocol Buffers are lightweight and efficient binary data exchange formats, so that the data transmission speed can be improved and the memory occupation can be reduced.
At this time, before sending the log query request, the front-end device encodes the log query request by using a predetermined target encoding rule, where the target encoding rule is a rule generated based on a protocol buffer bi-directional flow remote procedure call protocol (Remote Procedure Call Protocol, RPC), for example, may be a serialization rule, and may perform serialization processing on the log query request, and then send the encoded request to the K8s system, where the K8s system forwards the request to the target service component through the corresponding target physical machine.
At this time, after the target service component receives the log query request again, in order to identify the content of the request, the request needs to be decoded first to obtain the specific content of the request, specifically, before step 103, the method further includes: and the target service component decodes the log query request according to a preset target decoding rule to acquire the target container identifier and the target period information.
Wherein the target decoding rule may be an inverse serialization rule.
104. The target service component sends the target container log path and the target period information to a target eBPF component, wherein the first corresponding relation comprises a corresponding relation between a container identifier and the log path, and the target eBPF component is the eBPF component in the target physical machine.
In this embodiment, after the target service component obtains the target container log path, the target period information in the request needs to be extracted, and then the target container log path and the target period information are transmitted to the target eBPF component.
In some embodiments, the target eBPF component in this embodiment is mounted on a kernel function of the target physical machine, so that, in order to save device resources, the eBPF component is only operated when being called, and is in a stop operation state when not being called, and before the target service component sends the target container log path and the target period information to the target eBPF component, the kernel function needs to be called first to start the target eBPF component, so as to operate the target eBPF component.
The method comprises the steps of loading eBPF byte codes into kernel space to obtain an eBPF component in order to ensure the safety of the target eBPF component, wherein the kernel space of a target physical machine is provided with a target eBPF verifier and a target kernel virtual machine, and the steps of loading the eBPF byte codes into the kernel space to obtain the eBPF component are as follows: the target service component sends target eBPF byte codes preset in the user space to the target eBPF verifier; the target eBPF verifier performs verification processing on the target eBPF byte codes to obtain verification results, wherein the verification processing comprises infinite loop verification processing, crash code verification processing and running time verification processing; if the result is that the verification is passed, the target eBPF verifier sends the target eBPF byte code to the target kernel virtual machine; and a target eBPF compiler in the target kernel virtual machine compiles the target eBPF byte codes to obtain the target eBPF component, and mounts the target eBPF component on a kernel function of the target physical machine.
Specifically, the target eBPF compiler may compile the target eBPF bytecode into a target eBPF component, which is Native Code (Native Code), using Just-In-Time (JIT) technology.
The target eBPF component in the embodiment runs in the target kernel virtual machine, and the target eBPF component is operated in the sandbox without compiling the kernel, so that the safety and stability of the kernel can be improved.
105. And the target eBPF component acquires a target container log from a target log file according to the target container log path and the target period information.
And the target log file stores container logs of all containers in the target physical machine.
Specifically, in some embodiments, step 105 includes:
the target eBPF component determines a target container log subfile from the target log file according to the target container log path; the target eBPF component obtains the target container log corresponding to the target period information from the target container log subfile.
That is, the target eBPF component first finds a target container log subfile storing a log of a target container from the target log file, and then acquires a log conforming to target period information from the target container log subfile as a target container log, for example, the target period information is 2023, 8, 20, 10, 3, 20, 11, and at this time, the log in the period is screened out from the target container log subfile.
106. The target eBPF component stores the target container log into a target eBPF cache space in the target physical machine.
In this embodiment, in order to send the filtered target container log to the user space, the target container log needs to be stored in the target eBPF cache space, and the data in the target eBPF cache space can be read-write shared with the target service component.
107. The target service component obtains the target container log from the target eBPF cache space.
In this embodiment, the target server component may monitor in real time whether there is new data (target container log) in the target eBPF cache space, and if there is new data, read the new data to the user space in real time, or read the data in the target eBPF cache space once at intervals of a preset duration (for example, 1 s), and the specific reading mode is not limited in this embodiment.
In this embodiment, after the target service component obtains the target container log from the target eBPF cache space, the read target container log is deleted from the target eBPF cache space.
108. The target service component sends the target container log to the front-end equipment through the cluster gateway.
In some embodiments, in order to improve the transmission efficiency of the log, after the target server component obtains the target container log, further encoding operation needs to be performed on the target container log according to a preset target encoding rule to obtain a target encoded log, the target encoded log is used as the target container log, and then the target container log is sent to the front-end device through the cluster gateway.
In this embodiment, the target encoding rule may be a serialization rule.
In summary, in the solution provided in the embodiment of the present application, in the first aspect, since the kernel space of the physical machine in the present embodiment is configured with the eBPF component and the eBPF cache space, the target container log corresponding to the log query request may be directly obtained by the eBPF component, the target container log is stored in the target eBPF cache space, the service component in the user space may obtain the target container log from the target eBPF cache space, and return the target container log to the front-end device, so as to complete the query of the front-end device container log file, and compared with the ELK/EFK log acquisition solution in the prior art, the solution directly obtains the required log from the log file through the eBPF component and the eBPF cache space, without additional disk space to backup the container log, so that the query of the container log by the solution may save storage resources; in the second aspect, the solution can directly realize the inquiry of the container log through the eBPF component, the eBPF cache space and the service component deployed in the user space, and does not need to call the cluster component, so that the burden of the cluster component is not increased by the log inquiry performed by the solution, and the occurrence of unstable cluster caused by the log inquiry performed by calling the cluster component is avoided.
In addition, it is verified that the present embodiment deploys the go-program-based service component, the eBPF component, and the eBPF cache space on the physical machine, and the required memory space is smaller than 2gb,0.5 vCPU (CPU in the virtual machine) and no additional disk space is required. For the ELK/EFK log acquisition scheme in the prior art, the installation files of the log acquisition scheme of the scheme are far smaller than the installation files required by the ELK/EFK, so that storage resources are further saved.
Any technical features mentioned in the embodiments corresponding to any one of fig. 1 to fig. 2 are also applicable to the embodiments corresponding to fig. 3 to fig. 5 in the embodiments of the present application, and the following similar parts will not be repeated.
The description of a container log query method in the embodiment of the present application is provided above, and the description of a K8s system (e.g., server, user terminal) for executing the container log query method is provided below.
Referring to fig. 3, a schematic diagram of a K8s system shown in fig. 3 is shown, which can be applied to a container log query scenario. The K8s system 30 in the embodiment of the present application can implement the steps corresponding to the seed container log query method performed in the embodiment corresponding to any one of the above-described fig. 1-2. The functions implemented by the K8s system 30 may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above, which may be software and/or hardware. The K8s system 30 may include a cluster gateway 301 and a plurality of physical machines, where the plurality of physical machines includes a target physical machine 302 (physical machines other than the target physical machine 302 are not identified in fig. 3), service components are deployed in a user space of each physical machine, an eBPF component and an eBPF cache space are deployed in a kernel space of each physical machine, a target service component 3021 is deployed in a user space of the target physical machine 302, and a target eBPF component 3022 and a target eBPF cache space 3023 are deployed in a kernel space of the target physical machine 302, where:
The target physical machine 302 is configured to receive a log query request from the front-end device from the trunking gateway 301, where the log query request carries a target container identifier and target period information;
the target service component 3021 in the target physical machine 302 is configured to determine a target container log path according to the target container identifier and a preset first correspondence, and send the target container log path and the target period information to a target eBPF component 3022, where the first correspondence includes a correspondence between a container identifier and a log path, and the target eBPF component 3022 is an eBPF component in the target physical machine 302;
the target eBPF component 3022 is configured to obtain a target container log from a target log file according to the target container log path and the target period information, and store the target container log into a target eBPF cache space 3023 in the target physical machine 302, where the target log file stores container logs of containers in the target physical machine 302;
the target service component 3021 is further configured to obtain the target container log from the target eBPF cache space 3023, and send the target container log to the front-end device through the cluster gateway 301.
In some embodiments, the target physical machine 302 is specifically configured to, when executing the log query request step received from the front-end device in the trunking gateway 301:
receiving, by the trunking gateway 301, the log query request sent by the front-end device; determining, by the trunking gateway 301, a target physical machine 302 identifier according to the target container identifier and a preset second correspondence, where the second correspondence includes a correspondence between a container identifier and a physical machine identifier; sending the log query request to the target physical machine 302 according to the target physical machine 302 identifier through the cluster gateway 301; or alternatively, the first and second heat exchangers may be,
receiving, by the trunking gateway 301, the log query request sent by the front-end device; and sending the log query request to the target physical machine 302 according to the target physical machine 302 identifier carried in the log query request through the cluster gateway 301.
In some embodiments, before the target service component 3021 in the target physical machine 302 performs the step of determining a target container log path according to the target container identifier and the preset first correspondence:
The target service component 3021 is further configured to decode the log query request according to a preset target decoding rule, and obtain the target container identifier and the target period information;
at this time, after the target service component 3021 in the target physical machine 302 acquires the target container log from the target eBPF cache space 3023, and before the step of sending the target container log to the front-end device through the cluster gateway 301:
the target server component is further configured to perform an encoding operation on the target container log according to a preset target encoding rule, obtain a target encoded log, and use the target encoded log as the target container log.
In some embodiments, the target eBPF component 3022 is specifically configured to, when performing the step of obtaining the target container log from the target log file according to the target container log path and the target period information:
determining a target container log subfile from the target log file according to the target container log path; and acquiring the target container log corresponding to the target time period information from the target container log subfile.
In some embodiments, the target eBPF component 3022 is mounted on the kernel function of the target physical machine 302; before the target service component 3021 performs the step of sending the target container log path and the target period information to a target eBPF component 3022:
the target service component 3021 is further configured to initiate the target eBPF component 3022 by calling the kernel function to run the target eBPF component 3022;
the target eBPF assembly 3022 is further configured to obtain a target container log from a target log file according to the target container log path and the target period information, and store the target container log in a target eBPF cache space 3023 in the target physical machine 302, where the method further includes:
the target service component 3021 is further configured to close the target eBPF component 3022 by calling the kernel function, so that the target eBPF component 3022 stops operating.
In some embodiments, the target physical machine 302 has a target eBPF verifier deployed in its kernel space along with a target kernel virtual machine; before the target service component 3021 performs the step of starting the target eBPF component 3022 by calling the kernel function:
The target service component 3021 is further configured to send a target eBPF bytecode preset in the user space to the target eBPF verifier;
the target eBPF verifier is further used for performing verification processing on the target eBPF byte codes to obtain verification results, wherein the verification processing comprises infinite loop verification processing, crash code verification processing and running time verification processing;
if the result is that the verification passes, the target eBPF verifier is further configured to send the target eBPF byte code to the target kernel virtual machine;
the target eBPF compiler in the target kernel virtual machine is further configured to compile the target eBPF bytecode to obtain the target eBPF component 3022, and mount the target eBPF component 3022 onto a kernel function of the target physical machine 302.
In some embodiments, before executing the step of receiving the log query request from the front-end device in the cluster gateway 301, the target physical machine 302 is further configured to:
a go package is obtained and the target service component 3021 is built in user space according to the go package.
In summary, in the solution provided in the embodiment of the present application, in the first aspect, since the kernel space of the target physical machine 302 in the present embodiment is deployed with the target eBPF component 3022 and the target eBPF cache space 3023, the target container log corresponding to the log query request may be directly obtained through the target eBPF component 3022, the target container log is stored in the target eBPF cache space 3023, the service component in the user space may obtain the target container log from the target eBPF cache space 3023, and return the target container log to the front-end device, so as to complete the query of the front-end device container log file, compared with the ELK/EFK log collection solution in the prior art, the solution directly obtains the required log from the log file through the target eBPF component 3022 and the target eBPF cache space 3023, and does not need an additional disk space to back up the container log, so that the query of the container log through the solution can save storage resources; in the second aspect, the solution can directly implement the query of the container log through the target eBPF component 3022, the target eBPF buffer space 3023 deployed in the kernel and the service component deployed in the user space, and does not need to call the cluster component, so that the burden of the cluster component is not increased by the log query performed by the solution, and thus the occurrence of unstable cluster caused by calling the cluster component to perform the log query is avoided.
The container log query in the embodiment of the present application is described below from the viewpoint of hardware processing.
The embodiment of the present application further provides a terminal, as shown in fig. 4, for convenience of explanation, only the relevant parts of the embodiment of the present application are shown, and specific technical details are not disclosed, please refer to the method part of the embodiment of the present application. The terminal can be any terminal including a mobile phone, a tablet personal computer, a personal digital assistant (English: personal Digital Assistant, english: PDA), a Sales terminal (English: point of Sales, english: POS), a vehicle-mounted computer and the like, taking the terminal as the mobile phone as an example:
fig. 4 is a block diagram showing a part of a structure of a mobile phone related to a terminal provided by an embodiment of the present application. Referring to fig. 4, the mobile phone includes: radio Frequency (RF) circuit 55, memory 520, input unit 530, display unit 540, sensor 550, audio circuit 560, wireless fidelity (wireless fidelity, wi-Fi) module 570, processor 580, and power supply 590. Those skilled in the art will appreciate that the handset configuration shown in fig. 4 is not limiting of the handset and may include more or fewer components than shown, or may combine certain components, or may be arranged in a different arrangement of components.
The following describes the components of the mobile phone in detail with reference to fig. 4:
the RF circuit 55 may be used for receiving and transmitting signals during the process of receiving and transmitting information or communication, in particular, after receiving downlink information of the base station, the downlink information is processed by the processor 580; in addition, the data of the design uplink is sent to the base station. Generally, RF circuitry 55 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (English full name: low Noise Amplifier; LNA), a duplexer, and the like. In addition, the RF circuitry 55 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to global system for mobile communications (english: global System of Mobile communication, english: GSM), general packet radio service (english: general Packet Radio Service, english: GPRS), code division multiple access (english: code Division Multiple Access, CDMA), wideband code division multiple access (english: wideband Code Division Multiple Access, english: WCDMA), long term evolution (english: long Term Evolution, english: LTE), email, short message service (english: short Messaging Service, english: SMS), and the like.
The memory 520 may be used to store software programs and modules, and the processor 580 performs various functional applications and data processing of the cellular phone by executing the software programs and modules stored in the memory 520. The memory 520 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
The input unit 530 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the handset. In particular, the input unit 530 may include a touch panel 531 and other input devices 532. The touch panel 531, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 531 or thereabout by using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a predetermined program. Alternatively, the touch panel 531 may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 580, and can receive commands from the processor 580 and execute them. In addition, the touch panel 531 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The input unit 530 may include other input devices 532 in addition to the touch panel 531. In particular, other input devices 532 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc.
The display unit 540 may be used to display information input by a user or information provided to the user and various menus of the mobile phone. The display unit 540 may include a display panel 541, and optionally, the display panel 541 may be configured in the form of a liquid crystal display (english: liquid Crystal Display, abbreviated as LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 531 may cover the display panel 541, and when the touch panel 531 detects a touch operation thereon or thereabout, the touch operation is transferred to the processor 580 to determine the type of the touch event, and then the processor 580 provides a corresponding visual output on the display panel 541 according to the type of the touch event. Although in fig. 4, the touch panel 531 and the display panel 541 are two independent components to implement the input and input functions of the mobile phone, in some embodiments, the touch panel 531 and the display panel 541 may be integrated to implement the input and output functions of the mobile phone.
The handset may also include at least one sensor 550, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel 541 according to the brightness of ambient light, and a proximity sensor that may turn off the display panel 541 and/or the backlight when the mobile phone moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured with the handset are not described in detail herein.
Audio circuitry 560, speakers 561, microphone 562 may provide an audio interface between the user and the handset. The audio circuit 560 may transmit the received electrical signal converted from audio data to the speaker 561, and the electrical signal is converted into a sound signal by the speaker 561 and output; on the other hand, microphone 562 converts the collected sound signals into electrical signals, which are received by audio circuit 560 and converted into audio data, which are processed by audio data output processor 580 for transmission to, for example, another cell phone via RF circuit 55, or for output to memory 520 for further processing.
Wi-Fi belongs to a short-distance wireless transmission technology, and a mobile phone can help a user to send and receive e-mails, browse web pages, access streaming media and the like through a Wi-Fi module 570, so that wireless broadband Internet access is provided for the user. Although fig. 4 shows Wi-Fi module 570, it is to be understood that it does not belong to the necessary constitution of the handset, and can be omitted entirely as required within the scope of not changing the essence of the application.
Processor 580 is the control center of the handset, connects the various parts of the entire handset using various interfaces and lines, and performs various functions and processes of the handset by running or executing software programs and/or modules stored in memory 520, and invoking data stored in memory 520, thereby performing overall monitoring of the handset. Optionally, processor 580 may include one or more processing units; preferably, processor 580 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 580.
The handset further includes a power supply 590 (e.g., a battery) for powering the various components, which can be logically connected to the processor 580 by a power management system so as to perform functions such as managing charging, discharging, and power consumption by the power management system.
Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which will not be described herein.
In the embodiment of the present application, the processor 580 included in the mobile phone further has a flowchart for controlling the execution of the container log query method shown in fig. 2.
Fig. 5 is a schematic diagram of a server structure according to an embodiment of the present application, where the server 620 may have a relatively large difference due to different configurations or performances, and may include one or more central processing units (in english: central processing units, in english: CPU) 622 (for example, one or more processors) and a memory 632, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 642 or data 644. Wherein memory 632 and storage medium 630 may be transitory or persistent storage. The program stored on the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Still further, the central processor 622 may be configured to communicate with a storage medium 630 and execute a series of instruction operations in the storage medium 630 on the server 620.
The Server 620 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input/output interfaces 658, and/or one or more operating systems 641, such as Windows Server, mac OS X, unix, linux, freeBSD, and the like.
The steps performed by the server in the above embodiments may be based on the structure of the server 620 shown in fig. 5. The steps of the server shown by fig. 2 in the above embodiments may be based on the server structure shown in fig. 5, for example. For example, the processor 622 performs the following operations by invoking instructions in the memory 632:
the target physical machine receives a log query request from front-end equipment from the cluster gateway, wherein the log query request carries a target container identifier and target period information;
the target service component in the target physical machine determines a target container log path according to the target container identifier and a preset first corresponding relation, and sends the target container log path and the target period information to a target eBPF component, wherein the first corresponding relation comprises a corresponding relation between the container identifier and the log path, and the target eBPF component is the eBPF component in the target physical machine;
The target eBPF component acquires a target container log from a target log file according to the target container log path and the target period information, and stores the target container log into a target eBPF cache space in the target physical machine, wherein the container log of each container in the target physical machine is stored in the target log file;
and the target service component acquires the target container log from the target eBPF cache space and sends the target container log to the front-end equipment through the cluster gateway.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, apparatuses and modules described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.
In the embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.
The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the flow or functions according to the embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
The above description has been made in detail on the technical solutions provided by the embodiments of the present application, and specific examples are applied in the embodiments of the present application to illustrate the principles and implementation manners of the embodiments of the present application, where the above description of the embodiments is only for helping to understand the methods and core ideas of the embodiments of the present application; meanwhile, as for those skilled in the art, according to the idea of the embodiment of the present application, there are various changes in the specific implementation and application scope, and in summary, the present disclosure should not be construed as limiting the embodiment of the present application.

Claims (10)

1. The method is applied to a K8s system, wherein the K8s system comprises a cluster gateway and a plurality of physical machines, the physical machines comprise target physical machines, service components are respectively deployed in user spaces of the physical machines, and eBPF components and eBPF cache spaces are respectively deployed in kernel spaces of the physical machines, and the method comprises the following steps:
the target physical machine receives a log query request from front-end equipment from the cluster gateway, wherein the log query request carries a target container identifier and target period information;
The target service component in the target physical machine determines a target container log path according to the target container identifier and a preset first corresponding relation, and sends the target container log path and the target period information to a target eBPF component, wherein the first corresponding relation comprises a corresponding relation between the container identifier and the log path, and the target eBPF component is the eBPF component in the target physical machine;
the target eBPF component acquires a target container log from a target log file according to the target container log path and the target period information, and stores the target container log into a target eBPF cache space in the target physical machine, wherein the container log of each container in the target physical machine is stored in the target log file;
and the target service component acquires the target container log from the target eBPF cache space and sends the target container log to the front-end equipment through the cluster gateway.
2. The method of claim 1, wherein the target physical machine receiving a log query request from the head-end equipment from the cluster gateway comprises:
The cluster gateway receives the log query request sent by the front-end equipment;
the cluster gateway determines a target physical machine identifier according to the target container identifier and a preset second corresponding relation, wherein the second corresponding relation comprises a corresponding relation between the container identifier and the physical machine identifier;
the cluster gateway sends the log query request to the target physical machine according to the target physical machine identifier; or alternatively, the first and second heat exchangers may be,
the cluster gateway receives the log query request sent by the front-end equipment;
and the cluster gateway sends the log query request to the target physical machine according to the target physical machine identifier carried in the log query request.
3. The method of claim 1, wherein before the target service component in the target physical machine determines a target container log path according to the target container identifier and a preset first correspondence, the method further comprises:
the target service component decodes the log query request according to a preset target decoding rule to acquire the target container identification and the target period information;
after the target service component in the target physical machine obtains the target container log from the target eBPF cache space and before the target container log is sent to the front-end device through the cluster gateway, the method further includes:
And the target server component performs coding operation on the target container log according to a preset target coding rule to obtain a target coding log, and takes the target coding log as the target container log.
4. The method of claim 1, wherein the target eBPF component obtains a target container log from a target log file according to the target container log path and the target period information, comprising:
the target eBPF component determines a target container log subfile from the target log file according to the target container log path;
the target eBPF component obtains the target container log corresponding to the target period information from the target container log subfile.
5. The method of any one of claims 1 to 4, wherein the target eBPF component is mounted on a kernel function of the target physical machine; the target service component sends the target container log path and the target period information to a target eBPF component, the method further comprising:
the target service component starts the target eBPF component by calling the kernel function so as to run the target eBPF component;
The target eBPF component obtains a target container log from a target log file according to the target container log path and the target period information, and stores the target container log into a target eBPF cache space in the target physical machine, and the method further comprises:
and the target service component closes the target eBPF component by calling the kernel function, so that the target eBPF component stops running.
6. The method of claim 5, wherein a target eBPF verifier and a target kernel virtual machine are deployed in kernel space of the target physical machine; before the target service component starts the target eBPF component by calling the kernel function, the method further includes:
the target service component sends target eBPF byte codes preset in the user space to the target eBPF verifier;
the target eBPF verifier performs verification processing on the target eBPF byte codes to obtain verification results, wherein the verification processing comprises infinite loop verification processing, crash code verification processing and running time verification processing;
if the result is that the verification is passed, the target eBPF verifier sends the target eBPF byte code to the target kernel virtual machine;
And a target eBPF compiler in the target kernel virtual machine compiles the target eBPF byte codes to obtain the target eBPF component, and mounts the target eBPF component on a kernel function of the target physical machine.
7. The method of any of claims 1 to 4, wherein prior to the target physical machine receiving a log query request from a head-end from the cluster gateway, the method further comprises:
and the target physical machine acquires a go program package and constructs the target service component in a user space according to the go program package.
8. The K8s system is characterized by comprising a cluster gateway and a plurality of physical machines, wherein the physical machines comprise target physical machines, service components are respectively deployed in user spaces of the physical machines, and eBPF components and eBPF cache spaces are respectively deployed in kernel spaces of the physical machines, wherein the service components are respectively deployed in the kernel spaces of the physical machines:
the target physical machine is used for receiving a log query request from the front-end equipment from the cluster gateway, wherein the log query request carries a target container identifier and target period information;
the target service component in the target physical machine is configured to determine a target container log path according to the target container identifier and a preset first correspondence, and send the target container log path and the target period information to a target eBPF component, where the first correspondence includes a correspondence between the container identifier and the log path, and the target eBPF component is an eBPF component in the target physical machine;
The target eBPF component is used for acquiring a target container log from a target log file according to the target container log path and the target time period information, storing the target container log into a target eBPF cache space in the target physical machine, and storing the container log of each container in the target physical machine in the target log file;
the target service component is further configured to obtain the target container log from the target eBPF cache space, and send the target container log to the front-end device through the trunking gateway.
9. A computer device, characterized in that it comprises a memory on which a computer program is stored and a processor which, when executing the computer program, implements the method according to any of claims 1-7.
10. A computer readable storage medium, characterized in that the storage medium stores a computer program comprising program instructions which, when executed by a processor, can implement the method of any of claims 1-7.
CN202311083647.0A 2023-08-25 2023-08-25 Container log query method, related equipment and storage medium Pending CN117056387A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311083647.0A CN117056387A (en) 2023-08-25 2023-08-25 Container log query method, related equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311083647.0A CN117056387A (en) 2023-08-25 2023-08-25 Container log query method, related equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117056387A true CN117056387A (en) 2023-11-14

Family

ID=88655160

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311083647.0A Pending CN117056387A (en) 2023-08-25 2023-08-25 Container log query method, related equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117056387A (en)

Similar Documents

Publication Publication Date Title
CN111416845B (en) Method, device and storage medium for configuring edge equipment
CN107329985B (en) Page collection method and device and mobile terminal
US20180196703A1 (en) Parallel Execution Method and Apparatus in Terminal Database
CN105824958A (en) Method, device and system for inquiring log
CN115658348B (en) Micro-service calling method, related device and storage medium
CN111125269A (en) Data management method, blood relationship display method and related device
CN114694226B (en) Face recognition method, system and storage medium
CN108984374B (en) Method and system for testing database performance
CN104281610A (en) Method and device for filtering microblogs
CN114115895A (en) Code query method and device, electronic equipment and storage medium
CN116758362A (en) Image processing method, device, computer equipment and storage medium
CN116071614A (en) Sample data processing method, related device and storage medium
CN115546516A (en) Personnel gathering method and device, computer equipment and storage medium
CN115640567A (en) TEE integrity authentication method, device, system and storage medium
CN116563041A (en) Data reconciliation method and related device
CN117056387A (en) Container log query method, related equipment and storage medium
CN114691714A (en) Data acquisition method, device, equipment and medium based on SQL statement integration
CN115525554B (en) Automatic test method, system and storage medium for model
CN114490307A (en) Unit testing method, device and storage medium
CN115412726B (en) Video authenticity detection method, device and storage medium
CN114722970B (en) Multimedia detection method, device and storage medium
CN115909186B (en) Image information identification method, device, computer equipment and storage medium
CN116756350A (en) Multimedia data processing method, device, computer equipment and storage medium
CN115565215B (en) Face recognition algorithm switching method and device and storage medium
CN116954631A (en) Model deployment method, related device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination