CN111786973B

CN111786973B - Stream log acquisition method, device, equipment and storage medium

Info

Publication number: CN111786973B
Application number: CN202010568198.9A
Authority: CN
Inventors: 雷思源; 孙小强; 周磊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-06-19
Filing date: 2020-06-19
Publication date: 2022-09-23
Anticipated expiration: 2040-06-19
Also published as: CN111786973A

Abstract

The application discloses a method, a device, equipment and a storage medium for collecting a stream log, and relates to the cloud platform technology. The specific implementation scheme is as follows: the method is applied to the nodes of the software defined network, and comprises the following steps: acquiring a log flow table through a virtual switch in the node, and configuring the log flow table in at least one of a message incoming path and a message outgoing path; when the virtual switch hits and matches the message according to the log flow table, forwarding the hit message to a log processing component according to an execution action item in the log flow table; generating a flow log according to the hit message through a log processing component in the node; and reporting the stream log to a log server through the log processing component. The technology of the application solves the problem that the flow monitoring function in the software defined network in the prior art influences the network performance.

Description

Stream log acquisition method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a cloud platform technology.

Background

Virtual Private Cloud (VPC) is a dynamic configuration pool of public Cloud computing resources, and requires using encryption protocols, tunneling protocols, and other security programs to transmit data between a Private enterprise and a Cloud service provider. One VPC basically changes the multi-tenant architecture of the provider into a single-tenant architecture.

For VPC users, knowledge of the traffic in the VPC is required, and there is a need for basic network problem location, traffic monitoring and presentation.

In the related technology, the utilization rate of cloud product resources, the performance of application programs, the running condition of cloud products, multi-index monitoring, custom warning and the like are known through cloud monitoring. However, the above scheme needs to add a traffic check point, which may cause performance degradation of the cloud host, and an intrusive monitoring mode is adopted for normal traffic forwarding, so that there is an influence on normal services.

Disclosure of Invention

The embodiment of the application provides a method, a device, equipment and a storage medium for collecting a stream log.

According to an aspect of the embodiments of the present application, a method for collecting a flow log is provided, which is applied to a node of a software-defined network, and the method includes:

acquiring a log flow table through a virtual switch in the node, and configuring the log flow table in at least one of a message incoming path and a message outgoing path;

when the virtual switch hits and matches the message according to the log flow table, forwarding the hit message to a log processing component according to an execution action item in the log flow table;

generating a flow log according to the hit message through a log processing component in the node;

and reporting the stream log to a log server through the log processing component.

According to another aspect of the embodiments of the present application, there is provided a flow log collecting apparatus configured in a node of a software defined network, the apparatus including:

the flow table configuration module is configured in the virtual switch in the node, is used for acquiring a log flow table, and is configured in at least one of a message incoming path and a message outgoing path;

the message hit module is configured in the virtual switch and used for forwarding a hit message to the log processing component according to an execution action item in the log flow table when the message is matched according to the hit of the log flow table;

the log processing component is configured in the node and used for generating a flow log according to the hit message;

the log processing component is further used for reporting the stream log to a log server side.

According to another aspect of the embodiments of the present application, there is provided an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the flow log collection method provided by any embodiment of the application.

According to another aspect of embodiments of the present application, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method for collecting a stream log provided in any of the embodiments of the present application.

The technology of the application solves the problem that the flow monitoring function in the software defined network in the prior art influences the network performance.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

fig. 1A is a flowchart of a flow log collection method according to an embodiment of the present application;

FIG. 1B is a block diagram of a computing node according to an embodiment of the present disclosure;

fig. 2A is a flowchart of a flow log collection method according to an embodiment of the present application;

FIG. 2B is a block diagram of a computing node according to an embodiment of the present disclosure;

fig. 3A is a schematic diagram of a message path applied in the embodiment of the present application;

fig. 3B is a schematic diagram of an IPv4 message applied in the embodiment of the present application;

fig. 3C is a schematic diagram of an IPv6 message applied in the embodiment of the present application;

fig. 4A is a flowchart of a flow log collection method according to an embodiment of the present application;

FIG. 4B is a block diagram of a computing node according to an embodiment of the present disclosure;

FIG. 5 is a block diagram of a flow log collection apparatus according to an embodiment of the present application;

fig. 6 is a block diagram of an electronic device for implementing a flow log collection method according to an embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The technical scheme of the embodiment of the application is suitable for a Software Defined Network (SDN), the SDN technology is an implementation mode of Network virtualization, and the core technology OpenFlow separates a control surface and a data surface of Network equipment, so that the flexible control of Network flow is realized, the Network becomes more intelligent as a pipeline, and a good platform is provided for innovation of the core Network and application.

The cloud platform can be realized based on the SDN technology, and nodes realized based on the SDN technology are deployed in the cloud platform and provide network services for users together. The cloud platform may be, for example, a VPC, which may provide different independent VPCs for multiple users. The VPC may be deployed in one node or in multiple nodes. The types of nodes implemented based on SDN technology may include a variety, for example, an SDN implemented based on the OpenStack protocol may include a computing node, a control node, and a storage node, wherein the computing node is a node mainly carrying a virtual machine, a virtual switch (e.g., BVS) to complete service processing and packet forwarding functions.

In the embodiment of the application, the main object of flow log monitoring is the message flow forwarded by the virtual switch. Thus, the flow log collection scheme may be implemented by nodes of the SDN, and mainly nodes carrying virtual switches. In the embodiment of the present application, a virtual switch is borne in a Compute Node (CN) as an example for description, but a person skilled in the art should understand that other types of nodes bearing a virtual switch are also applicable to the technical solution of the embodiment of the present application.

Taking a computing node as an example, one computing node may carry at least one virtual machine, and message interaction between virtual machines inside the computing node is realized through a virtual switch between the virtual machines. For a VPC or a cloud platform, multiple computing nodes may be deployed, so the virtual switch is also used to implement message interaction between virtual machines of different computing nodes.

To facilitate network management, for a VPC, a user may selectively determine that its VPC includes one or more virtual subnets (Subnet), each virtual Subnet including one or more Virtual Machines (VMs), deployed on one or more computing nodes. Each virtual machine carries out message interaction through one or more virtual network cards configured by the virtual machine. The finest granularity of message interaction is in units of virtual network cards.

In the virtual switch, the flow direction of the message is controlled through a plurality of flow tables, specifically, the message can be matched by a plurality of flow tables in a message incoming path and a message outgoing path respectively; the flow table at least comprises a message matching item and an execution action item, and if the message is successfully matched with the message matching item in a certain flow table, the message is a hit message of the flow table; the virtual switch executes an action (action) on the hit message in accordance with the value of the execution action item set in the flow table. The execution action item generally specifies which network card port the packet needs to be forwarded to, which flow table to continue matching, or to drop, etc.

The technical solution of the embodiment of the present application is implemented based on the SDN node, and is specifically described below by an embodiment.

Fig. 1A is a flowchart of a flow log collection method provided in an embodiment of the present application, and the embodiment is suitable for a node of an SDN to monitor packet traffic processed by a virtual switch in a node so as to collect a situation of forming a flow log. The present embodiment is implemented by a flow log collecting apparatus, which may be implemented by software and/or hardware and is configured in a computing device deployed with a node, where the computing device is generally a server or a service cluster capable of bearing a large amount of computing tasks. Fig. 1B is a schematic structural diagram of a computing node applicable to the embodiment of the present application, which is described with reference to fig. 1A and 1B, and the method includes:

s110, acquiring a log flow table through a virtual switch in the node, and configuring the log flow table in at least one of a message incoming path and a message outgoing path;

as shown in fig. 1B, in this embodiment, the virtual switch (BVS)11 in the computing node 10 may obtain a log flow table, where the log flow table may be preset, or may be dynamically configured by the user to the virtual switch 11 according to the flow log collection requirement. The log flow table is similar to the data content and format of the other flow tables and may be configured in at least one of the message ingress path and the message egress path similarly to the other flow tables.

S120, when the virtual switch hits and matches the message according to the log flow table, forwarding the hit message to a log processing component according to an execution action item in the log flow table;

the virtual switch 11 controls the forwarding action of the received incoming packet and the outgoing packet to be sent based on the flow tables in the packet incoming path and the packet outgoing path, respectively. The incoming message and the flow table in the message incoming path are matched one by one according to the flow table sequence, and once the flow table hits the message, the virtual switch 11 processes the message according to the execution action item of the flow table. The outgoing message and the flow table in the message outgoing path are matched one by one according to the flow table sequence, and once the flow table hits the message, the virtual switch 11 processes the message according to the execution action item of the flow table.

In this process, if the log flow table hits the packet, the virtual switch 11 forwards the hit packet to the log processing component 12 according to the execution action item in the log flow table. The action-performing item of the log flow table may specify that the packet is forwarded to a specific port, which corresponds to a port monitored by the subsequent log processing component 12, so that the log processing component 12 can collect the hit packet.

S130, generating a flow log according to the hit message through a log processing component in the node;

in this operation, the log processing component 12 may further generate a flow log based on a plurality of sequentially hit messages, and specifically, the hit messages may be subjected to statistics, clustering, and other processing according to a reporting requirement of the flow log.

And S140, reporting the stream log to a log server through the log processing component.

In this operation, the log processing component 12 may report the stream log to the log server 13 periodically or as required. The logging service 13 may store the flow logs and may provide users with the ability to retrieve, query, and visualize the flow logs.

According to the technical scheme of the embodiment of the application, the flow table is configured in at least one of the message incoming path and the message outgoing path of the virtual switch, so that the virtual switch can collect messages hitting the log flow table in the normal message processing process, and a flow log is formed. The flow log collection process does not need an additional log collection instruction, has no influence on normal service processing of the virtual switch, and can realize non-invasive flow log collection. Thus, traffic statistics flowing into or out of a particular virtual machine can be provided to the user as desired.

Fig. 2A is a flowchart of a flow log collection method according to an embodiment of the present disclosure. The present embodiment specifically introduces a flow table configuration manner based on the foregoing embodiments. Fig. 2B is a schematic diagram illustrating an architecture of a computing node to which the embodiment of the present invention is applied, and as shown in fig. 2B, at least one virtual switch 21, a virtual switch agent (ovs agent)22, and a log processing component (also referred to as a flow log agent)23 are configured in the computing node 20. Each virtual switch 21 may aggregate the message ingress path and the message egress path to form a message path (Datapath). The externally transmitted message may be received through an interface of a cloud Server (Neutron Server)24 and distributed to the respective computing nodes 20.

The configuration of the flow table may be implemented by the cloud server 24 and the virtual switch agent 22, which is specifically as follows:

s210, obtaining a message from an interface of a cloud service end through a virtual switch agent end of the node;

s220, if the acquired message is identified to be a log flow table configuration message, generating a log flow table according to a flow log acquisition rule stored in a database of the cloud service end through the virtual switch agent end, wherein the log flow table configuration message is generated by the cloud service end according to the flow log acquisition rule;

the stream log collection rule is generally determined according to the log acquisition requirement of the user, and may be a common collection rule generated by default, or a stream log collection rule determined according to the user-defined requirement. Specifically, the user may send a stream log configuration request to a cloud Server (Neutron Server)24, where the request is used to determine the stream log collection rules. The flow log collection rule may be a rule to add, delete, or alter flow log collection. The stream log collection rule specifically indicates one or more of a message, a network card, a virtual machine, a subnet and a VPC which are required to be used as a monitoring target. The destination may be specified by setting conditions, such as specifying a network card address, a virtual machine identifier, a source address or a destination address of the message, and the like. The stream log collection rule may also specify the content and format of the stream log to be reported to the log server 28, and the path to be stored in the log server 28.

The cloud server 24 obtains a flow log configuration rule according to a flow log configuration request of a user, the flow log configuration request as user data may be stored in a first Database (DB)25 corresponding to the cloud server 24, and the flow log configuration rule as control data may be stored in a second database (etc) 26 corresponding to the cloud server 24. Specifically, the cloud server 24 may be additionally provided with a flow log application interface (flow API)24a and a flow log plug-in (flow plug-in) 24b, where the flow log application interface 24a is used to distribute flow log messages, and the flow log plug-in 24b is used to process flow log related messages.

The cloud server 24 generates a log flow table configuration message according to the flow log collection rule, and sends the log flow table configuration message to the virtual switch agent 22 through an interface (nonrethod) and a second database (ETCD) 26. Of course, the cloud server 24 will also send other messages to be processed to the virtual switch agent 22.

The message enters the message queue of the virtual switch agent 22 and is processed according to the preset priority of the message. When the virtual switch agent 22 recognizes that the message is a log flow table configuration message, the flow log collection rule may be read from the second database 26 of the cloud server 24, and a log flow table may be generated according to the read flow log collection rule.

S230, configuring the log flow table to the virtual switch through the virtual switch proxy end;

s240, configuring the log flow table in at least one of a message incoming path and a message outgoing path through the virtual switch;

the virtual switch proxy 22 may specifically configure the log flow table into the message path of the virtual switch 21 through a component (ovs-vswitch) of the virtual switch proxy 22. Meanwhile, the log stream table can also be stored in the corresponding agent database (ovs-db server) 27. Thereby, the virtual switch agent 22 realizes the configuration of the flow table.

The virtual switch agent 22 may also configure a port of the virtual switch 21 for exporting the hit packet to the log processing component 23, specifically, the port may be configured according to the port monitored by the log processing component 23.

To support the configuration function of the log flow table, a flow log extension (flow log extension) may be added to the virtual switch agent 22.

S250, when the virtual switch hits and matches the message according to the log flow table, forwarding the hit message to a log processing component according to an execution action item in the log flow table;

s260, generating a flow log according to the hit message through a log processing component in the node;

and S270, reporting the stream log to a log server through the log processing component.

According to the technical scheme, the log flow table configuration is realized through the virtual switch agent end in the node, the user is allowed to configure the flow log acquisition rule through the cloud service end, no additional flow log acquisition instruction is needed, and the user can monitor the flow in full time, full flow and non-invasion according to the requirement. The configuration process of the flow log acquisition rule can be realized by utilizing the message processing and flow table configuration functions of the virtual switch agent end, so that the existing node operation system is non-invasive, and the normal function operation of the node is not influenced.

Fig. 3A is a schematic diagram of a message path applied in the embodiment of the present application. Based on the foregoing embodiment, this embodiment further provides a specific implementation manner for configuring the flow table in the packet path. In this embodiment, the virtual switch processes the packet traffic through a virtual bridge, for example, in the virtual switch, the virtual bridge based on the Openstack protocol includes an internal bridge (br-int) and an external bridge (br-tun), which are respectively responsible for receiving a packet and sending a packet. The virtual bridge matches the messages with the message matching items in the flow tables one by one according to the sequence of the flow table or the flow tables, and executes the specified action according to the execution action item after the matching is hit. The configuration purposes of each flow table are different, and different message forwarding functions can be realized.

The virtual switch optionally integrates all message flows (soft flow) into one message path (Datapath), as shown in fig. 3A. While a plurality of flow tables are configured in br-int and br-tun, it should be understood by those skilled in the art that each flow table and sequence shown in fig. 3A are only examples, and the flow tables and the sequence of the flow tables may be specifically set according to actual needs of users and functions of nodes. The br-int and the br-tun respectively comprise a message incoming path and a message outgoing path.

For the outgoing path of the message, the message passes through the SG communication interface in the br-int and then passes through the firewall in the br-tun, so that a log flow table is added to the outgoing path of the message of the br-int, as shown in fig. 3A, and table 51 is the log flow table. The execution action item entered into the log flow table is specifically configured as "flow, respmit (62)", that is, the hit message is forwarded to the acquisition port of the log processing component, and is retransmitted to the table 62 for further processing.

For the incoming path of the message, the message firstly passes through the firewall in the br-tun and then passes through the SG communication interface in the br-int, so that an incoming log flow table is added to the incoming path of the message in the br-tun, as shown in fig. 3A, table 6 is the incoming log flow table, an execution action item of the outgoing log flow table is specifically configured as "flow, resume (, 7)", the message to be hit is forwarded to the acquisition port of the log processing component, and is retransmitted to table 7 for continuous processing.

By the technical scheme of the embodiment, the log flow table can be reasonably configured in the path of the message, so that full-flow and full-time monitoring of the message is realized, and omission is avoided.

On the basis of the foregoing embodiment, in order to support the flow logging function, the virtual switch needs to support the following functions:

adding execution action items (flow action) of log message processing: the execution action item of the log message processing is the same as the processing mode of other execution action items, and the flow can be added to the action of the flow table. For example, the flow table is:

ovs-opposite add-flow br-tun 'table 10, priority 2, dl _ vlan 2, tcp, nw _ src 192.168.0.11, nw _ dst 192.168.0.10, tp _ src 80actions flow, resume (,11)', indicating that two actions are included in the action.

After the action configuration, the log message processing action in the flow table acts on a message flow (dp flow), when a message hits the flow table, the virtual switch extracts information of the hit message and sends the information in a unified manner through a User Datagram Protocol (UDP), the virtual switch may send meta information of the hit message to a log processing component by using the UDP, and a message format is shown in fig. 3B and 3C. The message can be sent to a configurable port of the machine, and the log processing component can monitor the port so as to collect the hit message. That is, the virtual switch may forward the meta information of the hit packet to the set port corresponding to the log processing component based on the user datagram protocol according to the execution action item in the log flow table. And the meta information of the hit message is sent, so that the data transmission quantity forwarded to the set port by the virtual switch can be reduced, the data acquisition quantity and the data processing quantity of the log processing component can be reduced to a certain extent, and only the key meta information required by the flow log is processed.

The specific fields of the forwarded message of the virtual switch can be defined as shown in table 1 below:

TABLE 1

The length of the IPv4 message meta-information is as follows: 56 bytes, one message 25batch is sent. Length of IPv6 message meta information: 80 bytes, a message 17batch is sent.

Fig. 4A is a flowchart of a flow log collection method according to an embodiment of the present application. The present embodiment further provides a specific implementation manner of stream log collection based on the foregoing embodiments. Fig. 4B is a schematic diagram illustrating an architecture of a computing node to which the embodiment of the present application is applied, and as shown in fig. 4B, the log processing component 23 provided in the computing node 20 may be implemented by an independent process. The functions of generating and reporting the stream log implemented by the log processing component 23 can be implemented by sub-components respectively. As shown in fig. 4A, the method of the present embodiment includes:

s410, acquiring a log flow table through a virtual switch in the node, and configuring the log flow table in at least one of a message incoming path and a message outgoing path;

s420, when the virtual switch hits and matches the message according to the log flow table, forwarding the hit message to a log processing component according to an execution action item in the log flow table;

s430, acquiring a flow log acquisition template determined based on a flow log acquisition rule through a synchronization sub-assembly in the log processing assembly;

the flow log collection rule can determine a flow log collection template, which specifies which field information the flow log should collect from the message, and can specify how to perform statistical combing on the field information, and according to what format to generate the flow log, so as to meet the requirement of the user for viewing the flow log. The log processing component 23 may be configured with a flow log collection rule in advance, and preferably, in combination with the foregoing embodiment, the synchronization subcomponent of the log processing component 23 may obtain the log collection rule from a location where the flow log collection rule is arbitrarily stored, such as the database 27 of the virtual switch agent 22 or the second database 26 of the cloud service end 24.

S440, acquiring and obtaining the hit message forwarded by the virtual switch from a set port through an acquisition subassembly in the log processing assembly;

the virtual switch 21 forwards the hit packet to the log processing component 23, which may specifically be controlled by an execution action item of the flow table, and forwards the hit packet to a set port, where an acquisition subcomponent (collector)231 in the log processing component 23 is responsible for monitoring and receiving the packet, that is, the hit packet is acquired from the set port.

S450, extracting information from the acquired hit message according to the flow log acquisition template through a formatting sub-assembly in the log processing assembly, and filling the extracted information into the flow log acquisition template to form a flow log.

In this operation, the formatting sub-component (formatter) 232 further integrates the information extracted from the hit message based on the specification of the flow log collection template, and then fills the flow log collection template according to the template requirement to form a flow log with the specific information content of the message. The synchronization subcomponent may be provided integrally with the formatting subcomponent 232.

And S460, reporting the stream log to a log server through a derivation subassembly in the log processing assembly.

The export subcomponent (exporter)233 may report the flow log to the log server according to a log storage path specified in the flow log collection rule, and may specifically store the flow log in a specified path space.

As described above, the log processing component may include four subcomponents, namely the synchronization subcomponent, the acquisition subcomponent, the formatting subcomponent and the export subcomponent. The log processing component may be implemented by a process, and the synchronization subcomponent, the acquisition subcomponent, the formatting subcomponent and the export subcomponent may be implemented by separate threads in the process. Alternatively, the synchronization subcomponent may execute using the same thread as the formatting subcomponent 232.

Therefore, each function of generating and reporting the stream log can be realized in parallel by adopting an independent thread, and the processing efficiency of the stream log is ensured.

Specifically, the log processing component is implemented in the form of a flow log agent (flow agent), and after the flow log agent is started to run, the following 4 tasks are started:

1) synchronization subcomponent (cache sync thread, bthread): the method comprises the steps of periodically removing ovsdb and synchronizing a flow log collection template in ETCD, wherein the flow log collection template comprises information such as MAC (media access address), port, subnet, VPC (virtual private network), flow log instance ID (identity) and the like, and the synchronization period is preferably smaller than the flow log export period so that the exported flow log meets the latest flow log collection requirement.

2) Collection subcomponent (collector thread): monitoring a set port, collecting hit message data sent by the BVS, clustering the messages, and periodically scanning. Adding the collected message into a formatting queue (format queue);

3) format sub-component (formatter thread): reading message data from the formatting queue, backfilling fields of the flow logs according to a flow log acquisition template acquired by the synchronization sub-component, and writing the flow logs into an export queue (export queue) after backfilling;

4) export subcomponent (exporter thread): and reading the stream logs from the export queue, finding a corresponding log storage path according to a stream log acquisition rule, and pushing (push) the log storage path to a log server.

The log processing component can also realize statistics of message arrival flow, maintain expiration time and the like, and specifically can record the time of each message through a hash table and maintain regularly.

The log processing component has the main function of forming the flow log based on the message information from the virtual switch and combining the control plane information to finally fill in the relevant fields according to the template format requirement of the flow log.

For a flow log, determining, by a log processing component in a node, at least one item of information from the hit packet: quintuple, packet number, byte number, timestamp, media access address (mac), traffic direction (i.e. incoming or outgoing direction of the packet, IN/OUT), traffic action (specifically including ACCEPT or REJECT, ACCEPT/REJECT), stream log version, PORT identifier (PORT _ ID), virtual machine identifier (VM _ ID), SUBNET identifier (SUBNET _ ID), virtual private cloud identifier (VPC _ ID), tenant identifier, and stream log record status (which may include correct or ERROR, OK/ERROR). The above information is preferably determined as fields added to the flow log collection template to facilitate the user to know the flow situation from various dimensions.

In addition, the data plane of the log processing component needs to be able to know which specific log storage path a certain flow log should be sent to the log server.

In the above information fields, information of five tuples, byte number, message number, timestamp and flow action (ACCEPT/REJECT) of the flow exists in the message meta information derived by the virtual switch, and the log processing component can directly extract the information.

Three aspects of data that need to be additionally determined are: the direction of flow; short IDs for port, vm, subnet, and vpc; and, a log storage path at the time of export. The determination modes are described below respectively.

First, determining, by a log processing component in the node, a traffic direction according to the hit packet may specifically include: and determining the flow direction of the hit message as an incoming direction or an outgoing direction according to the value-taking rules of the incoming port and the outgoing port of the hit message by the log processing component. The concrete description is as follows:

when determining the traffic direction of the message, the log processing component may determine by using a message ingress port (in _ port) and an egress port (out _ port) in the message forwarded by the virtual switch.

For example, the value rules of in _ port and out _ port in a virtual switch are shown in table 2 below:

TABLE 2

The value rules of the ingress port and the egress port can be specifically summarized as follows:

1. to identify this behavior when a message is dropped (drop) by a security group or firewall, the virtual switch will set out port to a set drop value, e.g., "0", and the set drop value (0) will not actually send or receive messages in the configuration of the virtual switch in the message path port. So if the output port is the set discard value, it indicates that the hit message is discarded. That is, determining, by the log processing component in the node, a traffic action according to the hit packet may specifically include: and if the output port value of the hit message is a set discarding value, the flow action of the hit message is discarding.

2. Ingress and egress across a Compute Node (CN) may be directly distinguished by ingress and egress ports.

2.1, for the case of crossing CN, the message flow direction can be directly determined based on in _ port and out _ port:

for the computing node, the host network card of the computing node is connected with the virtual network cards of the virtual machines through the network bridge, and the network cards of the virtual machines can also be connected through the network bridge. The ports between the virtual machine bridges are internal ports, taking values of, for example, qvo. The host network card of the computing node is uniformly provided with a flow inlet and a flow outlet, wherein the flow inlet is used for receiving the flow outside the computing node, and the flow outlet is used for transmitting the internal flow of the computing node outwards. Based on the above-mentioned rule, the method,

if the value of the in _ port is a traffic inlet (such as vxlan _ sys _4789) of the computing node, determining that the traffic flow direction of the message is an inlet direction crossing the CN;

if the value of out _ port is the traffic outlet of the compute node (e.g., dpdk0), it is determined that the traffic flow of the packet is the outgoing direction across CNs.

The flow action of the hit message can be correspondingly recorded as ACCEPT or REJECT when the virtual switch performs flow hit, so that the virtual switch can provide the record to the log processing component.

The log processing component can determine that the flow action of the cross-CN incoming message can be ACCEPT or REJECT flow through the hit message information provided by the virtual switch. The traffic movement across the CN outbound message may also be determined as ACCEPT.

2.2 for the same CN, forwarding is performed between virtual machines inside the compute node. For example, if the port value of the internal bridge of the compute node is vpo, when the virtual machine switch hits the message in the same CN, the virtual machine switch can directly determine the traffic direction and the traffic action of the message, then record the information, and notify the log processing component. And the log processing component can directly determine that the flow direction of the hit message is the outgoing direction according to the notification of the virtual switch.

Because of the message traffic of the same CN, no matter whether the traffic action is ACCEPT or REJECT, the message is only hit by the outbound log flow table, and the message is recorded as the outbound message of the source virtual machine. Because it is uncertain whether the target virtual machine of the packet is also configured with a log stream table to capture the incoming packet, optionally, according to the same CN packet hit by the outgoing log stream table, a stream log in the opposite direction is generated based on the destination virtual machine address of the same CN packet, that is, the incoming stream log is directly generated, or information of the incoming stream log can be generated. It is equivalent to "reverse" copying a flow log destined for a VM on the same CN, thereby effectively covering the same CN's incoming ACCEPT and REJECT. For example, the virtual machine a sends a message to the virtual machine B, and the message hits the log traffic in the message outbound path of the virtual machine a, so that the flow log of the message outbound from the virtual machine a can be generated. And the log flow table in the message incoming path of the virtual machine B cannot be hit, so that the log cannot be embodied in the incoming message flow log of the virtual machine B. For this purpose, information representing the incoming packet of the virtual machine B may be generated based on the outgoing packet of the virtual machine a, thereby generating a flow log of the incoming packet for the virtual machine B.

The above-mentioned traffic flow direction and the determination action of the traffic action may be implemented by a format thread at the flow log agent side.

In summary, the determining, by the log processing component in the node, a traffic direction according to the hit packet may specifically include:

and determining the flow direction of the hit message as an incoming direction or an outgoing direction according to the value-taking rules of the incoming port and the outgoing port of the hit message by the log processing component. This approach is particularly applicable to the cross-CN case.

Or, determining the flow direction of the hit message through the virtual switch according to the internal bridge port of the computing node in the hit message, and notifying the log processing component; and determining the flow direction of the hit message as an outgoing direction according to the notification of the virtual switch by the log processing component. This approach is particularly applicable to the same CN case.

Optionally, determining, by the log processing component, that the flow direction of the hit packet is an ingress direction or an egress direction according to the ingress port and egress port value-taking rule of the hit packet specifically includes:

if the input port value of the hit message points to the flow inlet of the node, determining the flow direction of the hit message as the input direction through the log processing component;

and if the output port value of the hit message points to the flow outlet of the node, determining the flow direction of the hit message as the output direction through the log processing component.

Optionally, determining, by the log processing component in the node, a flow action according to the hit packet includes:

and if the output port value of the hit message is a set discard value, determining that the flow action of the hit message is discard through the log processing component.

Optionally, after determining that the flow direction of the hit packet is an outgoing direction according to the notification of the virtual switch by the log processing component, the method further includes:

and reversely generating information for determining the incoming flow log according to the outgoing hit message through the log processing component.

Second, in general, the port, virtual machine, subnet, and virtual private cloud are provided with long identity and short identity, respectively. The long identification is operated when the node runs in the background and is a longer character string; the short identifier is presented to the user as a relatively short string of characters. The long mark and the short mark have corresponding relation and can play a role in marking. The hit message information provided by the virtual switch carries a long identifier, and the flow log is viewed by the user, so that the long identifier needs to be converted into a short identifier.

Determining, by a log processing component in the node, at least one of a port identifier, a virtual machine identifier, a subnet identifier, and a virtual private cloud identifier according to the hit packet includes:

reading the corresponding relation between the long identification and the short identification of at least one of a port, a virtual machine, a subnet and a virtual private cloud from a database of a cloud server through the log processing component;

and extracting a long identifier of at least one of the port, the virtual machine, the subnet and the virtual private cloud according to the hit message through the log processing component, and determining a short identifier corresponding to the long identifier according to the corresponding relation to be used as at least one of the port identifier, the virtual machine identifier, the subnet identifier and the virtual private cloud identifier.

Thereby, it is possible to make short identifications easily recognizable to the user presented in the stream log.

The above process is specifically described as follows:

the short IDs of the port, vm, subnet and VPC related in the flow log only have long ID information in the control plane, so an entity needs to be provided in neutron to obtain the long ID and the short ID, and let the flow log proxy side perceive them.

The mapping correspondence between the long ID and the short ID is actually stored in the bce _ local DB of the console, in the database of the cloud server.

When the flow log agent end processes, the long ID of port, instance and subnet vpc can be found back through the media access address, the short ID can be found through the corresponding relation of local cache, and if the short ID fails, the ETCD can be found and cached to the local.

Thirdly, the log processing component determines the mode of the log storage path (log _ store) of the stream log reported to the log server. Before the stream log is reported to the log server side through the log processing component, the method further comprises the following steps:

determining a log storage path reported to a log server by the log processing component according to the configuration granularity in the stream log acquisition rule;

wherein a granularity level of the configuration granularity comprises at least one of: port granularity, device granularity, subnet granularity, virtual private cloud granularity, and tenant granularity.

Through setting different configuration granularities, a user can know the flow conditions from different granularities conveniently. And the stream logs with different configuration granularities are directly stored into different storage paths of a log server, so that the management and the query are easy.

The specific introduction is as follows:

the configuration granularity of the flow log may be various, and specifically, the configuration granularity may be the granularity of different levels, such as a port (port), a virtual network card (device), a subnet (subnet), a vpc (vpc), and a tenant (tenant). For example, a stream log of a subnet granularity needs to summarize all message information that conforms to a stream log collection rule in units of subnets.

As introduced in the foregoing embodiment, the granularity of the hit packet is actually port granularity, and does not differentiate between other configuration granularities, and does not directly form a flow log of other configuration granularities. Therefore, when exporting the flow log, the flow log agent needs to determine how to export the flow log to each log storage path corresponding to the configuration granularity according to the flow log collection rule. Different log storage paths can differentiate different configuration granularities.

If the log collection rule includes four configuration granularities, vpc1 → subnet1 → device1 → port1, the flow log agent performs secondary summary statistics on the flow logs determined based on the port granularities according to the configuration granularity of the log collection rule to form flow logs of other granularities, and then the flow logs are respectively exported to corresponding log storage paths:

the flow log for port1 is sent to log _ store1

Stream Log for device1 is sent to log _ store2

The stream logs of subnet1 are sent to log _ store3

The stream logs for vpc1 are sent to log _ store4

In this case, only the mac of port1 is actually included in the flow table, and the action is the flow table of flow.

Through the technical scheme of the embodiment of the application, the following stream log collection function can be preferably realized:

according to user configuration, the stream log collection function should collect the flow information flowing into/out of the user specific network card, virtual machine, subnet and VPC with the network card as the minimum granularity, and push the flow information to the log server after preprocessing and aggregation.

The flow information is collected, transmitted, stored and displayed in a log mode, each flow log can take quintuple (source IP, destination IP, protocol, source port number and destination port number), message number, byte number, collection state and flow action (allowing flow or rejecting flow) as core information, and also comprises collection starting time and necessary meta information, such as port _ id, VM _ id, subnet _ id, VPC _ id, tenant identification (tenant _ id) and the like.

The log server can store the stream log and provide retrieval and display in a time dimension.

The technical scheme of the embodiment of the application actually relates to three main parts of configuration, collection and derivation of the stream log:

the log processing component is also called as a flow acquisition component: the specific flow acquisition function is realized, and the packet → flow statistical conversion is realized;

cloud services, such as Neutron components: receiving a flow log configuration request of a user, and calling a bottom-layer log processing component to realize flow log collection, in particular to Neutron Server and ovs-agent (vnet controller).

The flow preprocessing and exporting component, the format sub-component and the exporting sub-component in the log processing component: according to the collected information, adding meta information such as port _ id, VM id, subnet id, network id and the like to each flow log. And finally, calling an application interface of the log server to export the stream log to the log server.

The log server is responsible for establishing indexes for the keywords of the stream log, storing the keywords and providing a search display function. The flow log can provide the user with capabilities including, but not limited to: checking whether a certain flow reaches a specific machine; checking whether the security group or the firewall rule is effective or whether the firewall rule is mistakenly killed; flow monitoring, display or abnormal alarm; and performing targeted network optimization according to the flow characteristics.

FIG. 5 is a block diagram of a flow log collection apparatus according to an embodiment of the present application; as shown in fig. 5, the flow log collecting apparatus 500 is configured in a node of a software defined network, and includes:

a flow table configuration module 510 configured in the virtual switch in the node, configured to obtain a log flow table, and configured in at least one of a packet ingress path and a packet egress path;

a message hit module 520 configured in the virtual switch, and configured to forward a hit message to the log processing component according to an execution action item in the log flow table when the message is hit and matched according to the log flow table;

a log processing component 530 configured in the node, configured to generate a stream log according to the hit packet;

the log processing component 530 is further configured to report the stream log to a log server.

Optionally, the apparatus further comprises:

the message acquisition module is configured at the proxy end of the virtual switch and used for acquiring messages from an interface of the cloud service end;

the flow table generating module is configured at the virtual switch agent end and used for generating a log flow table according to a flow log acquisition rule stored in a database of the cloud service end if the acquired message is identified to be the log flow table configuration message, wherein the log flow table configuration message is generated by the cloud service end according to the flow log acquisition rule;

the flow table providing module is configured at the proxy end of the virtual switch and used for configuring the log flow table to the virtual switch;

correspondingly, the flow table configuration module is specifically configured to configure the log flow table in at least one of a message ingress path and a message egress path.

Optionally, the log processing component includes:

the synchronization subassembly is used for acquiring a flow log acquisition template determined based on a flow log acquisition rule;

the acquisition subassembly is used for acquiring the hit message forwarded by the virtual switch from a set port;

and the formatting subassembly is used for extracting information from the acquired hit message according to the flow log acquisition template and filling the extracted information into the flow log acquisition template to form a flow log.

Optionally, the log processing component further includes:

and the export subassembly is used for reporting the stream log to a log server.

Optionally, the log processing component is implemented by a process, and the synchronization sub-component, the acquisition sub-component, the formatting sub-component, and the derivation sub-component are implemented by independent threads in the process, respectively.

Optionally, the log processing component is specifically configured to:

determining at least one item of information according to the hit message: quintuple, packet number, byte number, timestamp, media access address, flow direction, flow action, flow log version, port identification, virtual machine identification, subnet identification, virtual private cloud identification, tenant identification and flow log record state;

and generating a flow log according to the determined information.

Optionally:

the log processing component is specifically configured to: determining the flow direction of the hit message as an incoming direction or an outgoing direction according to the value-taking rules of the incoming port and the outgoing port of the hit message; or

The virtual switch is specifically configured to determine a flow direction of the hit packet according to a bridge port inside a compute node in the hit packet, and notify the log processing component; the log processing component is specifically configured to determine, according to the notification of the virtual switch, that the flow direction of the hit packet is an outgoing direction.

Optionally, the log processing component is specifically configured to:

if the input port value of the hit message points to the flow inlet of the node, determining the flow direction of the hit message as the input direction;

and if the output port value of the hit message points to the flow outlet of the node, the log processing component determines that the flow direction of the hit message is the output direction.

Optionally, the log processing component is specifically configured to:

and if the output port value of the hit message is a set discard value, determining that the flow action of the hit message is discard.

Optionally, the log processing component is further configured to: and after determining that the flow direction of the hit message is an outgoing direction according to the notification of the virtual switch, reversely generating information for determining an incoming flow log according to the outgoing hit message.

Optionally, the log processing component is specifically configured to:

reading the corresponding relation between the long identification and the short identification of at least one of the port, the virtual machine, the subnet and the virtual private cloud from a database of the cloud server;

and extracting a long identifier of at least one of the port, the virtual machine, the subnet and the virtual private cloud according to the hit message, and determining a short identifier corresponding to the long identifier according to the corresponding relation to be used as at least one of the port identifier, the virtual machine identifier, the subnet identifier and the virtual private cloud identifier.

Optionally, the log processing component is further configured to: before the stream log is reported to a log server, determining a log storage path of the stream log reported to the log server according to the configuration granularity in the stream log acquisition rule;

wherein a level of granularity of the configuration granularity comprises at least one of: port granularity, device granularity, subnet granularity, virtual private cloud granularity, and tenant granularity.

Optionally, the virtual switch includes an internal bridge and an external bridge, the log flow table includes an ingress log flow table and an egress log flow table, the ingress log flow table is configured in a packet ingress path of the external bridge, and the egress log flow table is configured in a packet egress path of the internal bridge.

Optionally, the message hit module is specifically configured to: and according to the execution action item in the log flow table, forwarding the meta information of the hit message to a set port corresponding to the log processing component based on a user datagram protocol.

According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.

Fig. 6 is a block diagram of an electronic device according to the flow log collection method in the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 6, the electronic apparatus includes: one or more processors 601, memory 602, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, if desired. Also, multiple electronic devices may be connected, with each device providing some of the necessary operations (e.g., as an array of servers, a group of blade servers, or a multi-processor system). In fig. 6, one processor 601 is taken as an example.

The memory 602 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the method of stream log collection provided herein. A non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method of stream log collection provided herein.

The memory 602, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the method for stream log collection in the embodiment of the present application (for example, the stream table configuration module 510, the message hit module 520, and the log processing component 530 shown in fig. 5). The processor 601 executes various functional applications and data processing of the server by running non-transitory software programs, instructions and modules stored in the memory 602, that is, the method for collecting the stream logs in the above method embodiment is implemented.

The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created from use of the electronic device collected by the stream log, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 602 optionally includes memory located remotely from the processor 601, and these remote memories may be connected to the flow log collection electronics over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of the method for collecting the flow log may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected by a bus or other means, and fig. 6 illustrates the connection by a bus as an example.

The input device 603 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device for stream log collection, such as a touch screen, keypad, mouse, track pad, touch pad, pointing stick, one or more mouse buttons, track ball, joystick, or other input device. The output devices 604 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A flow log collection method is applied to a node of a software defined network, and comprises the following steps:

reporting the stream log to a log server through the log processing component;

the method comprises the following steps of obtaining a log flow table through a virtual switch in the node, and configuring the log flow table in at least one of a message incoming path and a message outgoing path, wherein the log flow table comprises: acquiring a message from an interface of a cloud service end through a virtual switch agent end of the node;

if the acquired message is identified to be a log flow table configuration message, generating a log flow table by the virtual switch agent according to a flow log acquisition rule stored in a database of the cloud service terminal, wherein the log flow table configuration message is generated by the cloud service terminal according to the flow log acquisition rule;

configuring the log flow table to the virtual switch through the virtual switch proxy terminal;

and configuring the log flow table in at least one of a message incoming path and a message outgoing path through the virtual switch.

2. The method of claim 1, wherein generating, by a log processing component in the node, a flow log from the hit messages comprises:

acquiring a flow log acquisition template determined based on a flow log acquisition rule through a synchronization sub-assembly in the log processing assembly;

acquiring and acquiring a hit message forwarded by the virtual switch from a set port through an acquisition subassembly in the log processing assembly;

and extracting information from the acquired hit message according to the flow log acquisition template through a formatting sub-assembly in the log processing assembly, and filling the extracted information into the flow log acquisition template to form a flow log.

3. The method of claim 2, wherein reporting the stream log to a log server by the log processing component comprises:

and reporting the stream log to a log server through an export subassembly in the log processing subassembly.

4. The method of claim 3, wherein the log processing component is implemented by a process, and the synchronization sub-component, the acquisition sub-component, the formatting sub-component, and the export sub-component are implemented separately by separate threads in the process.

5. The method of claim 1, wherein generating, by a log processing component in the node, a flow log from the hit messages comprises:

determining, by a log processing component in the node, at least one item of information from the hit packet: quintuple, packet number, byte number, timestamp, media access address, flow direction, flow action, flow log version, port identification, virtual machine identification, subnet identification, virtual private cloud identification, tenant identification and flow log record state;

generating, by the log processing component, a flow log according to the determined information.

6. The method of claim 5, wherein determining, by a log processing component in the node, a traffic direction from the hit message comprises:

determining the flow direction of the hit message as an incoming direction or an outgoing direction according to the value-taking rules of the incoming port and the outgoing port of the hit message by the log processing component; or

Determining the flow direction of the hit message through the virtual switch according to the internal bridge port of the computing node in the hit message, and informing the log processing component; and determining the flow direction of the hit message as an outgoing direction according to the notification of the virtual switch by the log processing component.

7. The method of claim 6, wherein determining, by the log processing component, that the traffic direction of the hit packet is an ingress direction or an egress direction according to ingress and egress port value-taking rules of the hit packet comprises:

8. The method of claim 5, wherein determining, by a log processing component in the node, a traffic action from the hit message comprises:

9. The method of claim 5, wherein after determining, by the log processing component, that the hit packet traffic direction is outbound according to the notification of the virtual switch, further comprising:

10. The method of claim 5, wherein determining, by a log processing component in the node, from the hit packet, at least one of a port identification, a virtual machine identification, a subnet identification, and a virtual private cloud identification comprises:

11. The method of claim 1, wherein before reporting the stream log to a log server via the log processing component, further comprising:

12. The method of claim 1, wherein the virtual switch comprises an internal bridge and an external bridge, the log flow table comprising an ingress log flow table configured in a packet ingress path of the external bridge and an egress log flow table configured in a packet egress path of the internal bridge.

13. The method of claim 1, wherein forwarding, by the virtual switch, the hit packet to a log processing component according to the execution action entry in the log flow table comprises:

and forwarding the meta information of the hit message to a set port corresponding to the log processing component based on a user datagram protocol through the virtual switch according to the execution action item in the log flow table.

14. A flow log collection apparatus configured in a node of a software defined network, the apparatus comprising:

the message hit module is configured in the virtual switch and used for forwarding a hit message to the log processing component according to an execution action item in the log flow table when the message is hit and matched according to the log flow table;

the log processing component is configured in the node and used for generating a stream log according to the hit message;

the log processing component is also used for reporting the stream log to a log server;

wherein the apparatus further comprises:

a flow table generating module configured at the virtual switch agent end, and configured to generate a log flow table according to a flow log acquisition rule stored in a database of the cloud service end if it is identified that the acquired message is a log flow table configuration message, where the log flow table configuration message is generated by the cloud service end according to the flow log acquisition rule;

the flow table providing module is configured at the proxy end of the virtual switch and is used for configuring the log flow table to the virtual switch;

15. The apparatus of claim 14, wherein the log processing component comprises:

the synchronization subassembly is used for acquiring the stream log acquisition template determined based on the stream log acquisition rule;

the acquisition subassembly is used for acquiring and acquiring the hit message forwarded by the virtual switch from a set port;

16. The apparatus of claim 15, wherein the log processing component further comprises:

17. The apparatus of claim 16, wherein the log processing component is implemented by a process, the synchronization sub-component, the acquisition sub-component, the formatting sub-component, and the export sub-component are implemented separately by separate threads in the process.

18. The apparatus of claim 14, wherein the log processing component is specifically configured to:

and generating a flow log according to the determined information.

19. The apparatus of claim 18, wherein:

the log processing component is specifically configured to: determining the flow direction of the hit message as an inlet direction or an outlet direction according to the value rules of an inlet port and an outlet port of the hit message; or

20. The apparatus of claim 19, wherein the log processing component is specifically configured to:

21. The apparatus of claim 19, wherein the log processing component is specifically configured to:

22. The apparatus of claim 19, wherein the log processing component is further to: and after determining that the flow direction of the hit message is an outgoing direction according to the notification of the virtual switch, reversely generating information for determining an incoming flow log according to the outgoing hit message.

23. The apparatus of claim 19, wherein the log processing component is specifically configured to:

24. The apparatus of claim 14, wherein the log processing component is further to: before the stream log is reported to a log server, determining a log storage path of the stream log reported to the log server according to the configuration granularity in the stream log acquisition rule;

25. The apparatus of claim 14, wherein the virtual switch comprises an internal bridge and an external bridge, the log flow table comprising an ingress log flow table configured in a packet ingress path of the external bridge and an egress log flow table configured in a packet egress path of the internal bridge.

26. The apparatus of claim 14, wherein the message hit module is specifically configured to: and according to the execution action item in the log flow table, forwarding the meta information of the hit message to a set port corresponding to the log processing component based on a user datagram protocol.

27. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the flow log collection method of any one of claims 1-13.

28. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of stream log collection of any one of claims 1-13.