CN113347258B - Method and system for data acquisition, monitoring and analysis under cloud flow - Google Patents
Method and system for data acquisition, monitoring and analysis under cloud flow Download PDFInfo
- Publication number
- CN113347258B CN113347258B CN202110626536.4A CN202110626536A CN113347258B CN 113347258 B CN113347258 B CN 113347258B CN 202110626536 A CN202110626536 A CN 202110626536A CN 113347258 B CN113347258 B CN 113347258B
- Authority
- CN
- China
- Prior art keywords
- flow
- cloud
- environment
- identification
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/18—Protocol analysers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a method for data acquisition, monitoring and analysis under cloud flow, which comprises the following steps: step 1: carrying out mirror image processing on the flow in the environment in the cloud; step 2: collecting, packaging, marking and sending the flow obtained in the step 1 in the cloud environment; and 3, step 3: collecting the flow obtained in the step 2 in an out-of-cloud environment; and 4, step 4: and (3) carrying out network protocol analysis on the flow obtained in the step (3) in the out-of-cloud environment. The method is effectively suitable for data flow acquisition and conversion in various cloud environments, effectively solves the problem of difficult network flow analysis caused by IP dynamic change when the container is created and destroyed in the cloud environment, and can accurately and quickly identify and process the data packet in the complex network environment.
Description
Technical Field
The invention relates to the technical field of data acquisition, monitoring and analysis, in particular to a method and a system for data acquisition, monitoring and analysis under cloud flow.
Background
In the current bank, security dealer and network operation business, the data flow collection, monitoring and analysis are an important operation and maintenance means for ensuring the stable and good operation of the business. Fig. 1 is a diagram of a typical network data transmission topology. Data is collected in the monitored environment and transmitted to the monitoring system for subsequent processing via a network switching device such as a switch.
A conventional data acquisition processing scheme is shown in fig. 2. Through a switch or a router, the data traffic of a port is mirrored to a specific port, the port is connected by using a network cable, a packet is grabbed after a hybrid mode is set, the traffic of the specific port is collected, and the collected traffic packet can be subjected to subsequent protocol analysis and monitoring analysis to generate corresponding service data.
The chinese patent invention with publication number CN101188531A discloses a method and system for monitoring network traffic abnormality, which solves the problems of switch performance loss, data packet loss and the like caused by the existing switch mirroring technology used for monitoring traffic abnormality. The method comprises the following steps: the method comprises the steps of firstly positioning a port with abnormal flow in the switch by a coarse-grained monitoring method, then automatically adjusting a mirror image port of the switch, copying the abnormal port flow to the mirror image port, and finally analyzing the mirror image port flow by a fine-grained monitoring method to obtain the reason of abnormal flow. The method is an improvement on the traditional switch mirroring technology, the switch mirroring can be triggered only when the flow is abnormal, and the mirroring function of the switch cannot be used in most of time, so that the performance of the switch cannot be influenced by increasing the load; moreover, the mirror image ports are adjusted in a dynamic and targeted mode, and the packet loss condition caused by mirroring a plurality of ports to one port is avoided.
For the prior art in the foregoing, the inventor thinks that with the arrival of the cloud era and the continuous development of the cloud technology, the traditional data acquisition scheme faces the following two problems: firstly, a monitoring system is generally positioned outside a cloud, and a monitored environment in the cloud cannot acquire data in a network cable connection mode, but needs an acquisition and transmission scheme of a service network; secondly, in the cloud environment, the creation and destruction of the container can cause the IP of the data packet to change during each creation, and the traditional configuration mode of IP + PORT is used for identifying and analyzing the network flow which cannot meet the dynamic scene of the scene.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method and a system for data acquisition, monitoring and analysis under cloud flow.
The invention provides a method for data acquisition, monitoring and analysis under cloud flow, which comprises the following steps:
step 1: carrying out mirror image processing on the flow in the environment in the cloud;
step 2: collecting, packaging, marking and sending the flow obtained in the step 1 in the cloud environment;
and 3, step 3: collecting the flow obtained in the step 2 in an out-of-cloud environment;
and 4, step 4: and (3) carrying out network protocol analysis on the flow obtained in the step (3) in the out-of-cloud environment.
Preferably, the flow rate in the in-cloud environment in step 1 includes Vmware flow rate, docker flow rate and K8S flow rate.
Preferably, the step 2 comprises the following steps:
collecting the flow in the cloud: collecting the flow obtained in the step 1 in the cloud environment;
and (3) packaging identification: packaging and identifying the flow obtained in the cloud flow acquisition step according to configuration in the cloud environment;
an intra-cloud flow sending step: and sending the flow obtained in the step of packaging identification to the environment outside the cloud in the environment outside the cloud.
Preferably, in the step of encapsulating and identifying, the acquired traffic is encapsulated and identified by an encapsulation protocol, where the encapsulation protocol includes GRE, VLAN, IPX, VXLAN, and MPLS.
Preferably, the step 3 comprises the following steps:
acquiring the cloud external flow: collecting the flow obtained in the step 2 in an out-of-cloud environment, wherein the collected flow comprises mixed flow processed according to different encapsulation protocols;
a standardization step: and in the out-of-cloud environment, carrying out standardization processing on the flow obtained in the out-of-cloud flow acquisition step.
The invention provides a system for data acquisition, monitoring and analysis under cloud flow, which comprises the following modules:
a module M1: carrying out mirror image processing on the flow in the environment in the cloud;
a module M2: collecting, packaging, marking and sending the flow obtained by the module M1 in the cloud environment;
a module M3: collecting the flow obtained by the module M2 in the out-of-cloud environment;
a module M4: and in the out-of-cloud environment, performing network protocol analysis on the flow obtained by the module M3.
Preferably, the flow in the in-cloud environment in the module M1 includes Vmware flow, docker flow and K8S flow.
Preferably, the module M2 includes the following modules:
the cloud internal flow acquisition module: collecting the flow obtained by the module M1 in the environment in the cloud;
and (3) packaging the identification module: packaging and identifying the flow obtained by the in-cloud flow acquisition module according to configuration in the in-cloud environment;
the cloud internal flow sending module: and sending the flow obtained by the encapsulation identification module to the out-of-cloud environment in the in-cloud environment.
Preferably, the encapsulation identification module encapsulates and identifies the collected traffic through an encapsulation protocol, where the encapsulation protocol includes GRE, VLAN, IPX, VXLAN, and MPLS.
Preferably, the module M3 includes the following modules:
the cloud external flow acquisition module: collecting the flow obtained by the module M2 in the out-of-cloud environment, wherein the collected flow comprises mixed flow processed according to different encapsulation protocols;
a standardization module: and in the out-of-cloud environment, the flow acquired by the out-of-cloud flow acquisition module is subjected to standardized processing.
Compared with the prior art, the invention has the following beneficial effects:
1. the method is effectively suitable for data traffic acquisition and conversion under various cloud environments, including but not limited to Vmware, docker and K8S, and can be expanded and deployed to the cloud environment only by simply setting a monitoring system under the traditional scheme;
2. the method effectively solves the problem of difficult network flow analysis caused by IP dynamic change when the container is created and destroyed in the cloud environment, and can accurately and quickly identify and process the data packet in the complex network environment;
3. the invention supports a plurality of package schemes, including but not limited to GRE, VLAN, IPX, VXLAN, MPLS, with strong compatibility;
4. the invention simplifies the subsequent processing and greatly improves the data processing efficiency after the unified optimization and the extraction of the identification are carried out.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a network data transmission topology;
FIG. 2 is a diagram of a conventional data acquisition monitoring scheme;
FIG. 3 is a flow chart of the steps of the present invention;
FIG. 4 is a GRE encapsulation packet structure;
FIG. 5 is a diagram of a VLAN encapsulated packet architecture;
FIG. 6 is a diagram of an IPX encapsulated packet structure;
fig. 7 is a diagram of a VXLAN encapsulated packet structure;
FIG. 8 is a diagram of an MPLS encapsulated packet structure;
FIG. 9 is a schematic of mixed flow acquisition and normalization;
FIG. 10 is an exemplary illustration of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
The embodiment of the invention discloses a method and a system for data acquisition, monitoring and analysis under cloud flow, as shown in FIG. 3, the method comprises the following steps:
step 1: and (4) mirroring the flow in the environment in the cloud, and carrying out mirroring treatment on the flow in the environment in the cloud. Traffic in the in-cloud environment includes Vmware traffic, docker traffic, and K8S traffic. Network traffic in an intra-cloud environment. The step 1 comprises the following steps: vmware traffic mirroring step: when the cloud is a Vmware virtual machine environment, the flow mirroring to the specified destination port can be realized by configuring the virtual switch and the virtual network card and setting the port mirroring. Docker flow mirroring step: and when the cloud is in a Docker environment, the flow is forwarded to the destination port through a Docker0 network bridge on the host machine. K8S flow mirroring step: when the cloud is a K8S (kubernets) environment, traffic can be forwarded from the service container of the host to the target address by creating a mirror container and configuring a calico network.
And exporting the data of the monitored nodes by adopting a network flow mirroring scheme, wherein the environment in the cloud comprises but is not limited to Vmware cloud mirror, docker cloud mirror and K8S cloud mirror.
Step 2: and (3) acquiring, packaging, identifying and sending the flow obtained in the step (1) in the cloud environment, and acquiring, labeling and transmitting the flow in the cloud environment. The step 2 comprises the following steps: collecting the flow in the cloud: and (3) collecting the flow obtained in the step (1) in the cloud environment, and collecting the flow in the cloud. And collecting the flow in the cloud, wherein the collected flow can be the flow aiming at a single network card or the flow of multiple network cards. Meanwhile, various filtering conditions (such as sending end/receiving end IP addresses, port numbers, protocols and the like) and complex combinations of AND, OR and NOT of the filtering conditions can be set according to the requirements of users aiming at the acquired data packets.
And (3) packaging and marking: encapsulating and identifying the flow obtained in the cloud flow acquisition step according to configuration in the cloud internal environment, and encapsulating and identifying the flow according to the configuration; and in the step of encapsulating the identifier, the acquired flow is encapsulated and identified through an encapsulation protocol, wherein the encapsulation protocol comprises GRE, VLAN, IPX, VXLAN and MPLS. There are many general encapsulation protocols, and a ServiceTag (service identifier) and a PacketDir (packet direction, send/receive) may be set in a header of the protocol. The encapsulation protocols include GRE, VLAN, IPX, VXLAN, and MPLS, which are described below:
GRE (Generic Routing Encapsulation) is a Generic Routing Encapsulation protocol that encapsulates datagrams of certain network layer protocols so that they can be transported in a network. When the protocol is adopted for encapsulation, an ethernet layer and an ip layer need to be additionally added on the outer layer of the original data packet, and the ServiceTag and the PacketDir are set in the key field, as shown in fig. 4.
The VLAN (Virtual Local Area Network) Virtual Network protocol adds a VLAN layer between an ethernet layer and an ip layer, and the ServiceTag and PacketDir are set in the tti field, and the packet structure is as shown in fig. 5.
The IPX (Internet Packet Exchange protocol) internet Packet switching protocol provides functions of Packet addressing and routing, ensures reliable arrival and is equivalent to the function of datagram. The SAP listener knows the multicast range in which the notification is located through the multicast range area notification protocol (or other protocol) and listens for notifications on SAP addresses and ports within those ranges. In this way, the listener will eventually know all session announcements and allow those sessions to join the multicast scope. The location of the IPX and SAP in the protocol stack is shown in figure 6. The ServiceTag and PacketDir may be provided in the SAP header at specific positions as shown in fig. 6.
VXLAN (Virtual eXtensible Local Area Network) technology well solves the problem that VLAN technology cannot meet the requirements of large two-tier networks. When encapsulating the vxlan layer, an ethernet layer, an ip layer and an udp layer need to be added in addition to the original packet, the ServiceTag and the PacketDir are set in the vni field, and the structure of packet encapsulation is as shown in fig. 7.
MPLS (Multi-Protocol Label Switching) is a new technology for guiding high-speed and efficient transmission of data over an open communication network by using labels. The meaning of multiprotocol means that MPLS can not only support multiple protocols on the network layer level, but also be compatible with multiple data link layer technologies on the second layer. An mpls header may be inserted between the ethernet layer and the ip layer, with ServiceTag and PacketDir set in the corresponding fields, as shown in fig. 8.
And according to the cloud acquisition scheme, after acquisition, corresponding identification can be marked on the flow, and the identification comprises the belonged application and the flow direction. The flow identification scheme does not change the original service structure, can make identification according to the scheme accepted by the monitored cloud environment at the moment, and transmits the data packet to the outside of the cloud by using encapsulation technologies including but not limited to GRE, VLAN, IPX, VXLAN and MPLS.
An intra-cloud flow sending step: and in the cloud environment, sending the flow obtained in the step of encapsulating the identifier to the cloud environment, and sending the flow in the cloud. After the packet is identified by other protocol encapsulations, it can be sent to the destination address for processing. It should be noted here that the traffic may be forwarded to multiple destination addresses, and different output bandwidth limits may be set for different destination addresses.
And step 3: and (3) collecting the flow obtained in the step (2) in the out-of-cloud environment, and monitoring flow collection and standardization outside the cloud. The step 3 comprises the following steps: acquiring the cloud external flow: and (3) collecting the flow obtained in the step (2) in the out-of-cloud environment, wherein the collected flow comprises mixed flow processed according to different encapsulation protocols, and the out-of-cloud flow is collected. And (3) checking the target address outside the cloud after receiving the flow passing the label, determining the target address as a packet sent by the acquisition program inside the cloud, and acquiring the packet, wherein the acquired flow is the mixed flow identified according to different protocols and processed in the step (2).
A standardization step: and standardizing, namely standardizing the flow obtained in the cloud external flow acquisition step in the cloud external environment. According to different protocol structures, collected data packets are re-standardized into the preferred scheme of the current monitoring system, so that subsequent processing is simplified, namely, the collected data packets with any structure are mapped into a unified standardized structure. The mapping here refers to modifying the ServiceTag and PacketDir from the original encapsulation scheme to the selected standardized encapsulation scheme. The present solution includes, but is not limited to, the solution of encapsulating the identification step as a standardized solution. And simultaneously, MPLS encapsulation is preferably adopted, and the MPLS is particularly suitable for large-scale network system architecture and can be widely supported by hardware such as a switch and the like. And (4) performing unified standardized encapsulation outside the cloud in order to simplify subsequent processing.
And 4, step 4: and (4) network protocol analysis, namely performing network protocol analysis on the flow obtained in the step (3) in the out-of-cloud environment. The step 4 comprises the following steps: step 4.1: and extracting identification and reading and writing. The extraction and reading and writing of the service identifier configuration are carried out in the monitoring system, and the step replaces the mode of reading the IP + PORT configuration in the configuration table in the traditional scheme. In the traditional scheme, the IP quadruplets of the client IP/PORT and the server IP/PORT are configured and stored in a configuration file or a database, and after a data packet is collected on a server, the configuration of the IP quadruplets in the file or the database needs to be read again for matching. In the invention, single integer data with a service identifier of 0-4095 is simultaneously extracted, read and written in real time and is used for flow analysis in subsequent steps;
step 4.2: application identification and direction. After the service identification and the direction are read, matching decoding is carried out on the service identification and the direction stripped from the flow after standardized processing, different data sources are identified, and then the obtained data are continuously transmitted to a monitoring system service processing module.
Step 4.3: and (5) carrying out subsequent treatment. The part is shown for statistical storage of the service data, and is not described in detail. The method of the invention supports the butt joint of other various monitoring systems capable of processing the service data, and the monitoring systems extract and apply the identification according to a uniform mode.
As an example shown in fig. 10, the first step is to collect the raw traffic image within a k8s cloud environment. And in the second step, the data packet acquired in the first step is encapsulated by GRE and is sent out of the cloud environment. And thirdly, after the processed flow is collected outside the cloud environment, using MPLS to package a data packet, and using a standardized format to contain a data identifier and a flow direction. And fourthly, the monitoring system processes the transmitted standardized data by reading the configured identifier and performs subsequent service data processing and analysis.
It is well within the knowledge of a person skilled in the art to implement the system and its various devices, modules, units provided by the present invention in a purely computer readable program code means that the same functionality can be implemented by logically programming method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices, modules and units thereof provided by the present invention can be regarded as a hardware component, and the devices, modules and units included therein for implementing various functions can also be regarded as structures within the hardware component; means, modules, units for performing the various functions may also be regarded as structures within both software modules and hardware components for performing the method.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (6)
1. A method for data acquisition, monitoring and analysis under cloud flow is characterized by comprising the following steps:
step 1: carrying out mirror image processing on the flow in the environment in the cloud;
step 2: collecting, packaging, identifying and sending the flow obtained in the step 1 in the cloud environment;
and step 3: collecting the flow obtained in the step 2 in an out-of-cloud environment;
and 4, step 4: performing network protocol analysis on the flow obtained in the step 3 in the out-of-cloud environment;
the step 1 comprises the following steps:
vmware traffic mirroring step: when the cloud is a Vmware virtual machine environment, a virtual switch and a virtual network card are configured, and a port mirror image is set, so that the flow mirror image is realized to a specified destination port;
docker flow mirroring: when the cloud is in a Docker environment, the flow is forwarded to a destination port through a Docker0 network bridge on the host;
K8S flow mirroring step: when the cloud is in a K8S environment, forwarding the flow from a service container of a host to a target address by creating a mirror image container and configuring a calico network;
exporting data of the monitored node by adopting a network flow mirroring scheme;
the step 2 comprises the following steps:
collecting the flow in the cloud: collecting the flow obtained in the step 1 in the cloud environment, and setting various filtering conditions according to the requirements of users aiming at the collected data packets;
and (3) packaging identification: encapsulating the flow obtained in the cloud flow acquisition step according to configuration in the cloud environment, and encapsulating and identifying the flow according to configuration, wherein the identification comprises the application and the flow direction, the original service structure is not changed in the flow identification scheme, and the identification is made according to the scheme accepted by the monitored cloud environment at the time;
cloud internal flow sending: in the in-cloud environment, sending the traffic obtained in the step of encapsulating the identification to the out-cloud environment, forwarding the traffic to a plurality of target addresses, and setting different output bandwidth limits for different target addresses;
the step 3 comprises the following steps:
acquiring the cloud external flow: acquiring the flow obtained in the step 2 in an out-of-cloud environment, verifying the flow after the out-of-cloud target address receives the labeled flow, and acquiring after determining the flow as a packet sent by an in-cloud acquisition program, wherein the acquired flow comprises mixed flow processed according to different encapsulation protocols;
a standardization step: standardizing the flow obtained in the out-of-cloud flow acquisition step in an out-of-cloud environment, re-standardizing the acquired data packets into a preferred scheme of the current monitoring system according to different protocol structures, mapping the acquired data packets with any structure into a unified standardized structure, wherein the mapping refers to modifying the Servicetag and the PacketDir from an original encapsulation scheme into a selected standardized encapsulation scheme, and the scheme comprising the encapsulation identification step is used as a standardized scheme and is encapsulated by MPLS;
the step 4 comprises the following steps:
step 4.1: extracting and reading and writing the identification, extracting and reading and writing service identification configuration in the monitoring system, wherein the service identification is single integer data of 0-4095, and simultaneously, the real-time extraction and reading and writing are realized for flow analysis in the subsequent steps;
step 4.2: after the service identification and the direction are read by applying the identification and the direction, matching decoding is carried out on the service identification and the direction stripped from the flow after standardized processing, different data sources are identified, and then the obtained data are continuously transmitted to a monitoring system service processing module;
step 4.3: and subsequent processing, namely statistical storage and display of the service data, supporting the butt joint of other various monitoring systems capable of processing the service data, and extracting and applying the identification by the monitoring systems according to a uniform mode.
2. The method for data acquisition, monitoring and analysis under cloud flow according to claim 1, wherein the flow in the cloud environment in step 1 includes Vmware flow, docker flow and K8S flow.
3. The method of claim 1, wherein the step of encapsulating and identifying encapsulates and identifies the collected traffic with encapsulation protocols, and the encapsulation protocols include GRE, VLAN, IPX, VXLAN, and MPLS.
4. A system for monitoring and analyzing data acquisition under cloud traffic, which is characterized in that a method for monitoring and analyzing data acquisition under cloud lingering according to claims 1-3 is applied, and comprises the following modules:
a module M1: carrying out mirror image processing on the flow in the environment in the cloud;
a module M2: collecting, packaging, marking and sending the flow obtained by the module M1 in the cloud environment;
a module M3: collecting the flow obtained by the module M2 in the out-of-cloud environment;
a module M4: performing network protocol analysis on the flow obtained by the module M3 in the out-of-cloud environment;
the module M1 implements:
vmware traffic mirroring step: when the cloud is a Vmware virtual machine environment, a virtual switch and a virtual network card are configured, and a port mirror image is set, so that the flow mirror image is realized to a specified destination port;
docker flow mirroring: when the cloud is in a Docker environment, the flow is forwarded to a destination port through a Docker0 network bridge on the host;
K8S flow mirroring step: when the cloud is in a K8S environment, forwarding the flow from a service container of a host to a target address by creating a mirror image container and configuring a calico network;
exporting data of the monitored node by adopting a network flow mirroring scheme;
the module M2 implements:
collecting the flow in the cloud: collecting the flow obtained by the module M1 in the cloud environment, and setting various filtering conditions according to the requirements of users aiming at the collected data packets;
and (3) packaging identification: packaging the flow obtained in the cloud flow acquisition step according to configuration in the cloud environment, and packaging and identifying the flow according to configuration, wherein the identification comprises an application and a flow direction, the original service structure is not changed in the flow identification scheme, and the identification is made according to the scheme accepted by the monitored cloud environment at the moment;
an intra-cloud flow sending step: in the in-cloud environment, sending the traffic obtained in the step of encapsulating the identification to the out-cloud environment, forwarding the traffic to a plurality of target addresses, and setting different output bandwidth limits for different target addresses;
the module M3 implements:
acquiring the cloud external flow: acquiring the flow obtained by the module M2 in an out-of-cloud environment, verifying the flow after the out-of-cloud target address receives the labeled flow, and acquiring after determining the flow as a packet sent by an in-cloud acquisition program, wherein the acquired flow comprises mixed flow processed according to different encapsulation protocols;
a standardization step: standardizing the flow obtained in the off-cloud flow acquisition step in an off-cloud environment, re-standardizing the acquired data packets into a preferred scheme of the current monitoring system according to different protocol structures, mapping the acquired data packets of any structure into a uniform standardized structure, wherein the mapping refers to modifying the ServiceTag and the PacketDir from the original encapsulation scheme into a selected standardized encapsulation scheme, and the scheme comprising the encapsulation identification step is used as a standardized scheme and MPLS is used for encapsulation;
the module M4 implements:
step 4.1: extracting and reading and writing identification, wherein the service identification configuration is extracted and read and written in the monitoring system, the service identification is single integer data of 0-4095, and the real-time extraction and reading and writing are realized simultaneously for flow analysis in the subsequent steps;
step 4.2: after the service identification and the direction are read by applying the identification and the direction, matching decoding is carried out on the service identification and the direction stripped from the flow after standardized processing, different data sources are identified, and then the obtained data are continuously transmitted to a monitoring system service processing module;
step 4.3: and subsequent processing, namely statistical storage and display of the service data, supporting the butt joint of other various monitoring systems capable of processing the service data, and extracting and applying the identification by the monitoring systems according to a uniform mode.
5. The system for data acquisition, monitoring and analysis under cloud flow according to claim 4, wherein the flow in the in-cloud environment in the module M1 includes Vmware flow, docker flow and K8S flow.
6. The system of claim 5, wherein the encapsulation module encapsulates and identifies the collected traffic through encapsulation protocols, and the encapsulation protocols include GRE, VLAN, IPX, VXLAN, and MPLS.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110626536.4A CN113347258B (en) | 2021-06-04 | 2021-06-04 | Method and system for data acquisition, monitoring and analysis under cloud flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110626536.4A CN113347258B (en) | 2021-06-04 | 2021-06-04 | Method and system for data acquisition, monitoring and analysis under cloud flow |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113347258A CN113347258A (en) | 2021-09-03 |
| CN113347258B true CN113347258B (en) | 2023-02-07 |
Family
ID=77474404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110626536.4A Active CN113347258B (en) | 2021-06-04 | 2021-06-04 | Method and system for data acquisition, monitoring and analysis under cloud flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113347258B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114157458A (en) * | 2021-11-18 | 2022-03-08 | 深圳依时货拉拉科技有限公司 | Flow detection method, device, equipment and medium for hybrid cloud environment |
| CN114679391A (en) * | 2022-02-23 | 2022-06-28 | 网宿科技股份有限公司 | Method for collecting network traffic of docker container, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106254176A (en) * | 2016-07-29 | 2016-12-21 | 浪潮(北京)电子信息产业有限公司 | A kind of traffic mirroring method based on openvswitch |
| CN107426252A (en) * | 2017-09-15 | 2017-12-01 | 北京百悟科技有限公司 | The method and apparatus that web application firewall services are provided |
| CN111930388A (en) * | 2020-07-13 | 2020-11-13 | 银盛支付服务股份有限公司 | Continuous integration method and system based on k8s and docker container |
| CN112631737A (en) * | 2021-01-08 | 2021-04-09 | 重庆紫光华山智安科技有限公司 | Container mirror image construction method and device, electronic equipment and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101547112A (en) * | 2008-03-28 | 2009-09-30 | 华为技术有限公司 | Network flow sampling method and system |
| CN106100999B (en) * | 2016-08-28 | 2019-05-24 | 北京瑞和云图科技有限公司 | Image network flow control methods in a kind of virtualized network environment |
| CN108234315A (en) * | 2016-12-21 | 2018-06-29 | 青岛祥智电子技术有限公司 | Image network flow control protocol in a kind of virtualized network environment |
| CN108173695B (en) * | 2017-12-29 | 2021-10-19 | 深信服科技股份有限公司 | Flow monitoring system and method in cloud environment |
| CN109194747A (en) * | 2018-09-10 | 2019-01-11 | 四川长虹电器股份有限公司 | Traffic mirroring method and system under cloud environment |
| CN109271217B (en) * | 2018-10-23 | 2022-02-11 | 上海携程商务有限公司 | Network flow detection method and system under cloud environment |
| CN112787949B (en) * | 2020-09-17 | 2023-05-16 | 中兴通讯股份有限公司 | Flow acquisition and delivery management method, control device and storage medium |
| CN112653628B (en) * | 2020-12-23 | 2022-07-12 | 新华三技术有限公司 | ERSPAN method and network equipment |
-
2021
- 2021-06-04 CN CN202110626536.4A patent/CN113347258B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106254176A (en) * | 2016-07-29 | 2016-12-21 | 浪潮(北京)电子信息产业有限公司 | A kind of traffic mirroring method based on openvswitch |
| CN107426252A (en) * | 2017-09-15 | 2017-12-01 | 北京百悟科技有限公司 | The method and apparatus that web application firewall services are provided |
| CN111930388A (en) * | 2020-07-13 | 2020-11-13 | 银盛支付服务股份有限公司 | Continuous integration method and system based on k8s and docker container |
| CN112631737A (en) * | 2021-01-08 | 2021-04-09 | 重庆紫光华山智安科技有限公司 | Container mirror image construction method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113347258A (en) | 2021-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8073966B2 (en) | Virtual interface | |
| US9654303B2 (en) | Method and apparatus for emulating network devices | |
| US10887119B2 (en) | Multicasting within distributed control plane of a switch | |
| US6799220B1 (en) | Tunneling management messages over a channel architecture network | |
| US8661292B2 (en) | Network communication at unaddressed network devices | |
| US7106751B2 (en) | Apparatus for linking a SAN with a LAN | |
| US20160337146A1 (en) | Method of data delivery across a network fabric in a router or ethernet bridge | |
| CN113347258B (en) | Method and system for data acquisition, monitoring and analysis under cloud flow | |
| CN110601983A (en) | Method and system for forwarding routing without sensing source of protocol | |
| CN108270699B (en) | Message processing method, shunt switch and aggregation network | |
| CN106559302A (en) | Single tunnel method for building up, device and system | |
| CN108093041A (en) | Single channel VDI proxy servers and implementation method | |
| CN113328956B (en) | Message processing method and device | |
| CN109005080B (en) | Distributed packet forwarding implementation method, implementation system and automatic test method | |
| WO2005036834A1 (en) | Statistical information collecting method and apparatus | |
| US7496688B2 (en) | Label switched data unit content evaluation | |
| CN103200084A (en) | Message preprocessing method and device based on network processing unit and network processing unit | |
| CN108696398A (en) | Communication loopback fault detection method and device in a kind of communication network | |
| CN111478791A (en) | Data management method and device | |
| CN110768916B (en) | Port statistical parameter notification method and device | |
| CN112543142B (en) | Method and device for realizing RSTP ring network protocol based on FPGA | |
| US20230319168A1 (en) | Hardware ethernet header verification | |
| CN116668405B (en) | Method, system, medium and equipment for implementing multi-service message notification mechanism | |
| CN112787933B (en) | Apparatus, system, and method for offloading IPFIX lookup and translation operations from a view domain | |
| CN116055341A (en) | Method and system for counting flow of virtual machine in cloud network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |