WO2022247308A1

WO2022247308A1 - Flow measurement method and apparatus, and related device

Info

Publication number: WO2022247308A1
Application number: PCT/CN2022/071728
Authority: WO
Inventors: 杨永强; 贾正义; 李元鹏; 杨凯程; 杨仝
Original assignee: 华为云计算技术有限公司
Priority date: 2021-05-25
Filing date: 2022-01-13
Publication date: 2022-12-01
Also published as: CN115396345A

Abstract

The present application provides a flow measurement method and apparatus, and a related device. The method comprises: a server receives packets sent by forwarding network elements connected thereto and acquires a flow storage table, wherein the packets comprise an identifier of a data flow and information of a plurality of forwarding network elements processing the packets, and the flow storage table is used for storing the number of packets in the data flow; then the server adds the number of packets in the data flow to the flow storage table according to the identifier of the data flow and the information of the plurality of forwarding network elements processing the packets to obtain an updated flow storage table; finally, the server sends the updated flow storage table to a flow analysis device. The method can solve the problems in the prior art of incomplete measured link information and consumption of a large amount of bandwidth resources and computing resources.

Description

Flow measurement method, device and related equipment

This application claims priority to a Chinese patent application filed with the China Patent Office on May 25, 2021, with application number 202110573832.2, titled "Flow Measurement Method, Device, and Related Equipment," the entire contents of which are hereby incorporated by reference in this application middle.

technical field

The present application relates to the communication field, and in particular to a flow measurement method, device and related equipment.

Background technique

Traffic measurement is an important part of network management, providing indispensable information for service quality improvement, capacity planning, network billing, congestion control, anomaly detection in data centers and backbone networks. For example, when the network is congested, information about the data flow causing the congestion can be found as soon as possible through flow measurement.

The currently more popular traffic measurement method is the in-band network telemetry (in-band network telemetry, INT) method, but the inventors of the present application have found that this method has the problem of incomplete link information measured, and will consume a large amount of bandwidth resources And computing resources, the problem of high measurement cost.

Contents of the invention

The present application provides a traffic measurement method, device and related equipment, which can solve the problem of incomplete measured link information and the problem of consuming a large amount of bandwidth resources and computing resources existing in the prior art.

In the first aspect, a traffic measurement method is provided, which is applied to the transmission process in which the data generated by the client is transmitted to the server through multiple forwarding network elements, the server is connected to a traffic analysis device, and the multiple forwarding network elements Each forwarding network element is a network device that supports adding information for processing the message to the message in the data flow, and the method includes:

The server receives the message sent by the forwarding network element connected to it, and the message includes the identifier of the data flow and the information of processing the message by the multiple forwarding network elements;

The server obtains a flow storage table, and the flow storage table is used to store the number of packets in the data stream;

The server counts the number of packets in the data flow into the flow storage table according to the identifier of the data flow and the information on processing the packets by the plurality of forwarding network elements, and obtains the updated flow storage table;

The server sends the updated traffic storage table to the traffic analysis device.

In the above solution, the message received by the server includes the information of multiple forwarding network elements processing the message, including the forwarding network element connected to the server (that is, the last forwarding network element that the message passes before reaching the server, Hereinafter referred to as the information of the last forwarding network element) processing packets, that is to say, when the server counts the number of packets in the data flow to the flow storage table, it counts the information of the last forwarding network element processing packets Unlike in the prior art, when the last forwarding network element connected to the server sends information to the traffic analysis device, it does not send the information of its own processing of the packet to the traffic analysis device, resulting in inaccurate link information for traffic measurement. whole.

In addition, in the above solution, after the server receives the flow identifier and the information on processing packets by multiple forwarding network elements, it counts the number of packets in the data flow according to the flow identifier and information on processing packets by multiple forwarding network elements. The flow storage table obtains the updated flow storage table, and then sends the updated flow storage table to the traffic analysis device, unlike in the prior art, the last forwarding network element directly processes the flow identification and multiple forwarding network elements The message information is sent to the traffic analysis device, and the traffic analysis device performs traffic statistics according to the flow identifier sent by the last forwarding network element and the information of multiple forwarding network elements processing the message. Therefore, compared with the prior art, the above solution can reduce the bandwidth resources required to send data to the traffic analysis device, and does not need the traffic analysis device to perform traffic statistics, and can reduce the consumption of computing resources of the traffic analysis device, thereby reducing Measuring the role of cost.

In a possible implementation manner, before the server sends the updated traffic storage table to the traffic analysis device, the method further includes: the server performs an operation on the updated traffic storage table encapsulation.

In the above solution, the server encapsulates the updated traffic storage table, and then sends the encapsulated traffic storage table to the traffic analysis device, which can further reduce the bandwidth resource consumed for sending data to the traffic analysis device, and further reduce the measurement cost.

In a possible implementation manner, the identifier of the data flow includes one or more combinations of the following: an Internet protocol (internet protocol, IP) address of the client, a port through which the client sends the data flow number, the IP address of the server, the port number for the server to receive the data stream, the transport layer protocol used by the client to transmit the data stream to the server, virtual local area network (virtual local area network) , VLAN) identifier.

In the above solution, there are multiple combinations of data flow identification, so flow measurement in different dimensions can be realized, such as measuring the flow of data flow sent by each client, and measuring the data flow sent by each port of each client traffic, etc.

In a possible implementation manner, the information on processing the packet by each forwarding network element includes one or more combinations of the following: an identifier of each forwarding network element, an The port number for receiving the message, the port number for each forwarding network element sending the message, the queue number for the message to enter, the queue number for the message to leave, the number of the queue for each forwarding network element The time when the message is received, and the time when each forwarding network element sends the message.

In the above solution, there are multiple combinations of information for each forwarding network element to process packets, so traffic measurement in different dimensions can be realized, such as the measurement of the flow of data flows passing through each forwarding network element, and the measurement of each forwarding network element. The flow rate of the data flow received by each port of the network element, etc.

In the second aspect, a traffic measurement method is provided, which is applied to the transmission process in which the data flow generated by the client is transmitted to the server through multiple forwarding network elements, and the statistical device is connected to the multiple forwarding network elements to send the data flow For the last forwarding network element of the server, the statistics device is connected to a traffic analysis device, and each forwarding network element in the multiple forwarding network elements supports adding its processing information to the packets in the data flow. A network device that describes the information of the message, the method includes:

The statistical device receives a message sent by the last forwarding network element, the message includes the identifier of the data flow, and the message carries information on processing the message by the multiple forwarding network elements;

The statistical device obtains a flow storage table, and the flow storage table is used to store the number of packets in the data flow;

The statistics device counts the number of packets in the data flow to the flow storage table according to the identifier of the data flow and the information on processing the packets by the plurality of forwarding network elements, and obtains the updated flow storage table;

The statistical device sends the updated traffic storage table to the traffic analysis device.

In a possible implementation manner, before the statistical device sends the updated traffic storage table to the traffic analysis device, the method further includes: the statistical device performs an operation on the updated traffic storage table encapsulation.

In a possible implementation manner, the identifier of the data flow includes one or more of the following combinations: the IP address of the client, the port number through which the client sends the data flow, the IP address of the server address, the port number of the server receiving the data flow, the transport layer protocol used by the client to transmit the data flow to the server, and the VLAN identifier.

In the third aspect, a traffic measurement device is provided, which is applied to the transmission process in which the data generated by the client is transmitted to the server through multiple forwarding network elements, and is specifically applied to the server, and the server is also connected to a traffic analysis device Each forwarding network element in the plurality of forwarding network elements is a network device that supports adding information for processing the message to the message in the data flow, and the device includes:

The receiving module is configured to receive the message sent by the forwarding network element connected to the server, the message includes the identifier of the data flow, and the message carries the information processed by the multiple forwarding network elements information about the message;

An acquisition module, configured to acquire a flow storage table, where the flow storage table is used to store the number of packets in the data stream;

A statistics module, configured to count the number of packets in the data flow to the flow storage table according to the identifier of the data flow and the information on processing the packets by the multiple forwarding network elements, and obtain an updated Flow storage table;

A sending module, configured to send the updated flow storage table to the flow analysis device.

In a possible implementation manner, the device further includes: an encapsulation module, configured to encapsulate the updated flow storage table.

In a possible implementation manner, the identifier of the data flow includes one or more of the following combinations: the IP address of the client, the port number through which the client sends the data flow, the IP address of the server address, the port number of the server receiving the data flow, the transport layer protocol used by the client to transmit the data flow to the server, and the VLAN ID.

In the fourth aspect, a traffic measurement device is provided, which is applied to the transmission process of the data generated by the client through multiple forwarding network elements and transmitted to the server, and is specifically applied to a statistical device, and the statistical device is connected to the multiple forwarding networks The last forwarding network element that sends the data flow to the server in the unit, the statistical device is also connected to a traffic analysis device, and each forwarding network element in the multiple forwarding network elements is to support the data flow in the A network device that adds information on processing the message to the message in the message, the device includes:

A receiving module, configured to receive a message sent by the last forwarding network element, the message includes the identifier of the data flow, and the message carries information on processing the message by the multiple forwarding network elements ;

In a fifth aspect, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium stores instructions, and the instructions are used to implement any possible implementation manner or the second aspect of the first aspect above. A method provided by any possible implementation of the aspect.

In a sixth aspect, a computer program product is provided, including a computer program. When the computer program is read and executed by a cluster of computer equipment, the cluster of computer equipment executes any possible implementation manner or the first implementation mode of the first aspect above. The method provided by any possible implementation in the second aspect.

In a seventh aspect, a computing device cluster is provided, including at least one computing device, each computing device includes a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that The computing device executes the method provided in any possible implementation manner of the first aspect or any possible implementation manner of the second aspect.

In a possible implementation manner, the computing device cluster includes a computing device, and the computing device includes a processor and a memory; the processor is configured to execute instructions stored in the memory, so that the computing device performs the above-mentioned first aspect. Any possible implementation or the method provided by any possible implementation of the second aspect.

In a possible implementation manner, the computing device cluster includes at least two computing devices, and each computing device includes a processor and a memory; the processors of the at least two computing devices are used to execute the Stored instructions, so that the cluster of computing devices executes the method provided in any possible implementation manner of the first aspect or any possible implementation manner of the second aspect.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the drawings that need to be used in the description of the embodiments.

Fig. 1 is a schematic diagram of the first flow storage table involved in the present application;

Fig. 2 is the structural representation of the flow measurement system involved in the present application;

Fig. 3 is a schematic flow chart of a flow measurement method provided by the present application;

Fig. 4 is a schematic diagram of changes in the first flow storage table provided by the present application;

Fig. 5 is a schematic diagram of a second flow storage table provided by the present application;

Fig. 6 is a schematic diagram of an exemplary first flow storage table and an exemplary second flow storage table provided by the present application;

Fig. 7 is a schematic flow chart of another flow measurement method provided by the present application;

Fig. 8 is a schematic structural diagram of a flow measurement system provided by the present application;

FIG. 9 is a schematic diagram of a traffic query process provided by the present application;

FIG. 10 is a schematic diagram of a traffic query process provided by the present application;

Fig. 11 is a schematic structural diagram of a flow measuring device provided by the present application;

FIG. 12 is a schematic structural diagram of a computing device cluster provided by the present application;

Fig. 13 is a schematic structural diagram of a computing device provided by the present application.

Detailed ways

Embodiments of the present application are described below with reference to the drawings in the embodiments of the present application.

The terms "first" and "second" in the embodiments of the present application are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, a feature defined as "first" and "second" may explicitly or implicitly include one or more of these features.

In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item (unit) of a, b or c can represent: a, b, c, a-b, a-c, b-c or a-b-c, wherein a, b, c can be single or multiple.

The concepts or terms involved in this application are firstly introduced below.

(1) The flow rate of the data stream, which may also be referred to as the length of the data stream, or simply referred to as the flow rate, refers to the number of messages (also referred to as data packets) included in a data stream.

(2) The quintuple is a set composed of five parameters: source IP address, source port number, destination IP address, destination port number and transport layer protocol. Among them, the source IP address refers to the IP address of the client that sends out the data flow, the source port number refers to the port number that the client sends out the data flow, the destination IP address refers to the IP address of the server that receives the data flow, and the destination port number refers to the port number that the server receives. The port number of the data stream. The transport layer protocol refers to the protocol used by the client and the server when transmitting the data stream. The five-tuple can distinguish different data streams, and the corresponding data stream is unique.

(3) INT is a traffic measurement protocol, which collects information (such as the identity document of INT network element) on the data plane for each INT network element (referring to network devices such as switches or routers supporting INT technology) to process packets. , ID), the port number of the packet entering and exiting the INT network element, the queue number of the packet entering and exiting the INT network element, the time of the packet entering and exiting the INT network element, etc.), and then the last INT network element collects the INT network elements process message information and send it to traffic analysis equipment, so as to realize fine-grained acquisition of network status. This process has no control and participation from the control plane, which can greatly reduce the pressure on the controller.

(4) The probabilistic data structure is a data structure based on hash (also called hash). The probabilistic data structure records the information of each data flow in the network at the cost of sacrificing certain traffic measurement accuracy instead of The information of each message can reduce memory overhead. The probabilistic data structure saves key-value data with the same hash value (also called hash value) in the first flow storage table (see Figure 1) by setting a hash function (also called a hash function). in the same cell. The statistical value in the storage cell is used as the traffic measurement result, which is an estimate of the real traffic of each data flow. Ignoring the occurrence of hash collisions (collision), the error can be well controlled under a certain threshold. These algorithms use less memory than error-free methods. In short, probabilistic data structures possess a theoretically provable balance of flow estimation accuracy and memory. Currently commonly used probabilistic data structures include: Bloom filter, cardinality estimation method, and sketch. Among them, sketch has a minimum sketch (count-min sketch, abbreviated as cm sketch), a conservative update sketch (conservative- update sketch (referred to as cu sketch), count sketch (count sketch), FlowRadar, UnivMon, etc. Currently, the most popular probabilistic data structure in the field of flow measurement is cm sketch.

(5) The first flow storage table, as shown in FIG. 1 , includes d storage layers (rows), and each storage layer includes w storage cells (also called hash buckets). The number of bits of the storage cells in the first flow storage table is δ, therefore, the flow range that each storage cell can record is the same. Each storage layer is associated with a hash function, and the hash functions h ₁₁ , h ₁₂ , . . . , h _1d associated with the d storage layers are independent in pairs. Wherein, both d and w are natural numbers greater than 0. Usually, δ is 8 bits (bit), that is to say, the flow range that the memory cell can record is 0-255. In the initial state, the initial values of all storage cells on the first flow storage table are 0 or empty.

(6) The working principle of cm sketch is: when a message including a flow identifier arrives at a device deployed with cm sketch, the device uses h ₁₁ , h ₁₂ , ..., h _1d to hash the flow identifier included in the message, In this way, d storage cells are located in the first flow storage table, and then the device increases the statistical value of the storage cell whose statistical value has not overflowed among the located d storage cells by 1, and the statistical value of the storage cell whose statistical value has overflowed remains unchanged, so as to obtain the updated first flow storage table. When the device performs the above operation on the last packet in a data flow, the minimum statistical value in the d storage cells is the device's estimated value of the real traffic of the data flow.

Among them, the statistical value has not overflowed, indicating that the statistical value is less than the maximum value that can be recorded in the cell where the statistical value is located, and the statistical value has overflowed, indicating that the statistical value is equal to the maximum value that can be recorded in the cell where the statistical value is located. For example, suppose cell A The statistical value A' of cell B is 100, the statistical value B' of cell B is 1000, and the maximum value that can be recorded in cells A and B is both 255. B' has overflowed.

The working principle of other probabilistic data structures is similar to that of cm sketch, and will not be repeated here.

Scenarios in which the flow measurement method, device, and related equipment provided in this application can be applied are introduced below.

Referring to Fig. 2, Fig. 2 is a kind of traffic measurement system involved in the present application, as shown in Fig. 2, the system includes: client 110, multiple forwarding network elements 120, server 130 and traffic analysis device 140, client 110 , the multiple forwarding network elements 120, the server 130 and the traffic analysis device 140 can communicate through a network, and the network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

The client 110 refers to the device that sends out the data flow. The data flow sent by the client 110 may be forwarded by some or all of the multiple forwarding network elements 120 , and finally reaches the server 130 . The client 110 can be a terminal device, such as a smart phone, a tablet computer, a mobile notebook, a wearable device, etc., or a server, such as a personal computer, a cloud server, etc. The operating system of the client 110 may be IOS, Android, Windows, Linux, etc., which is not specifically limited here.

The forwarding network element 120 is a network device, such as a switch or a router, for implementing data flow forwarding tasks.

The server 130 refers to a device receiving a data stream, which may be a terminal device, such as a smart phone, a tablet computer, a mobile notebook, a wearable device, etc., or may be a personal computer, a server, etc. The operating system of the server 130 may be IOS, Android, Windows, Linux, etc., which is not specifically limited here.

The traffic analysis device 140 is a device for implementing a traffic analysis function. The traffic analysis device 140 can monitor the transmission process of the data stream and obtain a traffic analysis result. After obtaining the traffic analysis result, the traffic analysis device 140 may receive the query request input by the user, and feed back the traffic query result to the user. The traffic analysis device 140 may be a personal computer, a server, and the like.

At present, the flow measurement system shown in Figure 2 usually uses the INT method to realize the flow measurement task. When using the INT method, the multiple forwarding network elements 120 in the system shown in Figure 2 are INT network elements, and the process of realizing traffic measurement in the system shown in Figure 2 is:

When a message is sent by the client 110 and arrives at the first forwarding network element 120, the first forwarding network element 120 will add a data flow identifier (hereinafter referred to as the flow identifier) to the message and the first forwarding network element 120 Information about processing packets. Wherein, the flow identification can be any one or more of the five parameters included in the quintuple, and can also be a VLAN identification (identity document, ID), etc., and the information for forwarding network element 120 to process the message can be forwarding network element 120 The ID of the packet, the port number of the packet entering and leaving the forwarding network element 120, the queue number of the packet entering and leaving the forwarding network element 120, the time when the packet enters and exits the forwarding network element 120, etc. Usually, the flow identifiers added by the first forwarding network element 120 to the packets belonging to the same data flow are the same.

After adding the flow identifier and the information of the first forwarding network element 120 to process the message, the first forwarding network element 120 sends the message to the transit network element 120 (meaning that the message arrives at the last forwarding network element 120 after the first forwarding network element 120 The forwarding network element 120 passed by the network element 120 before sends a message carrying information about the processing of the message by the first forwarding network element 120 . When the message is forwarded to the transit network element 120, the transit network element 120 continues to add the information of the message processed by the network element on the message, and then forwards the message to the next forwarding network with the added information of the message processed by the network element. Yuan 120 forwarding.

When the message is forwarded to the last forwarding network element 120, the last forwarding network element 120 processes the message through the forwarding network element 120 (that is, the first forwarding network element 120 and the transit network element 120) passed by the message carried by the message The information (hereinafter referred to as the first carrying information) is stripped from the message to obtain the independent first carrying information and the message, and then the first carrying information is sent to the traffic analysis device 140, and the message is forwarded to the server 130 .

In a specific implementation, the last forwarding network element 120 usually sends the multiple pieces of first carried information to the traffic analysis device 140 after stripping off multiple packets to obtain multiple pieces of first carried information. After receiving the plurality of first carried information, the traffic analysis device 140 performs analysis and statistics according to the plurality of first carried information, and obtains traffic statistics results, such as the traffic of each data flow, each traffic flow passing through each forwarding network element 120 data flow, etc.

When the above-mentioned INT method implements traffic measurement, the last forwarding network element 120 does not continue to add information about its own processing of the message to the message before sending the first carried information to the flow analysis device 140, that is, the first The carried information does not include the information of the last forwarding network element 120 processing the packet, and the first carried information sent by the last forwarding network element 120 received by the traffic analysis device 140 does not include the information of the last forwarding network element 120 processing the packet, so , this method has the problem that the measured link information is incomplete.

In addition, the last forwarding network element 120 sends the stripped multiple first carried information together to the traffic analysis device 140. If the data volume of the multiple first carried information is very large, this will not only consume a large amount of bandwidth resources, but also A large amount of computing resources of the traffic analysis device 140 will be consumed, and the measurement cost is relatively high.

In view of the above problems, the present application provides a flow measurement method, which can be applied to the flow measurement system shown in Figure 2, and the multiple forwarding network elements 120 in the system shown in Figure 2 are all supported in the data flow The network element that adds the information of processing the message to the message, such as the INT network element, has a probabilistic data structure deployed on the server 130, and the process of realizing flow measurement by the system shown in Figure 2 is as follows:

When a message is sent by the client 110 and arrives at the first forwarding network element 120, the first forwarding network element 120 will add a flow identifier and information about processing the message by the first forwarding network element 120 to the message.

After adding the flow identifier and the information of the first forwarding network element 120 processing the message, the first forwarding network element 120 sends a message carrying the information of the first forwarding network element 120 processing the message to the transit network element 120 . When the message is forwarded to the transit network element 120, the transit network element 120 continues to add the information of the message processed by the network element on the message, and then forwards the message to the next forwarding network with the added information of the message processed by the network element. Yuan 120 forwarding.

When the message is forwarded to the last forwarding network element 120, the last forwarding network element 120 continues to add the information of the message processed by the network element on the message, and then sends the message added with the information of the message processed by the network element to the message. The server 130 forwards.

When the message is forwarded to the server 130, the server 130 processes the forwarding network element 120 (that is, the first forwarding network element 120, the transit network element 120, and the last forwarding network element 120) that the message carried by the message passes through. The information of the text (hereinafter referred to as the second carried information) is stripped from the message to obtain the independent second carried information and the message.

After the server 130 strips off the second carried information, since the probabilistic data structure is deployed on the server 130, the server 130 can count the flow of the data flow to the first flow storage table according to the second carried information, and obtain the updated The first flow storage table. Then, the server 130 sends the obtained updated first traffic storage table to the traffic analysis device 140 .

In a specific implementation, after the server 130 strips off multiple packets to obtain a plurality of second carrying information, it can count the traffic of the data flow in the first flow storage table according to the multiple second carrying information, and obtain the updated The first flow storage table, the updated flow storage table may include the flow of each data flow, the flow of each data flow passing through each forwarding network element 120, and the like.

It can be seen that when the traffic measurement method provided in this application implements traffic measurement, the stripping operation of the second carried information is not performed by the last forwarding network element 120, but the stripping operation of the second carried information is performed by the server 130, Moreover, the second carried information stripped by the server 130 includes the information of the last forwarding network element 120 to process the message. Therefore, this method can solve the problem of incomplete measured link information existing in the existing INT method.

In addition, in the flow measurement method provided in the present application, the server 130 counts the flow of the data flow into the first flow storage table according to the second carrying information, and after obtaining the updated first flow storage table, the updated first flow storage table is A traffic storage table is sent to the traffic analysis device 140, unlike the existing INT method, the last forwarding network element 120 directly sends the first carried information to the traffic analysis device 140, and the traffic analysis device 140 performs the first carry information Analyze and summarize to obtain the traffic of the data stream. From the above introduction to the probabilistic data structure and the first flow storage table, it can be seen that using the first flow storage table to count the flow of the data flow can reduce the memory overhead. Therefore, the updated data in the first flow storage table obtained by the server 130 The amount of data is smaller than the second data carrying information, and the server 130 sends the updated first flow storage table to the flow analysis device 140. Compared with the prior art, it can reduce the consumption of sending data to the flow analysis device 140. bandwidth resources, and the updated first flow storage table already includes the flow of the data flow, and the flow analysis device 140 is not required to perform analysis, which can reduce the consumption of computing resources on the side of the flow analysis device 140 and reduce measurement costs.

In order to facilitate a clearer understanding of the traffic measurement method provided by this application, here the probability data structure deployed on the server 130 is cm sketch, and the sequence of messages sent by the last forwarding network element 120 received by the server 130 is P1, P2 , ..., Pn as examples, in conjunction with the schematic flow chart shown in Figure 3, the flow measurement method provided by the application is described in detail, as shown in Figure 3, the method includes:

S101. The server 130 receives a message Pi including the second carrying information Pi' sent by the last forwarding network element 120.

Wherein, i and n are both natural numbers, 1≤i≤n.

Specifically, after receiving the packet Pi including the second carrying information Pi', the server 130 will strip the packet Pi to obtain the second carrying information Pi', and the second carrying information Pi' includes the flow identification ID _Pi .

S102. The server 130 uses hash functions h ₁₁ , h ₁₂ , . . . , h _1d to hash the ID _Pi included in the second carried information Pi' to obtain d first hash values.

Wherein, the d first hash values are h ₁₁ (ID _Pi ), h ₁₂ (ID _Pi ), ..., h _1d (ID _Pi ).

S103. The server 130 determines the first target storage cell from the first flow storage table according to the d first hash values.

Specifically, the forwarding network element 120 may first determine d storage cells corresponding to ID _Pi from the first flow storage table: the first storage cell=h ₁₁ (ID _Pi )%w, the second storage cell=h ₁₂ (ID _Pi )%w,..., the dth storage cell=h _1d (ID _Pi )%w, then, the storage cell whose statistical value does not overflow in the d storage cells corresponding to ID _Pi is determined as the first target storage cell grid.

S104. The server 130 increases the statistical value of the first target storage cell by 1.

For example, assuming that the first traffic storage table included before the forwarding network element 120 receives the message Pi is table A in FIG. 255. After receiving _the message Pi, the forwarding network element 120 locates the five storage cells h ₁₁ (ID _Pi )%5=3, h ₁₂ (ID _Pi )%5=1, h ₁₃ (ID _Pi )%5=4, h ₁₄ (ID _Pi )%5=1, h ₁₅ (ID _Pi )%5=3, see Table B in Fig. 4, Table B The shaded table in the table indicates the located cells. From Table B, it can be seen that the cells whose statistical values have not overflowed are the third cell in the first storage layer and the fourth cell in the third storage layer. storage cell, the first storage cell in the fourth storage layer, and the third storage cell in the fifth storage layer, and their statistical values correspond to 3, 25, 80, 25, then the forwarding network element 120 determines the first The target storage cell is the above four storage cells, and then the statistical values of the above four storage cells are all increased by 1, and the updated first flow storage table is shown in Table C in FIG. 4 .

S105. The server 130 executes steps S101 to S104 for each received message to obtain an updated first flow storage table.

S106. The server 130 sends the updated first traffic storage table to the traffic analysis device 140.

In a specific embodiment of the present application, before the server 130 sends the updated first traffic storage table to the traffic analysis device 140, it can also encapsulate the updated first traffic storage table, so as to reduce the traffic from the server 130 The bandwidth occupied when the analysis device 140 sends the updated first flow storage table. Specifically, the server 130 may use an algorithm with an encapsulation function (such as elastic sketch) to encapsulate the updated first flow storage table.

After the traffic analysis device 140 receives the updated first traffic storage table sent by the server 130, it can realize the query of the traffic of a certain data flow, the query of the significant flow, and the query of the data flow received by the server 130 within a certain time window. number of messages or query the total number of messages received by the server 130 within a certain time window. Taking the traffic analysis device 140 to query the traffic of a certain data stream as an example, refer to FIG. 9 and the related description of FIG. 9 below for the process.

The flow measurement method provided in the present application can also obtain the flow of each data flow passing through each forwarding network element 120, and the implementation method is as follows:

It can be seen from the above that, in addition to the flow identifier, the second carried information received by the server 130 may also include information on packet processing by multiple forwarding network elements 120, and the information on packet processing by multiple forwarding network elements 120 may be The IDs of multiple forwarding network elements 120, therefore, the probabilistic data structure deployed on the server 130 can be configured to include multiple first flow storage tables as shown in Figure 1, and configure the multiple first flow storage tables to associate a Hash function f ₁ .

When the packet including the second carrying information arrives at the server end 130, the server end 130 can use the hash function _f1 to hash the ID of each forwarding network element 120 included in the second carrying information, thereby Locate the first traffic storage table corresponding to each forwarding network element 120 in a traffic storage table, and then perform steps S102 to S106, and the traffic analysis device 140 can obtain the updated first traffic storage table corresponding to each forwarding network element 120 A flow storage table, so as to realize the acquisition of the flow of each data flow passing through each forwarding network element 120 . Wherein, the formula for locating the first flow storage table T ₁ corresponding to each forwarding network element 120 among the multiple first flow storage tables is: T ₁ =f ₁ (the ID of the forwarding network element 120)%m ₁ , m ₁ represents the number of multiple first flow storage tables.

It can be understood that after receiving the updated first traffic storage table corresponding to each of the multiple forwarding network elements 120 sent by the server 130, the traffic analysis device 140 can query a certain data flow passing through a certain forwarding network element 120 tasks such as querying the number of data streams received by a certain forwarding network element 120 within a certain time window, or querying the total number of packets received by a certain forwarding network element 120 within a certain time window. Taking the traffic analysis device 140 to query the traffic of a certain data flow passing through a certain forwarding network element 120 as an example, refer to FIG. 10 and the related description of FIG. 10 below for the process.

From the above description of the first flow storage table, it can be seen that the number of bits δ of the storage cells included in the first flow storage table is the same, usually 8 bits. Therefore, it can be understood that in the flow measurement method provided by this application, if The probabilistic data structure deployed on the server 130 is the existing cm sketch or cu sketch, etc., and the flow measurement method provided by this application is used to measure the flow of large data flows or the flow of small data flows. The number of bits in the cell is the same.

However, in real network traffic, the distribution of traffic size is highly skewed, which means that most data flows are small data flows, and the number of packets included in this type of data flow is usually only a few to one or two hundred. It is called mouse flow, and only a very small number of data flows are large data flows. This type of data flow includes hundreds or even thousands of packets, and is usually called elephant flow. Therefore, compared with the INT method, the flow measurement method provided by the present application has already played a role in saving memory resources, but the utilization rate of memory resources is still not high enough, and the measurement is accurate when measuring the flow of large data streams. Sex is not high enough.

For example, assuming that the number of bits in the storage cells included in the first flow storage table is 8 bits, the number of packets included in the small data flow A is 2, the number of packets included in the large data flow B is 1000, and the forwarding network element 120 is in When measuring the flow of small data flow A, the statistical value is 2, and 2 is far less than the maximum value 255 that can be recorded in the storage cell. At this time, the memory is wasted, and the forwarding network element 120 is measuring the flow of large data flow B , the statistical value has overflowed after reaching 255. Therefore, the estimated value of the flow of the large data flow B by the forwarding network element 120 is 255, which is quite different from the real flow of 1000 of the large data flow B. The flow measurement of the large data flow B Inaccurate.

In order to solve the above problems, the present application also provides another flow measurement method, in which the probability data structure deployed on the server 130 is not the existing probability data structure, but a new probability data structure ( That is, the tower sketch described below), this probabilistic data structure can improve memory utilization and improve the accuracy of flow measurement of large data streams.

The tower sketch saves the key-value data with the same hash value in the same storage cell in the second traffic storage table by setting the hash function. The statistical value in the storage cell is used as the traffic measurement result, which is an estimate of the real traffic of each data flow.

The second flow storage table is shown in FIG. 5 . It can be seen that the second flow storage table is the same as the first flow storage table shown in FIG. 1 in that both include d storage layers. The difference between the second flow storage table and the first flow storage table is that each storage layer of the second flow storage table includes a different number of storage cells, and the number of bits of the storage cells included in different storage layers is different. The number of storage cells included in one storage layer is w ₁ , the number of bits is δ ₁ , the number of storage cells included in the second storage layer is w ₂ , the number of bits is δ ₂ ,..., the dth storage layer includes The number of storage cells is w _d , and the number of digits is δ _d . Therefore, storage cells belonging to different storage layers can record different flow ranges. Each storage layer is associated with a hash function, and the hash functions h ₂₁ , h ₂₂ , . . . , h _2d associated with the d storage layers are independent in pairs. In this embodiment, the count values of all storage cells in the second flow storage table shown in FIG. 5 are 0 or empty in an initial state.

The working principle of tower sketch is similar to the working principle of cm sketch mentioned above. For details, please refer to the relevant description of the working principle of cm sketch above.

The following uses the flow storage table 1 and flow storage table 2 shown in Figure 6 as an example to explain the principle that the tower sketch can improve memory utilization and flow measurement accuracy. Wherein, the flow storage table 1 shown in FIG. 6 is an exemplary first flow storage table, and the flow storage table 2 shown in FIG. 6 is an exemplary second flow storage table. It can be seen from Figure 6:

The flow storage table 1 includes 5 storage layers, the storage layer 11 to the storage layer 15 all include 8 storage cells, and the number of digits of the 40 storage cells is 8 bits, that is to say, the flow range that can be recorded by the 40 storage cells is equal to 0-255.

The flow storage table 2 also includes 5 storage layers, the storage layer 21 includes 32 storage cells, and the number of digits is 2 bits, that is, the storage cells included in the storage layer 21 can record the flow range from 0 to 3, and the storage layer 22 includes 16 storage cells, the number of bits is 4 bits, that is, the storage cells included in the storage layer 22 can record the flow range of 0 to 15, and the storage layer 23 includes 8 storage cells, the number of bits is 8 bits, that is, the storage layer 23 includes The flow range that can be recorded in the storage cells is 0~255. The storage layer 24 includes 4 storage cells, and the number of digits is 16 bits. That is, the flow range that can be recorded in the storage cells included in the storage layer 24 is 0~65535. It includes 2 storage cells, both of which are 32 bits, that is, the storage cells included in the storage layer 25 can record traffic in the range of 0 to 4294967295.

It can be calculated from the above description of the flow storage table 1 that the memory occupied by the flow storage table 1 = 40*8bit = 320bit, and it can be calculated from the above description of the flow storage table 2 that the memory occupied by the flow storage table 2 = 2* 32bit+4*16bit+8*8bit+16*4bit+32*2bit=320bit, that is to say, the memory occupied by the two is the same.

When the memory occupied by flow storage table 1 and flow storage table 2 is the same, since the number of bits of storage cells included in each storage layer of flow storage table 2 is different, the device uses flow storage table 2 to store small data When measuring the flow rate of a flow, you can use the storage cell matching the flow rate of the small data flow in the flow storage table 2 to record the flow rate of the small data flow. For the flow rate of the data stream, for example, when the flow rate of the small data stream is 12, use the storage cell with a memory occupation of 4 bits to record the flow rate of the data stream. Compared with using the flow storage table 1 for flow measurement, it reduces the waste of memory and improves the memory utilization.

At the same time, since the maximum value that can be recorded in the storage cell in the flow storage table 2 is 4294967295, when the device uses the flow storage table 2 to measure the flow of a large data flow, it can use the storage cell that matches the flow of the large data flow to record For the flow of large data streams, for example, when the flow rate of large data streams is 1000, use a storage cell with a memory occupation of 16 bits to record the flow of the data stream. Recording the flow of the data flow in a grid will not cause a large difference between the measured flow of the large data flow and the real flow, so the accuracy of the measured flow of the large data flow can be improved.

In addition, it can also be seen from FIG. 6 that the flow storage table 2 includes more cells than the flow storage table 1 . It can be understood that when the number of storage cells included in the flow storage table 2 is greater than the number of storage cells included in the flow storage table 1, the probability of a hash collision occurring when the device uses the flow storage table 2 for flow measurement will also increase. is lower than the probability of hash collision when using flow storage table 1 for flow measurement, therefore, the accuracy of flow measurement using flow storage table 2 is higher than that of using flow storage table 1 for flow measurement.

Here, taking the probabilistic data structure deployed on the server 130 as a tower sketch, and the sequence of packets sent by the last forwarding network element 120 received by the server 130 as an example, combined with the flow shown in FIG. 7 The schematic diagram describes in detail another flow measurement method provided by this application, as shown in Figure 7, the method includes:

S201. The server 130 receives a message Pi including the second carrying information Pi' sent by the last forwarding network element 120.

S202. The server 130 uses the hash functions h ₂₁ , h ₂₂ , . . . , h _2d to hash the ID _Pi included in the second carried information Pi' to obtain d second hash values.

Wherein, the d second hash values are h ₂₁ (ID _Pi ), h ₂₂ (ID _Pi ), ..., h _2d (ID _Pi ).

S203. The server 130 determines a second target storage cell from the second traffic storage table according to the d second hash values.

Specifically, the server 130 may first determine d storage cells corresponding to ID _Pi from the second flow storage table: the first storage cell=h ₂₁ (ID _Pi )% w ₁ , the second storage cell=h ₂₂ (ID _Pi )%w ₂ ,..., the dth storage cell=h _2d (ID _Pi )%w _d , and then store the d storage cells corresponding to ID _Pi that have the minimum statistical value and the minimum statistical value has not overflowed cell is determined as the second target cell.

S204. The server 130 increases the statistical value of the second target storage cell by 1.

S205. The server 130 executes steps S201 to S204 for each received message to obtain an updated second flow storage table.

S206. The server 130 sends the updated second traffic storage table to the traffic analysis device 140.

Another flow measurement method provided in this application can also obtain the flow of each data flow passing through each forwarding network element 120, and the implementation method is as follows:

It can be seen from the above that, in addition to the flow identifier, the second carried information received by the server 130 may also include information on packet processing by multiple forwarding network elements 120, and the information on packet processing by multiple forwarding network elements 120 may be IDs of multiple forwarding network elements 120, therefore, the probabilistic data structure deployed on the server 130 can be configured to include multiple second flow storage tables as shown in FIG. Hash function f ₂ .

When the packet containing the carrying information arrives at the server 130, the server 130 can use the hash function _f2 to hash the ID of each forwarding network element 120 included in the carrying information, so that the information in the multiple second traffic storage tables Locate the second traffic storage table corresponding to each forwarding network element 120, and then perform steps S202 to S206, and the traffic analysis device 140 can obtain the updated second traffic storage table corresponding to each forwarding network element 120 , so as to realize the acquisition of the traffic of each data flow passing through each forwarding network element 120 . Wherein, the formula for locating the second flow storage table T ₂ corresponding to each forwarding network element 120 among the plurality of second flow storage tables is: T ₂ =f ₂ (ID of the forwarding network element 120)%m ₂ , m ₂ represents the number of multiple second flow storage tables.

It can be understood that after the traffic analysis device 140 receives the updated second traffic storage table corresponding to each of the multiple forwarding network elements 120 sent by the server 130, it can query a certain data flow passing through a certain forwarding network element 120 tasks such as querying the number of data streams received by a certain forwarding network element 120 within a certain time window, or querying the total number of packets received by a certain forwarding network element 120 within a certain time window.

It can also be understood that the traffic measurement method provided by the present application can also realize obtaining the traffic of each data flow received by each port on each forwarding network element 120, obtaining the traffic of each data flow sent by each client 110, Obtain the flow of each data flow passing through each queue on each forwarding network element 120, and its specific implementation method is the same as the above-mentioned method of obtaining the flow of each data flow passing through each forwarding network element 120 Similarly, reference may be made to the relevant description above, and details will not be repeated here.

It should be noted that although the above descriptions of the flow measurement method provided by this application all take the server 130 as the execution subject, but in a specific embodiment of this application, the flow measurement system shown in Figure 2 can also be Add a statistical device 150 with strong computing power and storage capacity, as the execution subject of the traffic measurement method provided by this application, as shown in Figure 8, the statistical device 150 is connected with the last forwarding network element 120, traffic analysis device 140 and The server 130 is connected. In a specific implementation, the statistical device 150 may be a personal computer, a server, or the like.

When the statistical device 150 is the executor, the process of implementing the traffic measurement method provided in this application is similar to that when the server 130 is the executor. For details, please refer to the relevant description above, which will not be repeated here.

FIG. 9 is a schematic flow diagram of a traffic analysis device 140 provided in the present application to query the traffic of a certain data stream. As shown in FIG. 9 , the process includes the following steps:

S301. The traffic analysis device 140 receives a query request input by a user that includes an ID _f of a data flow f.

S302. The traffic analysis device 140 uses hash functions h ₁₁ , h ₁₂ , . . . , h _1d to hash ID _f , so as to determine d storage cells corresponding to ID _f in the updated first traffic storage table.

S303. The traffic analysis device 140 determines the minimum statistical value in the d storage cells corresponding to the ID _f as the traffic query result.

It can be understood that the traffic query result is an estimation of the traffic of the real data flow f.

S304. The traffic analysis device 140 feeds back the traffic query result to the user.

FIG. 10 is a schematic flow diagram of a traffic analysis device 140 provided in the present application to query the traffic of a certain data flow passing through a certain forwarding network element 120. As shown in FIG. 10 , the process includes the following steps:

S401. The traffic analysis device 140 receives a query request input by a user including an ID _e of a forwarding network element e and an ID _f of a data flow f.

S402. The traffic analysis device 140 uses the hash function _f1 to hash the ID _e , so as to determine a first traffic storage table corresponding to the ID _e among multiple updated first traffic storage tables.

S403. The traffic analysis device 140 uses hash functions h ₁₁ , h ₁₂ , ..., h _1d to hash ID _f , thereby determining d storage cells corresponding to ID _f in the first traffic storage table corresponding to ID _e .

S404. The traffic analysis device 140 determines the minimum statistical value in the d storage cells corresponding to the ID _f as the traffic query result.

S405. The traffic analysis device 140 feeds back the traffic query result to the user.

The flow measurement method provided by the present application has been described in detail above, and based on the same inventive concept, the flow measurement device provided by the present application will be further described below.

Referring to FIG. 11 , FIG. 11 is a schematic structural diagram of a flow measurement device 200 provided in the present application. When the execution body of the flow measurement method provided in the present application is the server 130 in the flow measurement system shown in FIG. 2 , the flow measurement The device 200 is applied to the server 130, and the flow measurement device 200 is applied to the statistical device 150 when the execution subject of the flow measurement method provided in this application is the statistical device 150 in the flow measurement system shown in FIG. 8 . As shown in Figure 11, the flow measurement device 200 includes:

(1) In the case where the flow measuring device 200 is applied to the server 130:

The receiving module 210 is configured to receive the message sent by the forwarding network element 120 connected to the server 130, the message includes the identifier of the data flow, and the message carries the multiple forwarding network The element 120 processes the information of the message, that is, the second carrying information mentioned above;

The acquiring module 220 is configured to acquire a traffic storage table, that is, a first traffic storage table or a second traffic storage table, where the traffic storage table is used to store the number of packets in the data stream;

A statistics module 230, configured to count the number of packets in the data flow into the traffic storage table according to the identifier of the data flow and the information on processing the packets by the plurality of forwarding network elements 120, and update them After the flow storage table;

The sending module 240 is configured to send the updated traffic storage table to the traffic analysis device 140 .

(2) In the case where the flow measuring device 200 is applied to the statistical device 150:

The receiving module 210 is configured to receive the message sent by the forwarding network element 120 connected to the statistical device 150, the message includes the identifier of the data flow, and the message carries the multiple forwarding network The element 120 processes the information of the message, that is, the second carrying information mentioned above;

The acquiring module 220 is configured to acquire a flow storage table, that is, a first flow storage table or a second flow storage table, and the flow storage table is used to store the number of packets in the data stream;

In a possible implementation manner, the flow measurement device 200 further includes: an encapsulation module 250, configured to encapsulate the updated flow storage table.

In a possible implementation manner, the information on processing the packet by each forwarding network element 120 includes one or more of the following combinations:

The identifier of each forwarding network element 120, the port number of each forwarding network element 120 receiving the message, the port number of each forwarding network element 120 sending the message, the message entering The queue number of the message, the queue number from which the message leaves, the time when each forwarding network element 120 receives the message, and the time when each forwarding network element 120 sends the message.

Specifically, for the implementation of various operations performed by the above-mentioned flow measurement device 200 , reference may be made to the description in the related content in the above-mentioned flow measurement method embodiment, and details are not repeated here for the sake of brevity.

It should be understood that the flow measurement device 200 is only one example provided herein, and that the flow measurement device 200 may have more or fewer components than those shown in FIG. 11 , two or more components may be combined, or It can be realized with different configurations of components.

The present application also provides a computing device cluster 30, and the computing device cluster 30 can be used to deploy the flow measuring device 200 shown in FIG. 11 to execute the flow measuring method provided in the present application. As shown in FIG. 12 , the computing device cluster 30 includes at least one computing device 300 .

Specifically, in the case that the computing device cluster 30 only includes one computing device 300, all the modules in the flow measurement device shown in FIG. 11 can be deployed in the one computing device 300: receiving module 210, acquiring module module 230 , sending module 240 and packaging module 250 .

In the case that the computing device cluster 30 includes multiple computing devices 300, each computing device 300 in the multiple computing devices 300 can be used to deploy some modules in the flow measuring device 200 shown in FIG. Two or more computing devices 300 in the computing devices 300 are jointly used to deploy one or more modules in the flow measuring device 200 shown in FIG. 11 .

For example, assuming that multiple computing devices 300 include a computing device 300A and a computing device 300B, the computing device 300A can be used to deploy the receiving module 210 and the acquiring module 220, and the computing device 300B can be used to deploy the statistics module 230, the sending module 240 and the The encapsulation module 250, or, the receiving module 210, the acquisition module 220, and the statistics module 230 are deployed on the computing device 300A, and the statistics module 230, the sending module 240, and the encapsulation module 250 are deployed on the computing device 300B; it is assumed that multiple computing devices 300 include the computing device 300A , 300B, 300C and 300D, then the computing device 300A can be used to deploy the receiving module 210, the computing device 300B can be used to deploy the acquiring module 220, the computing device 300C can be used to deploy the statistics module 230, and the computing device 300D can be used to deploy the sending module 240 and packaging module 250.

In a specific implementation, at least one computing device 300 included in the computing device cluster 30 may be all terminal devices, all may be cloud servers, or partly be cloud servers and partly be terminal devices, which are not specifically limited here.

More specifically, each computing device 300 in the computing device cluster 30 may include a processor 310, a memory 320, a communication interface 330, etc., and the memory 320 in one or more computing devices 300 in the computing device cluster 30 There may be stored the same codes (also referred to as instructions or program instructions) for executing the flow measurement method provided by the embodiments of the present application, and the processor 310 may read the codes from the memory 320 and execute the codes to realize the present application. In the traffic measurement method provided in the embodiment, the communication interface 330 may be used to implement communication between each computing device 300 and other devices.

In some possible implementation manners, each computing device 300 in the computing device cluster 30 may also communicate with other devices through a network connection. Wherein, the network may be a wide area network or a local area network or the like.

Taking all the modules of the flow measuring device 200 deployed on one computing device 300 as an example, the computing device 300 provided in the present application will be described in detail with reference to FIG. 13 .

Referring to FIG. 13 , the computing device 300 includes: a processor 310 , a memory 320 and a communication interface 330 , wherein the processor 310 , the memory 320 and the communication interface 330 may be connected to each other through a bus 340 . in,

The processor 310 can read the code stored in the memory 320, and cooperate with the communication interface 330 to execute some or all steps of the flow measurement method performed by the flow measurement device 200 in the above embodiments of the present application.

The processor 310 can have multiple specific implementation forms, for example, the processor 310 can be a central processing unit (central processing unit, CPU) or a graphics processing unit (graphics processing unit, GPU), and the processor 310 can also be a single-core processor or multi-core processor. The processor 310 may be a combination of a CPU and a hardware chip. The aforementioned hardware chip may be an application-specific integrated circuit (application-specific integrated circuit, ASIC), a programmable logic device (programmable logic device, PLD) or a combination thereof. The aforementioned PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a general array logic (generic array logic, GAL) or any combination thereof. The processor 310 may also be implemented solely by a logic device with built-in processing logic, such as an FPGA or a digital signal processor (digital signal processing, DSP).

The memory 320 can store codes as well as data. Among them, the code includes: the code of the receiving module 210, the code of the acquiring module 220, the code of the statistics module 230, the code of the sending module 240, the code of the packaging module 250, etc., and the data includes: the second carrying information, the first flow storage table, The updated first flow storage table, the second flow storage table, the updated second flow storage table, and so on.

In practical applications, the memory 320 can be a non-volatile memory, for example, a read-only memory (read-only memory, ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM) or flash memory. The memory 320 can also be a volatile memory, and the volatile memory can be a random access memory (random access memory, RAM), which is used as an external cache.

The communication interface 330 can be a wired interface (such as an Ethernet interface) or a wireless interface (such as a cellular network interface or using a wireless local area network interface) for communicating with other computing nodes or devices. When the communication interface 330 is a wired interface, the communication interface 330 can adopt a protocol family above the transmission control protocol/internet protocol (transmission control protocol/internet protocol, TCP/IP), for example, a remote function call (remote function call, RFC) protocol, simple object access protocol (simple object access protocol, SOAP) protocol, simple network management protocol (simple network management protocol, SNMP) protocol, common object request broker architecture (common object request broker architecture, CORBA) protocol and distributed protocol and many more.

The bus 340 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA for short) bus or the like. The bus 340 can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 13 , but it does not mean that there is only one bus or one type of bus.

The above-mentioned computing device 300 is used to execute the method in the above-mentioned embodiment of the flow measurement method, which belongs to the same concept as the above-mentioned method embodiment, and its specific implementation process is detailed in the above-mentioned method embodiment, and will not be repeated here.

It should be understood that the computing device 300 is only an example provided by the embodiment of the present application, and the computing device 300 may have more or fewer components than those shown in FIG. 13 , may combine two or more components, or It can be realized with different configurations of components.

The embodiment of the present application also provides a non-transitory computer-readable storage medium, in which codes are stored, and when it is run on a processor, the flow measurement method described in the above-mentioned embodiments can be realized some or all of the steps.

In the foregoing embodiments, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

In the above-mentioned embodiments, all or part may be implemented by software, hardware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product may comprise code. When the computer program product is read and executed by the computer, some or all steps of the method for distinguishing the heat of the data table described in the above method embodiments can be realized. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium, or a semiconductor medium.

The steps in the method of the embodiment of the present application can be adjusted in order, combined or deleted according to actual needs; the units in the device of the embodiment of the present application can be divided, combined or deleted according to actual needs.

The embodiments of the present application have been introduced in detail above, and specific examples have been used in this paper to illustrate the principles and implementation methods of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; meanwhile, for Those skilled in the art will have changes in specific implementation methods and application scopes based on the ideas of the present application. In summary, the contents of this specification should not be construed as limiting the present application.

Claims

A traffic measurement method, characterized in that it is applied to a transmission process in which data streams generated by a client are transmitted to a server through multiple forwarding network elements, the server is connected to a traffic analysis device, and the multiple forwarding network elements are Each forwarding network element is a network device that supports adding information for processing the message to the message in the data flow, and the method includes:

The server receives the message sent by the forwarding network element connected to it, and the message includes the identifier of the data flow and the information of processing the message by the multiple forwarding network elements;

The server obtains a flow storage table, and the flow storage table is used to store the number of packets in the data stream;

The server counts the number of packets in the data flow into the flow storage table according to the identifier of the data flow and the information on processing the packets by the plurality of forwarding network elements, and obtains the updated flow storage table;

The server sends the updated traffic storage table to the traffic analysis device.
The method according to claim 1, wherein before the server sends the updated traffic storage table to the traffic analysis device, the method further comprises:

The server encapsulates the updated flow storage table.
The method according to claim 1 or 2, wherein the identification of the data flow includes one or more combinations of the following: the IP address of the client's Internet protocol, the IP address of the client sending the data flow Port number, the IP address of the server, the port number of the server receiving the data flow, the transport layer protocol used by the client to transmit the data flow to the server, and the VLAN ID of the virtual local area network.
The method according to any one of claims 1 to 3, wherein the information on processing the message by each forwarding network element includes one or more of the following combinations:

The identifier of each forwarding network element, the port number of each forwarding network element receiving the message, the port number of each forwarding network element sending the message, and the queue number into which the message enters , the queue number from which the message leaves, the time when each forwarding network element receives the message, and the time when each forwarding network element sends the message.
A traffic measurement method, which is characterized in that it is applied to the transmission process of the data flow generated by the client through multiple forwarding network elements to the server, and the statistical device is connected to the multiple forwarding network elements to send the data flow to The last forwarding network element of the server, the statistics device is connected to a traffic analysis device, and each forwarding network element in the multiple forwarding network elements supports adding its processing described A network device for the information of the message, the method comprising:

The statistical device receives a message sent by the last forwarding network element, the message includes the identifier of the data flow, and the message carries information on processing the message by the multiple forwarding network elements;

The statistical device obtains a flow storage table, and the flow storage table is used to store the number of packets in the data flow;

The statistics device counts the number of packets in the data flow to the flow storage table according to the identifier of the data flow and the information on processing the packets by the plurality of forwarding network elements, and obtains the updated flow storage table;

The statistical device sends the updated traffic storage table to the traffic analysis device.
A flow measurement device, characterized in that it is applied to the transmission process in which the data generated by the client is transmitted to the server through a plurality of forwarding network elements, specifically applied to the server, and the server is also connected to a flow analysis device, Each forwarding network element in the plurality of forwarding network elements is a network device that supports adding information for processing the message to the message in the data flow, and the device includes:

The receiving module is configured to receive the message sent by the forwarding network element connected to the server, the message includes the identifier of the data flow, and the message carries the information processed by the multiple forwarding network elements information about the message;

An acquisition module, configured to acquire a flow storage table, where the flow storage table is used to store the number of packets in the data stream;

A statistics module, configured to count the number of packets in the data flow to the flow storage table according to the identifier of the data flow and the information on processing the packets by the multiple forwarding network elements, and obtain an updated Flow storage table;

A sending module, configured to send the updated flow storage table to the flow analysis device.
The device according to claim 6, wherein the device further comprises:

An encapsulation module, configured to encapsulate the updated flow storage table.
The device according to claim 6 or 7, wherein the identification of the data flow includes one or more combinations of the following: the IP address of the client, the port number through which the client sends the data flow , the IP address of the server, the port number of the server receiving the data flow, the transport layer protocol used by the client to transmit the data flow to the server, and the VLAN identifier.
The device according to any one of claims 6 to 8, wherein the information on processing the message by each forwarding network element includes one or more of the following combinations:

The identifier of each forwarding network element, the port number of each forwarding network element receiving the message, the port number of each forwarding network element sending the message, and the queue number into which the message enters , the queue number from which the message leaves, the time when each forwarding network element receives the message, and the time when each forwarding network element sends the message.
A flow measurement device, characterized in that it is applied to the transmission process of the data generated by the client through multiple forwarding network elements and transmitted to the server, and is specifically applied to statistical equipment, and the statistical equipment is connected to the multiple forwarding network elements Send the data flow to the last forwarding network element of the server, the statistical device is also connected to a traffic analysis device, and each forwarding network element in the multiple forwarding network elements supports A network device that adds information on processing the message to the message, and the device includes:

A receiving module, configured to receive a message sent by the last forwarding network element, the message includes the identifier of the data flow, and the message carries information on processing the message by the multiple forwarding network elements ;

An acquisition module, configured to acquire a flow storage table, where the flow storage table is used to store the number of packets in the data stream;

A statistics module, configured to count the number of packets in the data flow to the flow storage table according to the identifier of the data flow and the information on processing the packets by the multiple forwarding network elements, and obtain an updated Flow storage table;

A sending module, configured to send the updated flow storage table to the flow analysis device.
A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores instructions, and the instructions are used to implement the method according to any one of claims 1 to 4.
A computing device cluster, characterized in that it includes at least one computing device, each computing device includes a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the cluster of computing devices executes the method according to any one of claims 1-4.