WO2022111596A1

WO2022111596A1 - Data stream measurement method and apparatus

Info

Publication number: WO2022111596A1
Application number: PCT/CN2021/133234
Authority: WO
Inventors: 许延伟; 黄群; 章瑞
Original assignee: 华为技术有限公司
Priority date: 2020-11-30
Filing date: 2021-11-25
Publication date: 2022-06-02
Also published as: CN114584493A

Abstract

Disclosed in an embodiment of the present application are a data stream measurement method and apparatus, which belong to the technical field of communication. The method is applicable to network devices, and comprises: receiving a target data packet, and for each data row of a target data table, on the basis of a first feature value calculation function and a stream identifier that corresponds to the target data packet, determining a corresponding data storage space in the data rows for a target data stream to which the target data packet belongs; if the target data stream is a first data stream that corresponds to the data storage space, then on the basis of a measurement value of the target data packet, determining the measurement value of the target data stream, and recording same in the data storage space; and if the target data stream is not the first data stream that corresponds to the data storage space, then on the basis of the measurement value of the target data packet, recording the measurement value of the target data stream in the target data table. By using the present application, errors in the measurement value of a data stream can be relatively small.

Description

Method and apparatus for data flow measurement

This application claims the priority of the Chinese Patent Application No. 202011383567.3 filed on November 30, 2020 and entitled "Method and Device for Data Flow Measurement", the entire contents of which are incorporated herein by reference.

technical field

The present application relates to the field of communication technologies, and in particular, to a method and apparatus for data flow measurement.

Background technique

In the field of communication, data flow measurement plays a crucial role in analyzing the quality of communication links and formulating various communication strategies. The data flow measurement may include measurement of the number of packets of the data flow, measurement of the number of bytes of the data flow, and the like.

At present, the sketch algorithm is widely used in traffic measurement. Specifically, a sketch data table with m rows and n columns is set up, each grid in the sketch data table corresponds to a bucket, each data row in the sketch data table corresponds to a hash function, and each data row corresponds to a hash function different. When a data packet is received, for each data row of the sketch data table, use the hash function corresponding to the data row to perform hash calculation with the flow identifier corresponding to the data packet, and the obtained hash result is used for the target data packet. The measurements of are accumulated into the bucket corresponding to the hash result in the row of data. In this way, the measurement value of the data flow recorded in each bucket of the sketch data table is the sum of the measurement values corresponding to the flow IDs with the same hash result. When querying the measurement value corresponding to the specified flow ID, the specified flow ID is hashed with the hash function corresponding to each data row, and m buckets that record the measurement value corresponding to the specified flow ID can be determined. Then, the minimum measurement value recorded in the m buckets is taken as the measurement value corresponding to the specified flow identifier.

However, the error of the measured value of any data stream obtained through the sketch data table is related to the sum of the measured values of all data streams recorded in the sketch data table. The greater the error in the measurement value.

SUMMARY OF THE INVENTION

The embodiment of the present application provides a method and device for measuring data flow, which can solve the problem of large error in the measurement value of data flow in the related art. The technical solution is as follows:

In a first aspect, a data flow measurement method is provided, the method is applied to a network device, and the method includes:

Receiving the target data packet, for each data row of the target data table, based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet, determine that the target data stream to which the target data packet belongs is in the data The corresponding data storage space in the row. If the target data stream is the first data stream corresponding to the data storage space, the measurement value of the target data stream is determined based on the measurement value of the target data packet and recorded in the data storage space. If the target data stream is not the first data stream corresponding to the data storage space, the measurement value of the target data stream is recorded in the target data table based on the measurement value of the target data packet.

In the solution shown in the embodiment of the present application, the flow identifier can be set according to actual requirements and used to uniquely identify a data stream. For example, the flow identifier can be the source address and destination address carried in the data packet of the data stream, and for example, The flow identifier may also be quintuple information carried in the data packet of the data flow. A data stream includes multiple data packets, and the measurement value of the data stream is the sum of the measurement values of all the data packets included in the data stream. Specifically, the measurement value may be the number of data packets included in the data stream, and may also be the total number of bytes of the data packets included in the data stream, and so on. The above-mentioned first data stream may also be referred to as a large stream, and a large stream is relative to a data storage space.

The target data table is used to record data such as the flow identifier of the data flow, the measurement value of the data flow, and the like. For example, the target data table may be a sketch data table. Each data row of the target data table may correspond to a first eigenvalue calculation function, and different data rows correspond to different first eigenvalue functions. Wherein, the first feature value calculation function may be a hash function. For each data row, the first eigenvalue calculation function corresponding to the data row can have m types of calculation results, where m is the number of columns of the target data table, that is, the data storage space corresponding to each data row number. Each of the m types of calculation results corresponds to one data storage space of the data row, that is, the m types of calculation results correspond to the m data storage spaces one-to-one. The data storage space may also be called a bucket, and in the case that the target data table is a sketch data table, the data storage space corresponding to the target data table may be called a sketch bucket.

Through the solution shown in the embodiment of the present application, after receiving a data packet, it is first determined whether the data flow to which the data packet belongs is a large flow, and if it is a large flow, its flow identification and measurement value are recorded separately, which can make the large flow The measurements are relatively accurate. And since the measured values of the large flow are recorded separately, only the measured values of the data flow other than the large flow can be recorded in the target data table. In this way, because the number of recorded data flows is reduced, when restoring the measured values of these data flows , the accuracy of the obtained measurements is also relatively high.

In a second aspect, a data flow measurement method is provided, the method is used in a network device, and the method includes:

Receiving the target data packet, for each data row of the target data table, based on the first eigenvalue calculation function corresponding to the data row and the flow identifier corresponding to the target data packet, determine that the target data stream to which the target data packet belongs is corresponding to the data row data storage space. Based on the measurement value corresponding to the target data packet, the measurement value of the target data stream is recorded in the data storage space. If the target data stream is the first data stream corresponding to the data storage space, the measurement value of the target data stream is determined based on the measurement value of the target data packet and recorded in the data storage space.

With reference to the first aspect or the second aspect, in a possible implementation manner, if the target data stream is the first data stream corresponding to the data storage space, then the measurement value of the target data stream is determined based on the measurement value of the target data packet , recorded in the data storage space, including:

If the flow identifier and measurement value of the first data flow are not recorded in the data storage space, then determine that the target data flow is the first data flow corresponding to the data storage space, and use the measurement value of the target data packet as the measurement of the target data flow value, take the stream identifier of the target data stream as the stream identifier of the first data stream corresponding to the data storage space, and record the stream identifier of the target data and the measurement value in the data storage space correspondingly.

If the flow ID and measurement value of the first data flow are recorded in the data storage space, and the flow ID of the target data flow is the same as the flow ID of the first data flow recorded in the data storage space, the measurement value of the target data packet is The value is accumulated to the measured value of the first data stream recorded in the data storage space.

If the stream identifier and measurement value of the first data stream are recorded in the data storage space, and the stream identifier of the target data stream is different from the stream identifier of the first data stream recorded in the data storage space, determine whether the data storage space has The comparison measurement value is recorded. If there is no comparison measurement value recorded, the measurement value of the target packet is recorded in the data storage space as the comparison measurement value corresponding to the data storage space. If the comparison measurement value is recorded, the target The measurements of the data packets are accumulated on the comparison measurements recorded in the data storage space.

If the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the first data stream recorded in the data storage space, the target data stream is determined to be the first data stream corresponding to the data storage space, and the data storage The flow identification of the first data flow recorded in the space is updated to the flow identification of the target data flow, the measured value of the first data flow recorded in the data storage space is updated to the measured value of the target data packet, and the measured value of the first data flow recorded in the data storage space is updated. The recorded comparison measurement is updated to 0.

In the solution shown in the embodiment of the present application, for a data storage space of the target data table, the first data stream A that enters the data storage space for recording can be considered as a large stream of the data storage space.

After the second data stream B enters the data storage space, if the measurement value of the data stream B is greater than the measurement value of the data stream A, then replace the large stream of the data storage space with the data stream B. Next, when After the third data stream C enters the data storage space, compare the measured values of the data stream C and the data stream B to determine whether to replace the large stream.

If the measurement value of the data stream B is not greater than the measurement value of the data stream A, it is considered that the data stream A is still the large stream of the data storage space. Next, after the third data stream C enters the data storage space, Compare the measurement value of data stream A and the sum of the measurement values of data stream B and data stream C, if the sum of the measurement values is greater than the measurement value of data stream A, replace the large stream of the data storage space with data stream C , if the sum of the measured values is not greater than the measured value of data stream A, it is considered that data stream A is still a large stream in the data storage space.

That is, for a data storage space, its large stream can be replaced and updated.

In combination with the first aspect, in a possible implementation, if the flow identifier and measurement value of the first data stream are recorded in the data storage space, and the flow identifier of the target data stream is the same as the first data stream recorded in the data storage space. The flow identification of the data flow is different, and the method also includes:

If the accumulated comparative measurement value recorded in the data storage space is smaller than the measurement value of the first data stream recorded in the data storage space, it is determined that the target data stream is not the first data stream corresponding to the data storage space, based on the target data The measurement value of the packet, in the target data table, records the measurement value of the target data stream.

In the solution shown in the embodiment of the present application, for a large flow, the measured value and the flow identifier are recorded correspondingly. If it is determined that the target data flow is not a large flow, other methods are used to record in the target data table. In this way, when the measurement value of the non-large flow is subsequently restored, it is not affected by the measurement value of the large flow, so that the finally obtained statistical measurement value of the non-large flow is more accurate.

With reference to the first aspect or the second aspect, in a possible implementation manner, if the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the first data stream recorded in the data storage space, then The controller sends the storage space identifier of the data storage space, the flow identifier of the first data stream recorded in the data storage space, and the measurement value of the first data stream.

If the accumulated comparative measurement value recorded in the data storage space is smaller than the measurement value of the first data stream recorded in the data storage space, the flow identifier corresponding to the target data packet is sent to the controller as the flow identifier of the second data stream device. The second data stream may also be referred to as a non-large stream.

In addition, every preset time period, the locally stored target data table is sent to the controller, and the locally stored target data table is initialized.

In the solution shown in the embodiment of the present application, for a data storage space, after the large flow is replaced, the flow identifier and measurement value of the replaced large flow are uploaded to the controller, so that the subsequent controller can calculate the large flow. A measure of the flow total. In addition, when it is determined that the target data stream is not a large stream, the stream identifier of the target data stream is uploaded to the controller, so that the subsequent controller can know which data stream needs to be calculated by the optimization algorithm in the target data table.

After the above data is reported to the controller, the controller can restore the measured value of the data flow, which can effectively save the processing resources of the network equipment. And after the report, the local related data of the network device can be deleted, saving storage resources.

With reference to the first aspect or the second aspect, in a possible implementation manner, before sending the flow ID corresponding to the target data packet as the flow ID of the second data flow to the controller, the method further includes: determining that the flow ID corresponding to the target data packet is not sent to the controller The controller sends the flow ID corresponding to the target data packet, and records the flow ID corresponding to the target data packet.

In the solution shown in the embodiment of the present application, in order to avoid repeated recording or repeated transmission of the stream identifiers of non-large streams, when it is determined that the target data stream is not a large stream, it is possible to first determine whether the stream identifier of the target data stream has been uploaded. to the controller. If it has already been uploaded, it will not be repeated.

In combination with the first aspect or the second aspect, in a possible implementation manner, it is determined that the flow identifier corresponding to the target data packet is not sent to the controller, and the flow identifier corresponding to the target data packet is recorded, including:

It is determined that the flow identifier corresponding to the target data packet is not recorded in the locally stored Bloom filter, and the flow identifier corresponding to the target data packet is recorded in the Bloom filter.

In the solution shown in the embodiment of the present application, in order to save storage resources as much as possible while recording the flow identifiers of non-large flows, the flow identifiers of non-large flows may not be recorded directly, but Bloom filtering may be used. The Bloom Filter implements the recording of non-large flow measurements. Specifically, a Bloom Filter can be set, and before recording or sending a non-large stream stream identifier, you can first determine whether the stream identifier is recorded in the Bloom Filter, and if the stream identifier is not recorded, then record the stream identifier. or send. After recording or sending the stream identifier of the non-large stream, the stream identifier can be recorded in the Bloom Filter.

In addition, a Bloom Filter can be set for each data storage space in the target data table, or a Bloom Filter can be set for a target data table. In the case where a Bloom Filter is set in each data storage space, the Bloom Filter can be stored in the corresponding data storage space. Before recording or sending the flow ID of the non-large flow, the corresponding flow ID of the non-large flow can be determined first. Whether the stream identifier is recorded in the Bloom Filter stored in the data storage space of the data storage space, after recording or sending the stream identifier of the non-large stream, the stream identifier can be recorded in the Bloom Filter stored in the corresponding data storage space.

With reference to the first aspect, in a possible implementation manner, based on the measurement value of the target data packet, in the target data table, recording the measurement value of the target data stream, including:

Based on the second eigenvalue calculation function corresponding to the data row and the flow identifier of the target data packet, the accumulation coefficient corresponding to the target data flow is determined, and the measurement value of the target data packet is multiplied by the accumulation coefficient to obtain the adjusted measurement value of the target data packet. The adjusted measurement value of the target data packet is accumulated to the measurement value of the second data stream recorded in the data storage space.

In the solutions shown in the embodiments of the present application, the second data stream may be referred to as a non-large stream. For each data row of the target data table, in addition to corresponding to a first eigenvalue calculation function, it may also correspond to a second eigenvalue calculation function. The second feature value calculation function may also be a hash function, and the second feature value calculation function corresponding to each data row is different.

For each data storage space corresponding to the target data stream determined above, when recording the measurement value of the target data stream based on the measurement value of the target data packet, the value to be calculated corresponding to the target data packet can be input into the data storage space In the second eigenvalue calculation function corresponding to the data row, the accumulation coefficient corresponding to the stream identifier is obtained. Among them, the accumulation coefficient can be a decimal. Then, the measured value of the target data packet is multiplied by the accumulation coefficient to obtain the adjusted measured value of the target data packet. Finally, the adjusted measurement of the target packet is accumulated to the measurement of the non-large flow recorded in the data storage space.

It can be seen that in the present application, when recording the measured value of non-large flow in the target data table, it is multiplied by the corresponding accumulation coefficient and then recorded. In this way, when the measured value of the data flow is recovered by using the solution optimization problem subsequently, the constructed equation system can be made. Constrained by these cumulative coefficients, the number of solutions of the equations is relatively small, and accordingly, the error of the final solution (statistical measurement value of each non-large flow) is relatively small.

With reference to the second aspect, in a possible implementation manner, based on the measurement value corresponding to the target data packet, in the data storage space, recording the measurement value of the target data stream, including:

Based on the second eigenvalue calculation function corresponding to the data row and the flow identifier of the target data packet, an accumulation coefficient corresponding to the target data flow is determined, and the measurement value of the target data packet is multiplied by the accumulation coefficient, The adjusted measurement value of the target data packet is obtained, and the adjusted measurement value of the target data packet is accumulated to the measurement value of the data flow recorded in the data storage space.

In a third aspect, a data flow measurement method is provided, applied to a network device, and the method includes:

Receive the target data packet, and determine the first data storage space corresponding to the target data flow to which the target data packet belongs in the corresponding first data table based on the third feature value calculation function and the flow identifier corresponding to the target data packet. If the target data stream is a large stream corresponding to the first data storage space, determine the measurement value of the target data stream based on the measurement value of the target data packet, and record it in the first data storage space . If the target data stream is not the first data stream corresponding to the first data storage space, the measurement value of the target data stream is recorded in the target data table based on the measurement value of the target data packet.

In a possible implementation manner, if the target data stream is the first data stream corresponding to the first data storage space, the measurement value of the target data stream is determined based on the measurement value of the target data packet , recorded in the first data storage space, including:

If the first data storage space is empty, determine that the target data flow is the first data flow corresponding to the first data storage space, and use the measurement value of the target data packet as the measurement of the target data flow value, take the stream identifier of the target data stream as the stream identifier of the first data stream corresponding to the first data storage space, and record the stream identifier and measurement value of the target data correspondingly in the first data storage space middle.

If the first data storage space is not empty and the flow ID of the target data flow is the same as the flow ID of the first data flow recorded in the first data storage space, the measurement value of the target data packet is Accumulated to the measured value of the first data stream recorded in the first data storage space.

If the first data storage space is not empty and the stream identifier of the target data stream is different from the stream identifier of the first data stream recorded in the first data storage space, it is determined that the first data storage space is in the first data storage space. Whether a comparison measurement value is recorded, if no comparison measurement value is recorded, the measurement value of the target data packet is recorded in the first data storage space as the comparison measurement value corresponding to the first data storage space, if If the comparison measurement value is recorded, the measurement value of the target data packet is accumulated on the comparison measurement value recorded in the first data storage space.

If the accumulated comparative measurement value recorded in the first data storage space is greater than the measurement value of the first data stream recorded in the first data storage space, the target data stream is determined to be the first data storage space For the first data stream corresponding to the space, update the stream identifier of the first data stream recorded in the first data storage space to the stream identifier of the target data stream, and update the first data stream recorded in the first data storage space with the stream identifier of the target data stream. The measurement value of the data stream is updated to the measurement value of the target data packet, and the comparison measurement value recorded in the first data storage space is updated to 0.

In a possible implementation manner, if the first data storage space is not empty and the stream identifier of the target data stream is different from the stream identifier of the first data stream recorded in the first data storage space, the The method also includes:

If the accumulated comparative measurement value recorded in the first data storage space is smaller than the measurement value of the first data stream recorded in the first data storage space, it is determined that the target data stream is not the first data storage space The first data stream corresponding to the space records the measurement value of the target data stream in the target data table based on the measurement value of the target data packet.

In a possible implementation manner, based on the measurement value of the target data packet, in the target data table, recording the measurement value of the target data stream, including:

For each data row of the target data table, based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet, determine that the target data flow to which the target data packet belongs is in the data row In the second data storage space corresponding to the data row, based on the second feature value calculation function corresponding to the data row and the flow identifier of the target data packet, determine the accumulation coefficient corresponding to the target data flow, and calculate the value of the target data packet. The measured value is multiplied by the accumulation coefficient to obtain the adjusted measured value of the target data packet, and the adjusted measured value of the target data packet is accumulated to the measured value of the second data stream recorded in the second data storage space .

In a possible implementation manner, if the accumulated comparative measurement value recorded in the first data storage space is greater than the measurement value of the first data stream recorded in the first data storage space, the method further includes :

sending the stream identifier of the first data stream and the measured value of the first data stream correspondingly recorded in the first data storage space to the controller;

If the accumulated comparative measurement value recorded in the first data storage space is smaller than the measurement value of the first data stream recorded in the first data storage space, the method further includes:

sending the flow identification corresponding to the target data packet to the controller as the flow identification of the second data flow;

Methods also include:

Every preset time period, the locally stored first data table and the target data table are sent to the controller, and the locally stored first data table and the target data table are initialized.

In a possible implementation manner, before sending the flow identifier corresponding to the target data packet as the flow identifier of the second data stream to the controller, the method further includes:

It is determined that the flow identifier corresponding to the target data packet is not sent to the controller, and the flow identifier corresponding to the target data packet is recorded.

In a possible implementation manner, it is determined that the flow identifier corresponding to the target data packet is not sent to the controller, and the flow identifier corresponding to the target data packet is recorded, including:

In a fourth aspect, a data flow measurement method is provided, the method is applied to a controller, and the method includes:

Receive the flow identifier of the first data stream, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the stream identifier of the second data stream, and the target data table sent by the network device. In the measurement values of the first data stream recorded in the target data table, the measurement values of the first data streams corresponding to the same flow identifier are filtered to obtain the first measurement value of each first data stream recorded in the target data table . In the received measured values of the first data stream, the measured values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain a second received stream identifier corresponding to each first data stream. Measurements. In the received second measurement value corresponding to the flow identifier of each first data stream, the second measurement value of the first data stream corresponding to the same flow identifier is screened to obtain the received value of each first data stream. The flow identifies the corresponding third measurement value. In the first measurement value of each first data stream recorded in the target data table and the third measurement value corresponding to the received stream identifier of each first data stream, the first measurement value corresponding to the same stream identifier is Accumulate with the third measurement value to obtain the statistical measurement value corresponding to the stream identifier of each first data stream. Based on the received flow identification of the second data flow and the target data table, a statistical measurement value corresponding to the flow identification of each second data flow is determined.

In a possible implementation manner, among the measured values of the first data stream recorded in the target data table, the measured values of the first data stream corresponding to the same flow identifier are screened to obtain the measured values of the first data stream recorded in the target data table. First measurements for each first data stream, including:

Among the measured values of the first data stream recorded in the target data table, the maximum measured value of the first data stream corresponding to the same flow identifier is taken as the first measured value of the first data stream corresponding to the same stream identifier.

In the received second measurement value corresponding to the flow identifier of each first data stream, the second measurement value of the first data stream corresponding to the same stream identifier is screened to obtain each received first data stream. The third measurement value corresponding to the flow identifier of the flow, including:

In the received second measurement value corresponding to the flow identifier of each first data stream, the maximum second measurement value of the first data stream corresponding to the same flow identifier is used as the first data stream corresponding to the same stream identifier. Three measurements.

In a possible implementation manner, determining the statistical measurement value corresponding to the flow identifier of each second data stream based on the received stream identifier of the second data stream and the target data table, including:

For each data storage space in the target data table, based on the first eigenvalue calculation function corresponding to the data row where the data storage space is located, the data storage The stream identifier of the second data stream corresponding to the space, and based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located, determine the accumulation coefficient corresponding to the stream identifier of each second data stream corresponding to the data storage space . The statistical measurement value corresponding to the stream identifier of each second data stream corresponding to the data storage space is multiplied by the corresponding accumulation coefficient as an unknown, and the accumulation is performed, and the measurement value of the data stream recorded in the data storage space is used as the accumulation result. , to obtain the target equation corresponding to the data storage space. Based on the target equations corresponding to each data storage space in the target data table, a target equation group is formed. Based on a linear optimization algorithm, taking the minimum sum of the statistical measurement values corresponding to the received flow identifiers of the second data stream as the optimization objective, solving the objective equation system to obtain the statistical measurement values corresponding to the flow identifiers of each second data stream .

In this application, when recording the measured value of non-large flow in the target data table, it is multiplied by the corresponding accumulation coefficient and then recorded. Correspondingly, when constructing the above target equation, the unknowns are also multiplied by the corresponding accumulation coefficient. The above processing can make the number of solutions of the equation system relatively small, so that the error of the finally obtained solution (statistical measurement value of each non-large flow) is relatively small.

In a possible implementation manner, the method further includes: in the obtained statistical measurement value corresponding to the flow ID of the first data stream and the statistical measurement value corresponding to the flow ID of the second data stream, assigning the corresponding same flow ID The measured values are accumulated to obtain the total measured value corresponding to each flow identifier.

In a fifth aspect, a data flow measurement method is provided, the method is applied to a controller, and the method includes:

Receive the flow identifier of the first data stream, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the stream identifier of the second data stream, and the target data table sent by the network device. Among the measured values of the first data stream recorded in the target data table, the measured values of the first data streams corresponding to the same flow identifier are filtered to obtain the first measured value of each first data stream recorded in the target data table. In the received measured values of the first data stream, the measured values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain a second received stream identifier corresponding to each first data stream. Measurements. In the received second measurement value corresponding to the flow identifier of each first data stream, the second measurement value corresponding to the stream identifier of the first data stream corresponding to the same flow identifier is screened to obtain each received first data stream. The flow identification of a data flow corresponds to the third measurement value. In the first measurement value of each first data stream recorded in the target data table and the third measurement value corresponding to the received stream identifier of each first data stream, the first measurement value corresponding to the same stream identifier and the third measurement value corresponding to the same stream identifier The two measurement values are accumulated to obtain a statistical measurement value corresponding to the stream identifier of each first data stream. Based on the received flow identifier of the second data stream, the measurement value of the data stream recorded in each data storage space in the target data table, and the obtained statistical measurement value corresponding to the stream identifier of each first data stream, determine each The statistical measurement values corresponding to the flow identifiers of the two data streams.

In a possible implementation manner, based on the received stream identifier of the second data stream, the measured value of the data stream recorded in each of the data storage spaces in the target data table, and each obtained first data stream The statistical measurement value corresponding to the flow identifier of the second data stream is determined, and the statistical measurement value corresponding to the flow identifier of each second data stream is determined, including:

For each data storage space of the target data table, the accumulation coefficient corresponding to the stream identifier of each first data stream corresponding to the data storage space is determined based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located. Multiply the statistical measurement value corresponding to the flow identifier of each first data stream corresponding to the data storage space by the corresponding accumulation coefficient to obtain the adjusted statistical measurement value corresponding to the flow identifier of each first data stream corresponding to the data storage space . Based on the first eigenvalue calculation function corresponding to the data row where the data storage space is located, in the received flow identifier of the second data stream, the stream identifier of the second data stream corresponding to the data storage space is determined, and based on the received stream identifier of the second data stream The second eigenvalue calculation function corresponding to the data row where the data storage space is located determines the accumulation coefficient corresponding to the stream identifier of each second data stream corresponding to the data storage space. The statistical measurement value corresponding to the flow identification of each second data stream corresponding to the data storage space is multiplied by the corresponding accumulation coefficient as an unknown, and the adjustment statistical measurement corresponding to the flow identification of each first data flow corresponding to the data storage space The values are added, and the measured value of the data stream recorded in the data storage space is used as the accumulation result to obtain the target equation corresponding to the data storage space pair. Based on the target equations corresponding to each data storage space in the target data table, a target equation group is formed. Based on a linear optimization algorithm, taking the minimum sum of the statistical measurement values corresponding to the received flow identifiers of the second data stream as the optimization objective, solving the objective equation system to obtain the statistical measurement values corresponding to the flow identifiers of each second data stream .

In a sixth aspect, a data flow measurement method is provided, the method is applied to a controller, and the method includes:

Receive the flow identifier and measurement value of the first data stream, the stream identifier of the second data stream, the target data table, and the first data table sent by the network device. In the received measurement value of the first data stream and the measurement value recorded in the first data table, the measurement values corresponding to the same flow identifier are accumulated to obtain the statistical measurement value corresponding to the stream identifier of each first data stream . Based on the received stream identifiers of the second data streams and the target data table, a statistical measurement value corresponding to the stream identifiers of each second data stream is determined.

In a possible implementation manner, based on the received stream identifier of the second data stream and the target data table, determining a statistical measurement value corresponding to the stream identifier of each second data stream, including:

For each data storage space in the target data table, based on the first eigenvalue calculation function corresponding to the data row where the data storage space is located, in the received stream identifier of the second data stream, determine the corresponding data The stream identifier of at least one target second data stream in the storage space, and the accumulation coefficient corresponding to the stream identifier of each target second data stream is determined based on the second feature value calculation function corresponding to the data row where the data storage space is located. The statistical measurement value corresponding to the stream identifier of each target second data stream in the stream identifiers of the at least one target second data stream is multiplied by the corresponding accumulation coefficient as an unknown, and the accumulation is performed, and the data storage space is recorded. The measured value of the data stream is used as the accumulation result to obtain the target equation;

Based on the target equations corresponding to each data storage space in the target data table, a target equation group is formed. Based on a linear optimization algorithm, taking the minimum sum of the statistical measurement values corresponding to the received flow identifiers of the second data stream as the optimization objective, solving the objective equation system to obtain the statistical measurement values corresponding to the flow identifiers of each second data stream .

In a possible implementation, the method further includes:

In the obtained statistical measurement values corresponding to the flow ID of the first data stream and the statistical measurement values corresponding to the flow ID of the second data stream, the measurement values corresponding to the same flow ID are accumulated to obtain the total measurement value corresponding to each flow ID value.

In a seventh aspect, an apparatus for measuring data flow is provided, which is used to execute the first aspect or any possible implementation manner of the first aspect. Specifically, the apparatus includes a module for performing the first aspect or any possible implementation manner of the first aspect.

In an eighth aspect, an apparatus for data flow measurement is provided, which is used to execute the second aspect or any possible implementation manner of the second aspect. Specifically, the apparatus includes a module for performing the second aspect or any possible implementation manner of the second aspect.

In a ninth aspect, an apparatus for measuring data flow is provided, which is used to execute the third aspect or any possible implementation manner of the third aspect. Specifically, the apparatus includes a module for performing the third aspect or any possible implementation manner of the third aspect.

In a tenth aspect, a data flow measurement apparatus is provided, which is used to perform the fourth aspect or any possible implementation manner of the fourth aspect. Specifically, the apparatus includes a module for performing the fourth aspect or any possible implementation manner of the fourth aspect.

In an eleventh aspect, an apparatus for measuring data flow is provided, which is used to execute the fifth aspect or any possible implementation manner of the fifth aspect. Specifically, the apparatus includes a module for performing the fifth aspect or any possible implementation manner of the fifth aspect.

A twelfth aspect provides an apparatus for measuring data flow, which is used to execute the sixth aspect or any possible implementation manner of the sixth aspect. Specifically, the apparatus includes a module for performing the sixth aspect or any possible implementation manner of the sixth aspect.

A thirteenth aspect provides a network device, the network device comprising a processor and a memory, wherein:

The memory stores instructions that are executed by the processor to implement the method of any of the first, second and third aspects above.

A fourteenth aspect provides a controller, the controller comprising a processor and a memory, wherein:

The memory stores instructions that are executed by the processor to implement the method of any one of the fourth, fifth and sixth aspects above.

A fifteenth aspect provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and the instructions are loaded and executed by a processor to implement any one of the first to sixth aspects above. The method described in one aspect.

A sixteenth aspect provides a computer program product, the computer program product includes instructions, the instructions are loaded and executed by a processor to implement the method according to any one of the first to sixth aspects .

A seventeenth aspect provides a data flow measurement system, the system includes a network device and a controller, wherein:

The network device is configured to execute the method according to the first aspect;

The controller is configured to perform the method as described in the fourth aspect above.

In an eighteenth aspect, a data flow measurement system is provided, the system includes a network device and a controller, wherein:

The network device is configured to execute the method according to the second aspect;

The controller is adapted to perform the method of the fifth aspect above.

A nineteenth aspect provides a data flow measurement system, the system includes a network device and a controller, wherein:

The network device is configured to execute the method according to the third aspect;

The controller is configured to perform the method as described in the sixth aspect above.

In a twentieth aspect, a processor is provided for executing the methods described in the first to sixth aspects. In the process of executing these methods, the process of sending data and receiving data in the above method can be understood as the process of outputting the above-mentioned data by the processor and the process of receiving the above-mentioned input data by the processor. When outputting the above-mentioned data, the processor outputs the above-mentioned data to the transceiver for transmission by the transceiver. After the above-mentioned data is output by the processor, other processing may be required before reaching the transceiver. Similarly, when the processor receives the above-mentioned data, the transceiver receives the above-mentioned data and inputs it into the processor. Further, after the transceiver receives the above-mentioned data, the above-mentioned data may need to be processed in other ways before being input to the processor.

Based on the above principles, for example, the flow identifier and measurement value of the first data stream, the identifier of the second data stream, and the target data table mentioned in the aforementioned method can be understood as the processor outputting the stream identifier of the first data stream. and measurement values, the identification of the second data stream, the target data table. For another example, receiving the stream identifier and measurement value of the first data stream, the identifier of the second data stream, and the target data table can be understood as the processor receiving the stream identifier and measurement value of the input first data stream, the identifier of the second data stream, target data sheet.

For the operations of transmitting, sending and receiving involved in the processor, if there is no special description, or if it does not contradict its actual function or internal logic in the relevant description, it can be more generally understood as the processor output and Receive, input, etc. operations, rather than transmit, transmit, and receive operations directly performed by radio frequency circuits and antennas.

In the implementation process, the above-mentioned processor may be a processor specially used to execute these methods, or may be a processor that executes computer instructions in a memory to execute these methods, such as a general-purpose processor. The above-mentioned memory can be a non-transitory (non-transitory) memory, such as a read-only memory (Read Only Memory, ROM), which can be integrated with the processor on the same chip, or can be set on different chips respectively. The embodiment does not limit the type of the memory and the setting manner of the memory and the processor.

In a twenty-first aspect, an embodiment of the present application provides a chip system, the chip system includes a processor and an interface, where the interface is used to obtain a program or an instruction, and the processor is used to call the program or instruction to implement the above-mentioned The function involved in any one of the first aspect to the sixth aspect, for example, determining or processing the data involved in the above-mentioned method, and the data may be a measurement value, a flow identification, and the like.

In a possible design, the chip system further includes a memory for storing necessary program instructions and data of the network device. The chip system may be composed of chips, or may include chips and other discrete devices.

Description of drawings

1 is a flowchart of a method for measuring data flow provided by an embodiment of the present application;

2 is a schematic diagram of a target data table provided by an embodiment of the present application;

3 is a flowchart of a method for judging a large flow provided by an embodiment of the present application;

4 is a schematic diagram of a target data table provided by an embodiment of the present application;

5 is a flowchart of a method for measuring data flow provided by an embodiment of the present application;

6 is a flowchart of a method for determining a measurement value corresponding to a flow identifier of a non-large flow provided by an embodiment of the present application;

7 is a flowchart of a method for measuring data flow provided by an embodiment of the present application;

FIG. 8 is a flowchart of a method for measuring data flow provided by an embodiment of the present application;

9 is a flowchart of a method for determining a measurement value corresponding to a flow identifier of a non-large flow provided by an embodiment of the present application;

10 is a flowchart of a method for measuring data flow provided by an embodiment of the present application;

11 is a flowchart of a method for measuring data flow provided by an embodiment of the present application;

FIG. 12 is a schematic structural diagram of an apparatus for data flow measurement provided by an embodiment of the present application;

FIG. 13 is a schematic structural diagram of an apparatus for data flow measurement provided by an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a network device provided by an embodiment of the present application;

FIG. 15 is a schematic structural diagram of a controller provided by an embodiment of the present application.

Detailed ways

The embodiment of the present application provides a data flow measurement method, and the method can be implemented by a network device. The network device may be a device for receiving and forwarding data packets, such as a router, a switch, and the like. The network device can calculate and record the measurements of the received packets. In this embodiment of the present application, after a data packet is received, it is necessary to determine whether the data flow to which the data packet belongs is a large flow, and if it is a large flow, the flow identification and measurement value of the flow are recorded separately, so that the measurement value of the large flow can be is relatively accurate. And since the measured values of the large flow are recorded separately, only the measured values of the data flow other than the large flow can be recorded in the target data table. In this way, because the number of recorded data flows is reduced, when restoring the measured values of these data flows , the accuracy of the obtained measurements is also relatively high.

The data flow measurement method provided by the embodiment of the present application can be applied in scenarios such as network security and forwarding policy planning. For example, in a network security scenario, a denial of service (Denial of Service, DOS) attack can be detected through data flow measurement. Specifically, a network device can obtain data flow data by executing the data flow measurement method provided by the embodiments of the present application. If there are a large number of data packets sent to a certain destination address within a period of time, it can be preliminarily determined that the host corresponding to the destination address may have been attacked by DOS, and further Take appropriate defenses.

Referring to FIG. 1 , an embodiment of the present application provides a data flow measurement method, and a processing flow of the method may include the following steps:

Step 101, receiving the target data packet, for each data row of the target data table, based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet, determine that the target data stream to which the target data packet belongs is in the data. The corresponding data storage space in the row.

The flow identifier can be set according to actual requirements and used to uniquely identify a data stream. For example, the flow identifier can be the source address and destination carried in the data packet of the data stream. For example, the flow identifier can also be the data of the data stream. The quintuple information carried in the packet. A data stream includes multiple data packets, and the measurement value of the data stream is the sum of the measurement values of all the data packets included in the data stream.

The target data table is used to record data such as the flow identifier of the data flow, the measurement value of the data flow, and the like. For example, the target data table may be a sketch data table. Each data row of the target data table may correspond to a first eigenvalue calculation function, and different data rows correspond to different first eigenvalue functions. Wherein, the first feature value calculation function may be a hash function.

For each data row, the first eigenvalue calculation function corresponding to the data row can have m types of calculation results, where m is the number of columns of the target data table, that is, the data storage space corresponding to each data row number. Each of the m types of calculation results corresponds to one data storage space of the data row, that is, the m types of calculation results correspond to the m data storage spaces one-to-one. The data storage space may also be called a bucket, and in the case that the target data table is a sketch data table, the data storage space corresponding to the target data table may be called a sketch bucket.

In implementation, when receiving a data packet, the network device determines the data storage space corresponding to the data flow in the target data table to which the data packet belongs. That is, it is determined in which data storage spaces corresponding to the target data table the measurement value of the data packet needs to be recorded. For convenience of description, the currently received data packet is referred to as a target data packet, and the data stream to which the data packet belongs is referred to as a target data stream.

For each data row of the target data table, the corresponding data storage space in the data row of the data stream to which the target data packet belongs can be determined separately. The determination method can be as follows:

First, the flow identifier corresponding to the target data packet is preprocessed to obtain the value to be calculated. The preprocessing may be converting the stream identifier into a binary number, and correspondingly, the value to be calculated is a binary number corresponding to the stream identifier. Then, the value to be calculated is input into the first eigenvalue calculation function to obtain an output result corresponding to the flow identifier. Finally, in the corresponding relationship between the output result and the data storage space, the data storage space corresponding to the output result corresponding to the stream identifier is determined. The output result of the first feature value calculation function is the storage space identifier of the data storage space, and the storage space identifier of each data storage space of the target data table is different. The storage space identifier can be a number, for example, 1, 2, 3, and so on.

As shown in Figure 2, the flow identifier corresponding to the target data packet is K ₁ , the measurement value is V ₁ , the first eigenvalue calculation function corresponding to the first data row of the target data table is h ₁ , and the second data row corresponds to The first eigenvalue calculation function of is h ₁ , and so on, the first eigenvalue calculation function corresponding to the mth data row is h _m . Preprocess K ₁ to obtain the value to be calculated, and then substitute the value to be calculated into h ₁ , h ₁ ...... h _m respectively, and obtain m output results, each data result corresponds to a data storage corresponding to the target data table space.

Step 102, if the target data stream is the first data stream corresponding to the data storage space, then based on the measurement value of the target data packet, determine the measurement value of the target data stream and record it in the data storage space.

The first data stream may also be referred to as a large stream. For ease of understanding, the first data stream is hereinafter referred to as a large stream. The measurement value can be set according to actual needs. For example, the measurement value may be the number of data packets included in the data stream, and, for example, the measurement value may also be the total number of bytes of data packets included in the data stream, and so on.

In an implementation, after receiving the target data packet, the network device may calculate measurements on the data packet. After the data storage space corresponding to the target data stream is determined, the measurement value of the target data packet can be recorded in different ways according to whether the target data stream is a large stream. For each data storage space corresponding to the determined target data stream, the method of judging whether the target data stream is a large stream and recording the measured value of the target data packet can be referred to in FIG. 3, which may specifically include the following steps:

S1021. Determine whether the flow identifier and measurement value of the large flow are recorded in the data storage space.

This step is to determine whether the target data packet is the first data packet that enters the data storage space for recording after the data storage space is initialized. It should be noted here that the initialization of the data storage space is performed at the end of each measurement period, and specifically, the data recorded in the data storage space may be deleted. Then, the above-mentioned "judging whether the target data packet is the first data packet to enter the data storage space for recording after the data storage space is initialized" is to judge whether the target data packet is the first data packet in the current measurement period. Data packets that entered this data storage space for recording.

S1022, if the flow identifier and measurement value of the large flow are not recorded in the data storage space, then determine that the target data flow is the large flow corresponding to the data storage space, and use the measurement value of the target data packet as the measurement value of the target data flow, The stream identifier of the target data stream is used as the stream identifier of the first data stream corresponding to the data storage space, and the stream identifier of the target data and the measurement value are correspondingly recorded in the data storage space.

If the flow identifier and measurement value of the large flow are not recorded in the data storage space, it means that the target data packet is the first data packet to enter the data storage space for recording after the data storage space is initialized, that is, the , the target data packet is the first data packet that enters the data storage space for recording in the current measurement period. In this case, the target data stream can be regarded as a large stream, and the measured values of the target data stream can be recorded in the data storage space.

For example, the flow ID corresponding to the target data packet is K ₁ , the measurement value is V ₁ , and the flow ID and measurement value of the large flow are not recorded in the data storage space, then the flow ID of the large flow is recorded as K ₁ , and the The measurement of the large current is recorded as V ₁ . In the case where the measurement value is the number of packets of the data flow, V ₁ =1.

S1023. If the flow identifier and measurement value of the large flow are recorded in the data storage space, determine whether the flow identifier of the target data flow is the same as the flow identifier of the large flow recorded in the data storage space.

If the flow identifier and measurement value of the large flow are recorded in the data storage space, it means that there are other data packets entering the data storage space for recording before the target data packet. In this case, it is possible to continue to determine whether the target data stream to which the target data packet belongs is a recorded large stream.

S1024. If the flow identifier of the target data stream is the same as the stream identifier of the large stream recorded in the data storage space, add the measurement value of the target data packet to the measurement value of the first data stream recorded in the data storage space.

If the flow ID of the target data flow is the same as the flow ID of the large flow recorded in the data storage space, it means that there are other data packets belonging to the target data flow entering the data storage space for recording before the target data packet, and the target data flow is recorded in the data storage space. A data stream is a large stream corresponding to the data storage space. In this case, the recorded measurements of the mass flow can be updated.

For example, the flow ID corresponding to the target data packet is K ₁ , the measurement value is V ₁ , the flow ID with a large flow recorded in the data storage space is K ₁ , and the measurement value is V, after determining the flow ID corresponding to the target data packet After the flow identification of the recorded large flow is the same, the measurement value V ₁ of the target data packet is accumulated to the recorded large flow measurement value V, that is, the large flow measurement value V is updated to (V+V ₁ ).

S1025. If the stream identifier of the target data stream is different from the stream identifier of the first data stream recorded in the data storage space, determine whether a comparison measurement value is recorded in the data storage space.

In order to determine whether a data stream is a large stream, it is necessary to know the measurement value of the data stream and the sum of the measurement values of other data streams after the data stream. Therefore, other data other than the large stream can also be recorded in the data storage space. The sum of the measurements of the flow, which is referred to here as the comparative measurement.

S1026. If no comparison measurement value is recorded, record the measurement value of the target data packet in the data storage space as a comparison measurement value corresponding to the data storage space.

If no comparison measurement value is recorded, it means that the target data packet is the first data packet to enter the data storage space for recording after the data storage space is initialized, except for the data packets of the recorded large flow, that is, The target data packet is the first data packet that enters the data storage space for recording in the current measurement period, except for the recorded data packets of the large flow. In this case, the measured value of the target data packet can be used as the comparative measured value corresponding to the data storage space.

For example, if the flow identifier corresponding to the marked data packet is K ₁ , the measurement value is V ₁ , and the comparison measurement value is not recorded in the data storage space, the comparison measurement value is recorded as V ₁ .

S1027. If the comparative measurement value is recorded, accumulate the measurement value of the target data packet on the comparative measurement value recorded in the data storage space.

If a comparative measurement value is recorded, it means that before the target data packet, other data packets other than the recorded large flow data packets enter the data storage space for recording. In this case, the recorded comparative measurements can be updated.

For example, the flow identifier corresponding to the target data packet is K ₁ , the measurement value is V ₁ , and the comparison measurement value C is recorded in the data storage space. The measurement value V ₁ of the target packet is accumulated to the recorded comparison measurement value C, ie, the comparison measurement value C is updated to (C+V ₁ ).

S1028. Determine whether the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the large flow recorded in the data storage space.

After the comparison measurement value recorded in the data storage space is updated, it is necessary to compare the recorded measurement value of the large flow with the updated comparison measurement value to determine whether the recorded large flow is still a large flow.

S1029: If the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the large flow recorded in the data storage space, then determine that the target data flow is the large flow corresponding to the data storage space, and record the data in the data storage space The flow ID of the large flow is updated to the flow ID of the target data flow, the measurement value of the large flow recorded in the data storage space is updated to the measurement value of the target data packet, and the comparison measurement value recorded in the data storage space is updated as 0.

If the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the large flow recorded in the data storage space, it means that the currently recorded large flow is no longer a large flow. In this case, it is necessary to take the data flow to which the target data packet belongs as a new large flow, and update the flow identifier of the large flow, the measured value of the large flow and the comparative measured value recorded in the data storage space.

For example, the flow identifier corresponding to the target data packet is K ₁ , and the measured value is V ₁ . After accumulating the measured value V ₁ of the target data packet to the recorded large flow measurement value C, the comparative measurement value is updated from C to (C+ V1 ₎ . In addition, if the updated comparative measurement value (C+V ₁ ) is greater than the recorded measurement value V of the large flow, then the target data flow to which the target data packet belongs is regarded as a new large flow, and the flow identifier of the large flow is updated as K ₁ , update the measured value of large flow to V ₁ . Since no other packets have been received after the target packet at this time, the comparison measurement value is updated to 0.

Step 103: If the target data stream is not the first data stream corresponding to the data storage space, record the measurement value of the target data stream in the target data table based on the measurement value of the target data packet.

In implementation, if the accumulated comparative measurement value recorded in the data storage space is not greater than the recorded measurement value of the large flow, it is determined that the target data flow to which the target data packet belongs is not the large flow corresponding to the data storage space.

For example, the flow identifier corresponding to the target data packet is K ₁ , and the measured value is V ₁ . After accumulating the measured value V ₁ of the target data packet to the recorded large flow measurement value C, the comparative measurement value is updated from C to (C+ V1 ₎ . If the updated comparative measurement value (C+V ₁ ) is not greater than the recorded measurement value V of the large flow, it can be determined that the target data flow to which the target data packet belongs is not the large flow corresponding to the data storage space.

In this case, there may be various methods for recording the measurement value of the target data stream in the data storage space, and two are listed below for description.

Method 1: Based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located and the flow identifier of the target data packet, determine the accumulation coefficient corresponding to the target data flow, and multiply the measurement value of the target data packet by the accumulation coefficient, The adjusted measurement value of the target data packet is obtained, and the adjusted measurement value of the target data packet is accumulated to the non-large flow measurement value recorded in the data storage space.

In each data storage space corresponding to the target data table, in addition to recording the flow identification of the large flow, the measured value of the large flow, and the comparative measured value, the measured value of the non-large flow can also be recorded.

For each data row of the target data table, in addition to corresponding to a first eigenvalue calculation function, it may also correspond to a second eigenvalue calculation function. The second feature value calculation function may also be a hash function, and the second feature value calculation function corresponding to each data row is different.

For each data storage space corresponding to the target data stream determined above, when recording the measurement value of the target data stream based on the measurement value of the target data packet, the value to be calculated corresponding to the target data packet can be input into the data storage space In the second eigenvalue calculation function corresponding to the data row, the accumulation coefficient corresponding to the stream identifier is obtained. Among them, the accumulation coefficient can be a decimal. It should be noted here that the value to be calculated corresponding to the target data packet may be obtained by preprocessing when determining the data storage space corresponding to the data stream to which the target data packet belongs.

Then, the measured value of the target data packet is multiplied by the accumulation coefficient to obtain the adjusted measured value of the target data packet. Finally, the adjusted measurement of the target packet is accumulated to the measurement of the non-large flow recorded in the data storage space.

As shown in Figure 4, on the basis of Figure 2, the first data row of the target data table also corresponds to a second eigenvalue calculation function g ₁ , and the second data row also corresponds to a second eigenvalue function g2 , and so on, the mth data row also corresponds to a second eigenvalue calculation function gm. Preprocess K ₁ to obtain the value to be calculated, and substitute the value to be calculated into h ₁ , h ₂ ...... h _m respectively, and obtain m output results, each data result corresponds to a data storage space corresponding to the target data table. Then, the values to be calculated are substituted into g1, g2...gm, respectively, to obtain m cumulative coefficients. Finally, multiply the accumulated coefficient X _i obtained by _gi by the measurement value V ₁ , and accumulate it to the non-large flow measurement value recorded in the data storage space corresponding to the output result obtained by hi, that is, the value of h in the _i -th row The measured value D _i of the non-large flow recorded in the data storage space corresponding to the output result of _i is updated to (D _i +X _i *V ₁ ), where i represents the number of rows, and 1≤i≤m.

The second method is to directly accumulate the measured value of the target data packet to the measured value of the non-large flow recorded in the data storage space.

For example, the flow corresponding to the target data packet is identified as K ₁ and the measurement value is V ₁ , and the measurement value V ₁ of the target data packet is accumulated to the recorded measurement value of the non-large flow, that is, the recorded measurement value of the non-large flow is The value D is updated to (D+V ₁ ).

After the end of each measurement period, the measurement value of the data flow in the measurement period can be recovered, that is, the measurement value of the data flow in the processing period can be determined. Restoring the measurement value of the data stream in the measurement period can be implemented in the device or by the controller. Controllers can be network devices such as routers and switches, terminal devices such as mobile phones, desktop computers, notebook computers, and tablet computers, and cloud devices such as single servers and server clusters.

If it is implemented by this device, in step S1029, when it is determined that the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the large flow recorded in the data storage space, the flow of the large flow currently recorded in the data storage space is Measured values for identification and mass flow are retrieved for storage. In step 103, after it is determined that the target data flow is not a large flow corresponding to the data storage space, the flow identifier corresponding to the target data packet may be recorded as a non-large flow flow identifier. At the end of each measurement period, the locally stored target data table can be taken out and stored in another storage space, and the currently stored target data table can be initialized.

If it is implemented by the controller, in step S1029, after it is determined that the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the large flow recorded in the data storage space, the storage space identifier of the data storage space can be identified. , The flow identifier of the large flow currently recorded in the data storage space and the measured value of the large flow are correspondingly sent to the controller. In step 103, after it is determined that the target data flow is not a large flow corresponding to the data storage space, the flow identifier corresponding to the target data packet may be sent to the controller as a non-large flow flow identifier. At the end of each measurement cycle, the locally stored target data table can be sent to the controller and initialized.

In addition, in order to avoid the repeated recording or repeated transmission of the non-large flow stream ID, a Bloom Filter can be set. Before recording or sending the non-large stream flow ID, you can first determine whether the flow ID is recorded in the Bloom Filter. If the stream ID is not recorded, then record or send the stream ID. After recording or sending the stream identifier of the non-large stream, the stream identifier can be recorded in the Bloom Filter. The usage of Bloom Filter is described below:

Bloom Filter is essentially a data structure consisting of a long array and multiple hash functions. When a stream ID needs to be recorded in the Bloom Filter, each hash function in the Bloom Filter can be used to calculate the hash value of the stream ID respectively. Then, dispose of 1 at the position corresponding to the above hash value in the array of the Bloom Filter. When you need to query whether a stream ID is recorded in the Bloom Filter, you must first use each hash function in the Bloom Filter to calculate a hash value of the stream ID, and then query the Bloom Filter array Whether the position corresponding to the calculated hash value is 1, if it is all 1, it means that the Bloom Filter has recorded the element, if it is not all 1, it means that the Bloom Filter has not recorded the element.

It should be noted here that a Bloom Filter can be set for each data storage space in the target data table, or a Bloom Filter can be set for a target data table. In the case where a Bloom Filter is set in each data storage space, the Bloom Filter can be stored in the corresponding data storage space. Before recording or sending the flow ID of the non-large flow, the corresponding flow ID of the non-large flow can be determined first. Whether the stream identifier is recorded in the Bloom Filter stored in the data storage space of the data storage space, after recording or sending the stream identifier of the non-large stream, the stream identifier can be recorded in the Bloom Filter stored in the corresponding data storage space.

In this embodiment of the present application, after a data packet is received, it is first determined whether the data flow to which the data packet belongs is a large flow, and if it is a large flow, its flow identification and measurement value are recorded separately, so that the measurement value of the large flow can be recorded. is relatively accurate. And since the measured values of the large flow are recorded separately, only the measured values of the data flow other than the large flow can be recorded in the target data table. In this way, because the number of recorded data flows is reduced, when restoring the measured values of these data flows , the accuracy of the obtained measurements is also relatively high.

The controller implements the measurement value recovery of the data flow to be described below. The method in which the device realizes the recovery of the measured value of the data stream is similar to the method in which the controller realizes the recovery of the measured value of the data stream, and will not be repeated here. It should be noted here that the measurement value recovery of the data flow is a part of the data flow measurement.

Referring to FIG. 5, for the data flow measurement method shown in FIG. 3, the method for the controller to realize the measurement recovery of the data flow may include the following steps:

Step 201: Receive the flow identifier of the large flow, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the flow identifier of the large flow, and the target data table sent by the network device.

Step 202 , in the measured values of the large flow recorded in the target data table, filter the measured values of the large flow corresponding to the same flow identification to obtain the first measured value of each large flow recorded in the target data table.

In the implementation, since the measured value of any data packet is recorded in the target data table, it is to be recorded in the corresponding data storage space of each data row, then, in the target data table, there may be multiple data The measurement values of the same large flow are recorded in the storage space. In this case, it is possible to filter the measured values of the large flow recorded in the target data table. There are many ways to filter, the following are listed below:

Method 1: Among the measured values of the large flow recorded in the target data table, the maximum measured value of the large flow corresponding to the same flow identification is used as the first measured value of the large flow.

For example, for the large stream K ₁ , its measured values V ₁ , V ₂ , and V ₃ are respectively recorded in the three data storage spaces of the target data table. Compare the magnitudes of V ₁ , V ₂ , and V ₃ , and take the largest measured value among them as the first measured value of the large flow K ₁ .

Method 2: Among the measured values of the large flow recorded in the target data table, the average value of the measured values of the large flow corresponding to the same flow identification is used as the first measured value of the large flow.

For example, for the large stream K ₁ , its measured values V ₁ , V ₂ , and V ₃ are respectively recorded in the three data storage spaces of the target data table. Calculate the average value of V ₁ , V ₂ , V ₃

Will

as the _first measurement of the large flow K1.

Method 3: Among the measured values of the large flow recorded in the target data table, any measured value of the large flow corresponding to the same flow identification is used as the first measured value of the large flow.

For example, for the large stream K ₁ , its measured values V ₁ , V ₂ , and V ₃ are respectively recorded in the three data storage spaces of the target data table. Any value of V ₁ , V ₂ , and V ₃ is randomly selected as the first measurement value of the large flow K ₁ .

Step 203: In the received measured values of the large flow, accumulate the measured values of the large flows corresponding to the same storage space identifier and the same flow identifier to obtain a second measured value corresponding to the received flow identifier of each large flow.

In the implementation, when the network device records the measurement value of the large flow in the target data table, it may occur that a large flow has been replaced many times in the same data storage space, and after each replacement, the measurement of the large flow values are sent to the controller. In this way, the controller will receive multiple measurement values of the large flow corresponding to the data storage space (that is, receive multiple measurement values corresponding to the same storage space ID and the same flow ID), and these measurement values are actually the The measurement values of the large flow at different times in the measurement period, in order to count the measurement values of the large flow in the measurement period, these measurement values may be accumulated to obtain the second measurement value of the large flow. In addition, if there is only one measurement value corresponding to the large flow with the same storage space identification and the same flow identification, then the measurement value may be regarded as the second measurement value of the large flow.

Step 204: In the received second measurement value corresponding to the flow identifier of each first data stream, filter the second measurement value of the first data stream corresponding to the same stream identifier to obtain each received first data stream. The flow identification of the data flow corresponds to the third measurement value.

In the implementation, since the measurement value of any data packet is recorded in the corresponding data storage space of each data row when recording the measurement value of any data packet in the target data table, there may be a large flow in multiple data storage spaces. Each data storage space is replaced, and after each data storage space is replaced, the measurement value of the large stream is sent to the controller. In this way, the controller will receive multiple measurement values of the large flow corresponding to multiple data storage spaces (that is, receiving measurement values of multiple large flows corresponding to different storage space identifiers and the same flow identifier). In this case, it is possible to filter the measurement values of the large streams received corresponding to different storage space identifiers and the same stream identifier. Because the measured values of the large flows corresponding to the same storage space identifier and the same flow identifier have been accumulated in step 203, here only the second measured value of the large flow corresponding to different storage space identifiers and the same flow identifier is required Just filter. Specifically, the screening method may be the same as the screening method for the measurement value of the large flow recorded in the target data table in the above step 202, and several are also listed below for description.

Method 1: In the received second measurement values corresponding to the flow identifiers of the major streams, the largest second measurement value of the major streams corresponding to the same flow identifier is used as the third measurement value of the major stream.

For example, for the large stream K ₂ , there are three second measurement values V ₄ , V ₅ , and V ₆ corresponding to different storage space identifiers. Compare the magnitudes of V ₄ , V ₅ , and V ₆ , and take the largest second measurement value as the third measurement value of the large flow K ₂ .

Method 2: In the received second measurement values corresponding to the flow identifiers of the major streams, the average value of the second measurement values of the major streams corresponding to the same flow identifier is used as the third measurement value of the major stream.

For example, for the large stream K ₂ , its measured values V ₄ , V ₅ , and V ₆ are recorded in the three data storage spaces of the target data table, respectively. Calculate the average value of V ₄ , V ₅ , V ₆

Will

as the first measurement of the large flow _K2 .

Method 3: In the received second measurement values corresponding to the flow identifiers of the major streams, use any second measurement value of the major stream corresponding to the same flow identifier as the third measurement value of the major stream.

For example, for the large stream K ₂ , its measured values V ₄ , V ₅ , and V ₆ are recorded in the three data storage spaces of the target data table, respectively. Any value of V ₄ , V ₅ , and V ₆ is randomly selected as the third measurement value of the large flow K ₂ .

Step 205: In the first measurement value of each large flow recorded in the target data table and the third measurement value corresponding to the received flow identification of each large flow, compare the first measurement value and the second measurement value corresponding to the same flow identification. The measurement values are accumulated to obtain the statistical measurement value corresponding to the flow ID of each large flow.

In the implementation, the network device may intermittently receive data packets belonging to the same data stream, then the following situation may occur: the data stream is first judged as a large flow, then replaced by other data flows, and finally judged as a large flow flow and continue until the end of a measurement period. In this case, the flow identification of the data flow will be sent to the controller as a measure of the large flow, and the data flow will also be recorded as a large flow in the target data table sent to the controller. Then, in order to count the measured value of the data flow in one measurement period, in the first measured value of each large flow recorded in the target data table and the third measured value corresponding to the received flow identifier of each large flow, The first measurement value and the third measurement value of the large flow corresponding to the same flow identification are accumulated as the statistical measurement value corresponding to the flow identification of the large flow.

In addition, if a large flow has only the first measurement value or the third measurement value, the first measurement value or the second measurement value of the large flow is used as the statistical measurement value of the large flow.

Step 206: Based on the received flow identifiers of the non-large flows and the target data table, determine the statistical measurement value corresponding to the flow identifiers of each non-large flow.

In implementation, for the first method of recording the measured value of the target data stream in the data storage space in the above-mentioned step 103, here, the method for determining the statistical measured value corresponding to the stream identifier of each non-large stream can refer to FIG. 6, Specifically, the following steps may be included:

S2061. For each data storage space in the target data table, based on the first eigenvalue calculation function corresponding to the data row where the data storage space is located, in the received stream identifiers of non-large streams, determine the corresponding data storage space. Flow ID for non-large flow.

For each received flow identifier of a non-large flow, the flow identifier of the non-large flow is preprocessed to obtain a value to be calculated. The value to be calculated is then input into the first eigenvalue calculation function corresponding to each data row to obtain a plurality of output results corresponding to the stream identifier. Then, determine the data storage space corresponding to each output result. In this way, the stream identifier of the non-large stream corresponding to each data storage space can be determined.

S2062. Based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located, determine the accumulation coefficient corresponding to the flow identifier of each non-large flow corresponding to the data storage space. Multiply the statistical measurement value corresponding to the flow identifier of each non-large stream corresponding to the data storage space as the unknown number by the corresponding accumulation coefficient, and perform accumulation, and use the measurement value of the data stream recorded in the data storage space as the accumulation result to obtain the target. equation.

For example, the measured value of the non-large flow recorded in one data storage space of the target data table is Y, and the three non-large flows corresponding to the data storage space are K ₇ , K ₈ , and K ₉ . The accumulation coefficients of K ₇ , K ₈ , and K ₉ are obtained by using the second eigenvalue calculation function corresponding to the data row where the data storage space is located, respectively, as X ₇ , X ₈ and X ₉ . Assuming that the three statistical measurements of non-large currents are the unknowns V ₇ , V ₈ and V ₉ , the following equations can be established:

X ₇ *V ₇ +X ₈ *V ₈ +X ₉ *V ₉ =Y.

It should be noted that this equation is only an example, and other transformations may be made on the basis of this equation, which will not be listed one by one here.

S2063. Form the target equations corresponding to each data storage space in the target data table into a target equation group.

S2064. Based on the linear optimization algorithm, taking the minimum sum of the statistical measurement values corresponding to the received flow identifiers of non-large flows as the optimization objective, solve the objective equation set, and obtain the statistical measurement values corresponding to each non-large flow flow identifier.

Among them, the linear optimization algorithm may be a branch and bound method, a cut plane method, a Monte Carlo method, and the like.

For the second method of recording the measurement value of the target data stream in the data storage space in the above step 103, here, the method for determining the statistical measurement value corresponding to the flow identifier of each non-large stream may be as follows:

For each non-large stream flow identifier, the data storage space corresponding to the non-large stream stream identifier in each data row is determined through the first eigenvalue calculation function corresponding to each data row of the target data table. Then, the minimum value among the measured values of the non-large flow recorded in the determined storage spaces is determined as the statistical measurement value corresponding to the flow identifier of the non-large flow, that is, the statistical measurement value of the non-large flow .

In a possible implementation manner, among the obtained statistical measurement values corresponding to the flow IDs of large flows and the statistical measurement values corresponding to the flow IDs of non-large flows, the measurement values corresponding to the same flow IDs are accumulated to obtain each The flow ID corresponds to the total measurement value.

In the embodiment of the present application, since the large flow is the corresponding recorded flow identification and measurement value, thus, the finally obtained measurement value of the large flow is relatively accurate. And because the measured value of non-large flow is recorded in the target data table, this part of the measured value does not include the measured value of large flow, so the finally calculated measured value of each non-large flow is also relatively accurate.

Referring to FIG. 7 , an embodiment of the present application further provides a method for measuring data flow, and the processing flow of the method may include the following steps:

Step 301, receiving the target data packet, for each data row of the target data table, based on the first eigenvalue calculation function corresponding to the data row and the flow identifier corresponding to the target data packet, determine the target data stream to which the target data packet belongs. The data storage space corresponding to the data row.

The specific implementation manner of this step 301 is the same as or similar to the specific implementation manner of the above-mentioned step 101, and will not be repeated here.

Step 302: Based on the measurement value corresponding to the target data packet, record the measurement value of the target data stream in the data storage space.

In implementation, there may be various methods for recording the measurement value of the target data stream in the data storage space, and two are listed below for description.

Method 1: Based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located and the flow identifier of the target data packet, determine the accumulation coefficient corresponding to the target data flow, and multiply the measurement value of the target data packet by the accumulation coefficient, The adjusted measurement value of the target data packet is obtained, and the adjusted measurement value of the target data packet is accumulated to the measurement value of the data flow recorded in the data storage space.

For each data storage space corresponding to the target data stream determined above, when recording the measurement value of the target data stream based on the measurement value of the target data packet, the to-be-calculated value corresponding to the target data packet may be input into the data storage first. In the second eigenvalue calculation function corresponding to the data row where the space is located, the accumulation coefficient corresponding to the stream identifier is obtained. Among them, the accumulation coefficient can be a decimal.

Then, the measured value of the target data packet is multiplied by the accumulation coefficient to obtain the adjusted measured value of the target data packet. Finally, the adjusted measurement of the target packet is accumulated to the measurement of the data flow recorded in the data storage space.

For example, the flow identifier corresponding to the target data packet is K ₁ , and the measured value is V ₁ . First, K ₁ is calculated by using the second eigenvalue calculation function, and the obtained accumulation coefficient is X ₁ . Then, the adjusted measurement value X ₁ *V ₁ of the target packet is calculated. Finally, the adjusted measured value X ₁ *V ₁ of the target data packet is accumulated to the measured value of the recorded data stream, that is, the measured value E of the recorded data stream is updated to (E+X ₁ *V ₁ ).

The second method is to directly accumulate the measured value of the target data packet to the measured value of the data stream recorded in the data storage space.

For example, the flow corresponding to the target data packet is identified as K ₁ and the measurement value is V ₁ , and the measurement value V ₁ of the target data packet is accumulated to the measurement value of the recorded data flow, that is, the measurement value E of the recorded data flow is added. Update to (E+V ₁ ).

Step 303: If the target data stream is the first data stream corresponding to the data storage space, determine the measurement value of the target data stream based on the measurement value of the target data packet, and record it in the data storage space.

The specific implementation of this step 303 is the same as or similar to the specific implementation of the above-mentioned step 102, and will not be repeated here.

In this embodiment of the present application, after a data packet is received, it is first determined whether the data flow to which the data packet belongs is a large flow, and if it is a large flow, its flow identification and measurement value are recorded separately, so that the measurement value of the large flow can be recorded. is relatively accurate. In addition, although in the embodiment of the present application, the measurement value of the data flow (including the large flow and the non-large flow) is recorded in the target data table instead of only the measurement value of the non-large flow. However, because of the separately recorded measurement values of the large flow, when the measurement value of the non-large flow is subsequently restored, the measured value of the non-large flow can be obtained according to the measured value of the recorded data flow and the separately recorded measurement value of the large flow. Therefore, , the technical effect that can be achieved by the above-mentioned embodiment shown in FIG. 1 can also be achieved.

Referring to FIG. 8 for the data flow measurement method shown in FIG. 7 , the method for the controller to restore the measurement value of the data flow may include the following steps:

Step 401: Receive the flow identifier of the large flow, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the flow identifier of the large flow, and the target data table sent by the network device.

Step 402: In the measured values of the large flows recorded in the target data table, filter the measured values of the large flows corresponding to the same flow identification to obtain the first measured value of each large flow recorded in the target data table.

The specific implementation manner of this step 402 is the same as or similar to the specific implementation manner of the above-mentioned step 202, and will not be repeated here.

Step 403: In the received measured values of the large flow, accumulate the measured values of the large flows corresponding to the same storage space identifier and the same flow identifier to obtain a second measured value corresponding to the received flow identifier of each large flow.

The specific implementation manner of this step 403 is the same as or similar to the specific implementation manner of the above-mentioned step 403, and will not be repeated here.

Step 404: In the received second measurement value corresponding to the flow identifier of each large stream, filter the second measurement value of the first data stream corresponding to the same stream identifier to obtain the received flow of each large stream. A corresponding third measurement value is identified.

The specific implementation manner of this step 404 is the same as or similar to the specific implementation manner of the above-mentioned step 204, and will not be repeated here.

Step 405: In the first measurement value of each large flow recorded in the target data table and the third measurement value corresponding to the received flow identification of each large flow, compare the first measurement value and the second measurement value corresponding to the same flow identification. The measurement values are accumulated to obtain the statistical measurement value corresponding to the flow ID of each large flow.

The specific implementation manner of this step 405 is the same as or similar to the specific implementation manner of the above-mentioned step 205, and will not be repeated here.

Step 406, based on the received flow identification of the non-large flow, the measured value of the data flow recorded in each data storage space in the target data table, and the obtained statistical measurement value corresponding to the flow identification of each large flow, determine each non-large flow. The flow ID of the large flow corresponds to the statistical measurement value.

In implementation, for the first method of recording the measurement value of the target data stream in the data storage space in the above step 302, here, the method for determining the measurement value corresponding to the flow identifier of each non-large stream can refer to FIG. It includes the following steps:

S4061. For each data storage space of the target data table, determine the accumulation coefficient corresponding to the flow identifier of each large stream corresponding to the data storage space based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located. Multiply the statistical measurement value corresponding to the flow identifier of each first data stream corresponding to the data storage space by the corresponding accumulation coefficient to obtain the adjusted statistical measurement value corresponding to the flow identifier of each first data stream corresponding to the data storage space .

The large flow corresponding to each data storage space in the target data table includes the large flow corresponding to the flow identification of the large flow recorded in the data storage space and the storage space identification corresponding to the data storage space sent by the network device. The flow ID of the big flow corresponds to the big flow.

For example, the data storage space ₁ of the target data table records the flow identifier K ₁ of the large stream, wherein the data storage space ₁ refers to the data storage space whose storage space identifier is ₁ . And the stream identifier K ₂ of the large stream corresponding to the storage space identifier ₁ is received. Therefore, the two large streams corresponding to the data storage space ₁ are K ₁ and K ₂ respectively. According to the second eigenvalue calculation function corresponding to the data row where the data storage space is located, the accumulated coefficients of K ₁ and K ₂ are obtained as X ₁ and X ₁ respectively.

S4062. Based on the first eigenvalue calculation function corresponding to the data row where the data storage space is located, in the received flow identifiers of the non-large stream, determine the stream identifier of the non-large stream corresponding to the data storage space, based on the location of the data storage space. The second eigenvalue calculation function corresponding to the data row determines the accumulation coefficient corresponding to each non-flow stream identifier corresponding to the data storage space.

For example, it is determined that the three non-large streams corresponding to the data storage space 1 of the target data table are K ₃ , K ₄ , and K ₅ . The accumulation coefficients of K ₃ , K ₄ , and K ₅ are obtained by using the second eigenvalue calculation function corresponding to the data row where the data storage space is located, as X ₃ , X ₄ and X ₅ , respectively.

S4063: Multiply the statistical measurement value corresponding to the flow identifier of each non-large stream corresponding to the data storage space as an unknown by the corresponding accumulation coefficient, and add the adjusted statistical measurement value corresponding to the flow identifier of each large stream corresponding to the data storage space , the measured value of the data stream recorded in the data storage space is used as the accumulation result, and the target equation corresponding to the data storage space pair is obtained.

For example, the measured value of the data flow recorded in the data storage space ₁ of the target data table is E ₁ , the corresponding large flows are K ₁ and K ₂ , and the corresponding non-large flows are K ₃ , K ₄ , and K ₅ , K ₁ The accumulation coefficients corresponding to , K ₂ , K ₃ , K ₄ , and K ₅ are X ₁ , X2 , X ₃ , X ₄ and X ₅ , respectively. Assuming that the statistical measurements of the three non-large currents K ₃ , K ₄ , and K ₅ are the unknowns V ₃ , V ₄ , and V ₅ , respectively, the following equations can be established:

X ₁ *V ₁ +X ₂ *V ₂ +X ₃ *V ₃₊ X ₄ *V ₄ +X ₅ *V ₅ +X ₆ *V ₆ =E ₁ .

S4064. Based on the target equations corresponding to each data storage space in the target data table, form a target equation group.

S4065. Based on the linear optimization algorithm, taking the minimum sum of the statistical measurement values corresponding to the received flow identifiers of non-large flows as the optimization objective, solve the objective equation system, and obtain the statistical measurement value corresponding to the flow identifier of each second data stream.

In the embodiment of the present application, since the large flow is the corresponding recorded flow identification and measurement value, thus, the finally obtained measurement value of the large flow is relatively accurate. And even if not only the measured value of the non-large flow is recorded in the target data table, the measured value of the non-large flow can be obtained according to the measured value of the recorded data flow and the measured value of the large flow. Therefore, the technical effect achieved by the above-mentioned embodiment shown in FIG. 5 can also be achieved.

Referring to FIG. 10 , an embodiment of the present application further provides a method for measuring data flow, and the processing flow of the method may include the following steps:

Step 501: Receive the target data packet, and determine the data storage space corresponding to the target data flow to which the target data packet belongs in the corresponding first data table based on the third feature value calculation function and the flow identifier corresponding to the target data packet.

The first data table may be a hash table, the third feature value calculation function may be a hash function, and the first data table is a third feature value calculation function. The third eigenvalue calculation function may have n kinds of calculation results, where n is the number of columns of the first data table, that is, the number of data storage spaces corresponding to the first data table. Each of the m types of calculation results corresponds to one data storage space, that is, the m types of calculation results are in one-to-one correspondence with the m data storage spaces. The data storage space may also be referred to as a bucket, and when the target data table is a hash table, the data storage space corresponding to the first data table may be referred to as a hash bucket.

In an implementation, when receiving a data packet, the network device determines the corresponding data storage space in the first data table of the data flow to which the data packet belongs. That is, it is determined in which data storage space corresponding to the first data table the measurement value of the data packet needs to be recorded. For convenience of description, the currently received data packet is referred to as a target data packet, and the data stream to which the data packet belongs is referred to as a target data stream.

Determining the data storage space corresponding to the target data stream in the first data table may include the following methods:

The flow identifier corresponding to the target data packet is preprocessed to obtain the value to be calculated. The preprocessing may be converting the stream identifier into a binary number, and correspondingly, the value to be calculated is the binary number corresponding to the stream identifier. Then, the value to be calculated is input into the third eigenvalue calculation function to obtain an output result corresponding to the flow identifier. Finally, in the corresponding relationship between the output result and the data storage space, the data storage space corresponding to the output result corresponding to the stream identifier is determined.

Step 502: If the target data stream is a large stream corresponding to the data storage space, determine the measurement value of the target data stream based on the measurement value of the target data packet, and record it in the data storage space.

The specific implementation of this step 502 is the same as or similar to the specific implementation of the above-mentioned step 102, and will not be repeated here.

It is worth mentioning that, in the above step 102, the flow identifier of the large flow, the measured value of the large flow and the comparative measured value recorded in the data storage space corresponding to the target data table, in this step 502, are corresponding to the first data table. Data storage space records.

Step 503: If the target data stream is not a large stream corresponding to the data storage space, record the measurement value of the target data stream in the target data table based on the measurement value of the target data packet.

The specific implementation of this step 503 is the same as or similar to the specific implementation of the above-mentioned step 103, and will not be repeated here. It should be noted that the difference between this step 503 and the specific implementation of the above-mentioned step 103 is that: this step 503 needs to first determine the target data stream according to the first eigenvalue calculation function corresponding to each data row of the target data table. The data storage space corresponding to each data row. Then, based on the measurement value of the target data packet, the measurement value of the target data stream is determined and recorded in the data storage space. In addition, in this step 503, only one Bloom Filter is required to record the flow identifier of the non-large flow.

In addition, it should be noted that, after each measurement period ends, different from the data flow measurement method shown in FIG. 1 above, the data flow measurement method shown in FIG. 5 also needs to store the data locally. The first data table is fetched and stored in other storage space, or sent to the controller. It is also necessary to initialize the currently stored first data table. Initializing the currently stored first data table may be to delete the data recorded in the first data table.

Referring to FIG. 11 , for the data flow measurement method shown in FIG. 10 , the method for the controller to restore the measurement value of the data flow may include the following steps:

Step 601: Receive the flow identifier and measurement value of the large flow, the flow identifier of the non-large flow, the target data table and the first data table sent by the network device.

Step 602 : Accumulate the measurement values corresponding to the same flow identifier in the received measurement value of the large flow and the measurement value recorded in the first data table to obtain a statistical measurement value corresponding to the flow identifier of each large flow.

In the implementation, the network device may intermittently receive data packets belonging to the same data stream, then the following situation may occur: the data stream is first judged as a large flow, then replaced by other data flows, and finally judged as a large flow flow and continue until the end of a measurement period. In this case, the flow identification of the data flow will be sent to the controller as a measurement value of the large flow, and the data flow will also be recorded as a large flow in the first data table sent to the controller. In order to count the measured value of the data flow in one measurement value period, the measured value of the large flow corresponding to the same flow identification can be calculated from the received measured value of the large flow and the measured value of the large flow recorded in the first data table. It is accumulated with the intermediate measurement value as the statistical measurement value corresponding to the flow ID of the large flow.

Step 603: Based on the received flow identifiers of the non-large flows and the target data table, determine the statistical measurement value corresponding to the flow identifiers of each non-large flow.

The specific implementation of this step 603 is the same as or similar to the specific implementation of the above-mentioned step 204, and will not be repeated here.

Based on the same technical concept, an embodiment of the present application also provides an apparatus for measuring data flow. As shown in FIG. 12 , the apparatus includes a receiving module 710 and a recording module 720 . The apparatus for data measurement can perform a variety of different data measurement methods. Correspondingly, when performing different data measurement methods, the functions implemented by the receiving module 710 and the recording module 720 can also be different. The following describes the functions implemented by the receiving module 710 and the recording module 720 when the apparatus for data flow measurement performs different data measurement methods.

1. When the apparatus for data flow measurement executes the data measurement method shown in FIG. 1 above.

The transceiver module 710 is configured to receive a target data packet, and for each data row of the target data table, determine the target data based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet The data storage space corresponding to the target data stream to which the packet belongs in the data row. Exemplarily, the transceiver module 710 is configured to perform step 101 .

The processing module 720 is configured to, if the target data stream is the first data stream corresponding to the data storage space, determine the measurement value of the target data stream based on the measurement value of the target data packet, and record it in the in the data storage space; if the target data stream is not the first data stream corresponding to the data storage space, then based on the measurement value of the target data packet, record the measurement value of the target data stream in the target data table . Exemplarily, the processing module 720 is configured to execute step 102 and step 103 .

2. When the apparatus for data flow measurement executes the data measurement method shown in FIG. 7 above.

The transceiver module 710 is configured to receive a target data packet, and for each data row of the target data table, determine the target data based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet The target data stream to which the packet belongs is in the data storage space corresponding to the data row. Exemplarily, the transceiver module 710 is configured to perform step 301 .

The processing module 720 is configured to, based on the measurement value corresponding to the target data packet, record the measurement value of the target data stream in the data storage space; if the target data stream is the first data stream corresponding to the data storage space For a data stream, the measurement value of the target data stream is determined based on the measurement value of the target data packet and recorded in the data storage space. Exemplarily, the processing module 720 is configured to execute

steps

302 and 303 .

It should be noted that: when the data flow measurement device provided in the above embodiments is used for data flow measurement, only the division of the above functional modules is used as an example for illustration. In practical applications, the above functions may be allocated to different functional modules as required. To complete, that is, to divide the internal structure of the network device into different functional modules to complete all or part of the functions described above. In addition, the apparatus for data flow measurement provided in the above embodiment and the method embodiment for data flow measurement belong to the same concept, and the specific implementation process thereof is detailed in the method embodiment, which will not be repeated here.

Based on the same technical concept, an embodiment of the present application further provides an apparatus for measuring data flow. As shown in FIG. 13 , the apparatus includes a receiving module 810 and a computing module 820 . The apparatus for data measurement can perform a variety of different data measurement methods. Correspondingly, when performing different data measurement methods, the functions implemented by the receiving module 810 and the computing module 820 can also be different. The following describes the functions implemented by the receiving module 810 and the computing module 820 when the apparatus for data flow measurement executes different data measurement methods.

1. When the apparatus for data flow measurement executes the data measurement method shown in FIG. 5 above.

The receiving module 810 is configured to receive the flow identifier of the first data stream and the corresponding measurement value, the stream identifier of the second data stream, and the target data table sent by the network device. Exemplarily, the receiving module 810 may be used to perform step 201 .

The processing module 820 is configured to filter the measured values of the first data stream corresponding to the same stream identifier among the measured values of the first data stream recorded in the target data table, and obtain each first data stream recorded in the target data table. A first measurement value of a data stream; in the received measurement values of the first data stream, the measurement values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain each received first data stream. The second measurement value corresponding to the flow identifier of the data stream; in the received second measurement value corresponding to the flow identifier of each first data stream, the second measurement value of the first data stream corresponding to the same flow identifier is screened , to obtain the third measurement value corresponding to the stream identifier of each first data stream received; the first measurement value of each first data stream recorded in the target data table and each received first data stream In the third measurement value corresponding to the stream identifier, the first measurement value and the third measurement value corresponding to the same stream identifier are accumulated to obtain the statistical measurement value corresponding to the stream identifier of each first data stream; The flow identifiers of the two data streams and the target data table determine the statistical measurement value corresponding to the stream identifiers of each second data stream. Exemplarily, the processing module 820 can be used to execute any of steps 201-206.

2. When the apparatus for data flow measurement executes the data measurement method shown in FIG. 8 above.

The receiving module 810 is configured to receive the stream identifier of the first data stream, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the stream identifier of the second data stream, and the target data table sent by the network device. Exemplarily, the receiving module 810 is configured to perform step 401 .

The calculation module 820 is configured to screen the measured values of the first data stream corresponding to the same stream identifier in the measured values of the first data stream recorded in the target data table, and obtain each first data stream recorded in the target data table. A first measurement value of a data stream; in the received measurement values of the first data stream, the measurement values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain each received first data stream. The second measurement value corresponding to the stream identifier of the data stream; in the received second measurement value corresponding to the stream identifier of each first data stream, the second measurement value corresponding to the stream identifier of the first data stream corresponding to the same stream identifier is The measurement value is screened to obtain a third measurement value corresponding to the stream identifier of each first data stream received; the first measurement value of each first data stream recorded in the target data table and each received In the third measurement value corresponding to the flow ID of the first data stream, the first measurement value and the second measurement value corresponding to the same flow ID are accumulated to obtain a statistical measurement value corresponding to the flow ID of each first data stream; based on The received flow identifier of the second data stream, the measured value of the data stream recorded in each data storage space in the target data table, and the obtained statistical measurement value corresponding to the stream identifier of each first data stream, determine each The statistical measurement value corresponding to the flow identifier of the second data flow. Exemplarily, the processing module 820 can be used to perform any one of steps 401-406.

Referring to FIG. 14 , an embodiment of the present application provides a schematic diagram of a network device 900 . The network device 900 may be a network device that executes any of the above methods in FIGS. 1 , 7 and 10 . The network device 900 includes at least one processor 901 , internal connections 902 , memory 903 and at least one transceiver 904 .

The network device 900 is an apparatus with a hardware structure, and can be used to implement the functional modules in the apparatus shown in FIG. 12 . For example, those skilled in the art can think that the recording module 720 in the apparatus shown in FIG. 12 can be implemented by calling the code in the memory 903 by the at least one processor 901, and the receiving module 710 in the apparatus shown in FIG. 12 can be implemented by the at least one processor 901 Transceiver 904 is implemented.

Optionally, the above-mentioned processor 901 may be a general-purpose central processing unit (central processing unit, CPU), a network processor (network processor, NP), a microprocessor, an application-specific integrated circuit (application-specific integrated circuit, ASIC) , or one or more integrated circuits used to control the execution of the program of this application.

The internal connection 902 described above may include a path to transfer information between the above described components. Optionally, the internal connection 902 is a single board or a bus or the like.

The above transceiver 904 is used to communicate with other devices or communication networks.

The above-mentioned memory 903 can be a read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, a random access memory (random access memory, RAM) or other types of storage devices that can store information and instructions. Types of dynamic storage devices, which can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), or other optical storage, CD-ROM storage (including compact discs, laser discs, compact discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by Any other medium accessed by the computer without limitation. The memory can exist independently and be connected to the processor through a bus. The memory can also be integrated with the processor.

Wherein, the memory 903 is used for storing the application program code for executing the solution of the present application, and the execution is controlled by the processor 901. The processor 901 is configured to execute the application program code stored in the memory 903, and cooperate with at least one transceiver 904, so that the network device 900 realizes the functions in this application.

In a specific implementation, as an embodiment, the processor 901 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 14 .

In a specific implementation, as an embodiment, the network device 900 may include multiple processors, for example, the processor 901 and the processor 907 in FIG. 14 . Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

Referring to FIG. 15 , an embodiment of the present application provides a schematic diagram of a controller 1000 . The controller 1000 may be a controller that performs any of the methods of FIGS. 5 , 8 and 11 described above. The controller 1000 includes at least one processor 1001 , internal connections 1002 , memory 1003 and at least one transceiver 1004 .

The controller 1000 is a device with a hardware structure, which can be used to implement the functional modules in the device shown in FIG. 13 . For example, those skilled in the art can think that the computing module 720 in the apparatus shown in FIG. 13 can be implemented by calling the code in the memory 1003 by the at least one processor 1001, and the receiving module 810 in the apparatus shown in FIG. 13 can be implemented by the at least one processor 1001 Transceiver 1004 is implemented.

Optionally, the above-mentioned processor 1001 may be a general-purpose central processing unit (central processing unit, CPU), network processor (network processor, NP), microprocessor, application-specific integrated circuit (application-specific integrated circuit, ASIC) , or one or more integrated circuits used to control the execution of the program of this application.

The internal connection 1002 described above may include a path to transfer information between the aforementioned components. Optionally, the internal connection 1002 is a single board or a bus or the like.

The above transceiver 1004 is used to communicate with other devices or communication networks.

The above-mentioned memory 1003 can be a read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, a random access memory (random access memory, RAM) or other types of storage devices that can store information and instructions. Types of dynamic storage devices, which can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), or other optical storage, CD-ROM storage (including compact discs, laser discs, compact discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by Any other medium accessed by the computer, but not limited to this. The memory can exist independently and be connected to the processor through a bus. The memory can also be integrated with the processor.

Wherein, the memory 1003 is used for storing the application program code for executing the solution of the present application, and the execution is controlled by the processor 1001 . The processor 1001 is configured to execute the application program code stored in the memory 1003, and cooperate with at least one transceiver 1004, so that the controller 1000 can realize the functions in this application.

In a specific implementation, as an embodiment, the processor 1001 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 15 .

In a specific implementation, as an embodiment, the controller 1000 may include multiple processors, such as the processor 1001 and the processor 1007 in FIG. 15 . Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

Embodiments of the present application further provide a computer-readable storage medium, where instructions or programs are stored in the computer-readable storage medium, and the instructions or programs are loaded and executed by a processor to implement the data flow measurement provided by the embodiments of the present application. method.

Embodiments of the present application also provide a computer program product, where the computer program product includes instructions, and the instructions are loaded and executed by a processor to implement the data flow measurement method provided by the embodiments of the present application.

An embodiment of the present application further provides a chip system, where the chip system includes a processor and an interface, where the interface is used to obtain a program or an instruction, and the processor is used to call the program or instruction to implement or support a terminal device to implement any of the above The functions involved in the method embodiments, for example, determine or process the data involved in the above method, and the data may be measured values, flow identifiers, and the like.

In a possible design, the chip system further includes a memory for storing necessary program instructions and data of the terminal device. The chip system may be composed of chips, or may include chips and other discrete devices.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof, and when implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions, and when the computer program instructions are loaded and executed on a device, all or part of the processes or functions described in the embodiments of the present application are generated. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that the device can access, or a data storage device such as a server, data center, or the like that includes an integration of one or more available media. The usable medium may be a magnetic medium (such as a floppy disk, a hard disk, and a magnetic tape, etc.), an optical medium (such as a digital video disk (Digital Video Disk, DVD), etc.), or a semiconductor medium (such as a solid-state disk, etc.).

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, etc.

The above descriptions are only examples of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application should be included within the protection scope of the present application.

Claims

A method for data flow measurement, wherein the method is applied to a network device, and the method includes:

Receive the target data packet, and for each data row of the target data table, determine the target data stream to which the target data packet belongs based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet the corresponding data storage space in the data row;

If the target data stream is the first data stream corresponding to the data storage space, determining the measurement value of the target data stream based on the measurement value of the target data packet, and recording it in the data storage space;

If the target data stream is not the first data stream corresponding to the data storage space, the measurement value of the target data stream is recorded in the target data table based on the measurement value of the target data packet.
The method according to claim 1, wherein, if the target data stream is a first data stream corresponding to the data storage space, determining the target data based on a measurement value of the target data packet Flow measurements, recorded in the data storage space, including:

If the flow identifier and measurement value of the first data flow are not recorded in the data storage space, determine that the target data flow is the first data flow corresponding to the data storage space, and use the measurement value of the target data packet As the measured value of the target data stream, the stream identifier of the target data stream is used as the stream identifier of the first data stream corresponding to the data storage space, and the stream identifier of the target data and the measured value are recorded correspondingly in the data storage space. in the data storage space;

If the stream identifier and measurement value of the first data stream are recorded in the data storage space, and the stream identifier of the target data stream is the same as the stream identifier of the first data stream recorded in the data storage space, the The measured value of the target data packet is accumulated to the measured value of the first data stream recorded in the data storage space;

If the flow identifier and measurement value of the first data stream are recorded in the data storage space, and the stream identifier of the target data stream is different from the stream identifier of the first data stream recorded in the data storage space, it is determined that the Whether the comparative measurement value is recorded in the data storage space, if there is no comparative measurement value recorded, then the measurement value of the target data packet is recorded in the data storage space as the comparative measurement value corresponding to the data storage space, If a comparative measurement value is recorded, accumulating the measured value of the target data packet on the comparative measurement value recorded in the data storage space;

If the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the first data stream recorded in the data storage space, it is determined that the target data stream is the first data corresponding to the data storage space stream, update the stream identifier of the first data stream recorded in the data storage space to the stream identifier of the target data stream, and update the measured value of the first data stream recorded in the data storage space to the target data The measured value of the package, the comparative measured value recorded in the data storage space is updated to 0.
The method according to claim 2, wherein, if the flow identifier and measurement value of the first data stream are recorded in the data storage space, and the flow identifier of the target data stream is the same as that recorded in the data storage space The stream identifiers of the first data streams are different, and the method further includes:

If the accumulated comparative measurement value recorded in the data storage space is smaller than the measurement value of the first data stream recorded in the data storage space, it is determined that the target data stream is not the first data corresponding to the data storage space The flow, based on the measurement value of the target data packet, records the measurement value of the target data flow in the target data table.
The method according to claim 3, wherein if the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the first data stream recorded in the data storage space, the method further comprises: :

sending the storage space identifier of the data storage space, the flow identifier of the first data stream recorded in the data storage space, and the measurement value of the first data stream to the controller;

If the accumulated comparative measurement value recorded in the data storage space is smaller than the measurement value of the first data stream recorded in the data storage space, the method further includes:

sending the flow identification corresponding to the target data packet to the controller as the flow identification of the second data flow;

The method also includes:

Every time a preset time period elapses, the locally stored target data table is sent to the controller, and the locally stored target data table is initialized.
The method according to claim 4, wherein before the sending the flow identifier corresponding to the target data packet as the flow identifier of the second data stream to the controller, the method further comprises:

It is determined that the flow identifier corresponding to the target data packet is not sent to the controller, and the flow identifier corresponding to the target data packet is recorded.
The method according to claim 5, wherein the determining that the flow identifier corresponding to the target data packet is not sent to the controller, and recording the flow identifier corresponding to the target data packet, comprises:

It is determined that the flow identifier corresponding to the target data packet is not recorded in the locally stored Bloom filter, and the flow identifier corresponding to the target data packet is recorded in the Bloom filter.
The method according to any one of claims 1-6, wherein, based on the measurement value of the target data packet, recording the measurement value of the target data stream in the target data table, comprising:

Based on the second eigenvalue calculation function corresponding to the data row and the flow identifier of the target data packet, an accumulation coefficient corresponding to the target data flow is determined, and the measurement value of the target data packet is multiplied by the accumulation coefficient, The adjusted measurement value of the target data packet is obtained, and the adjusted measurement value of the target data packet is accumulated to the measurement value of the second data stream recorded in the data storage space.
A method for data flow measurement, characterized in that the method is used in a network device, and the method includes:

Receive the target data packet, and for each data row of the target data table, determine the target data stream to which the target data packet belongs based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet in the data storage space corresponding to the data row;

Record the measurement value of the target data stream in the data storage space based on the measurement value corresponding to the target data packet;

If the target data stream is the first data stream corresponding to the data storage space, the measurement value of the target data stream is determined based on the measurement value of the target data packet and recorded in the data storage space.
The method according to claim 8, wherein, if the target data stream is the first data stream corresponding to the data storage space, determining the target data based on the measurement value of the target data packet Flow measurements, recorded in the data storage space, including:

If the flow identifier and measurement value of the first data flow are not recorded in the data storage space, determine that the target data flow is the first data flow corresponding to the data storage space, and use the measurement value of the target data packet As the measured value of the target data stream, the stream identifier of the target data stream is used as the stream identifier of the first data stream corresponding to the data storage space, and the stream identifier of the target data and the measured value are recorded correspondingly in the data storage space. in the data storage space;

If the stream identifier and measurement value of the first data stream are recorded in the data storage space, and the stream identifier of the target data stream is the same as the stream identifier of the first data stream recorded in the data storage space, the The measured value of the target data packet is accumulated to the measured value of the first data stream recorded in the data storage space;

If the flow identifier and measurement value of the first data stream are recorded in the data storage space, and the stream identifier of the target data stream is different from the stream identifier of the first data stream recorded in the data storage space, it is determined that the Whether the comparative measurement value is recorded in the data storage space, if there is no comparative measurement value recorded, then the measurement value of the target data packet is recorded in the data storage space as the comparative measurement value corresponding to the data storage space, If a comparative measurement value is recorded, accumulating the measured value of the target data packet on the comparative measurement value recorded in the data storage space;

If the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the first data stream recorded in the data storage space, it is determined that the target data stream is the first data corresponding to the data storage space stream, update the stream identifier of the first data stream recorded in the data storage space to the stream identifier of the target data stream, and update the measured value of the first data stream recorded in the data storage space to the target data The measured value of the package, the comparative measured value recorded in the data storage space is updated to 0.
The method according to claim 9, wherein if the accumulated comparative measurement value recorded in the data storage space is greater than the measurement value of the first data stream recorded in the data storage space, the method further comprises: :

sending the storage space identifier of the data storage space, the flow identifier of the first data stream correspondingly recorded in the data storage space, and the measurement value of the first data stream to the controller;

The method also includes:

If the accumulated comparative measurement value recorded in the data storage space is smaller than the measurement value of the first data flow recorded in the data storage space, the flow identifier corresponding to the target data packet is used as the flow of the second data flow sending an identification to the controller;

Every time a preset time period elapses, the locally stored target data table is sent to the controller, and the locally stored target data table is initialized.
The method according to claim 10, wherein before the sending the flow identifier corresponding to the target data packet as the flow identifier of the second data stream to the controller, the method further comprises:

It is determined that the flow identifier corresponding to the target data packet is not sent to the controller, and the flow identifier corresponding to the target data packet is recorded.
method according to claim 11, is characterized in that, described determining does not send the flow mark corresponding to described target data packet to described controller, recording the flow mark corresponding to described target data packet, comprising:

It is determined that the flow identifier corresponding to the target data packet is not recorded in the locally stored Bloom filter, and the flow identifier corresponding to the target data packet is recorded in the Bloom filter.
The method according to any one of claims 8-12, wherein the measurement value of the target data stream is recorded in the data storage space based on the measurement value corresponding to the target data packet, include:

Based on the second eigenvalue calculation function corresponding to the data row and the flow identifier of the target data packet, an accumulation coefficient corresponding to the target data flow is determined, and the measurement value of the target data packet is multiplied by the accumulation coefficient, The adjusted measurement value of the target data packet is obtained, and the adjusted measurement value of the target data packet is accumulated to the measurement value of the data flow recorded in the data storage space.
A method for data flow measurement, wherein the method is applied to a controller, and the method comprises:

receiving the stream identifier of the first data stream, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the stream identifier of the second data stream, and the target data table sent by the network device;

In the measurement values of the first data stream recorded in the target data table, the measurement values of the first data streams corresponding to the same flow identifier are filtered to obtain the first data stream of each first data stream recorded in the target data table. Measurements;

In the received measured values of the first data stream, the measured values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain a second received stream identifier corresponding to each first data stream. Measurements;

In the received second measurement value corresponding to the flow identifier of each first data stream, the second measurement value of the first data stream corresponding to the same flow identifier is filtered to obtain the received data of each first data stream. the third measurement value corresponding to the flow identifier;

In the first measurement value of each first data stream recorded in the target data table and the third measurement value corresponding to the received stream identifier of each first data stream, the first measurement value corresponding to the same stream identifier is Accumulate with the third measurement value to obtain the statistical measurement value corresponding to the stream identifier of each first data stream;

Based on the received flow identification of the second data flow and the target data table, a statistical measurement value corresponding to the flow identification of each second data flow is determined.
The method according to claim 14, wherein, in the measurement values of the first data stream recorded in the target data table, the measurement values of the first data stream corresponding to the same stream identifier are filtered to obtain the The first measurement value of each first data stream recorded in the target data table, including:

Among the measured values of the first data stream recorded in the target data table, the maximum measured value of the first data stream corresponding to the same stream identifier is used as the first measured value of the first data stream corresponding to the same stream identifier;

In the received second measurement value corresponding to the flow identifier of each first data stream, the second measurement value of the first data stream corresponding to the same stream identifier is screened to obtain each received first data stream. The third measurement value corresponding to the flow identifier of the flow, including:

In the received second measurement value corresponding to the flow identifier of each first data stream, the maximum second measurement value of the first data stream corresponding to the same flow identifier is taken as the first data stream corresponding to the same stream identifier the third measurement.
The method according to claim 14 or 15, wherein the statistical measurement value corresponding to the flow identifier of each second data stream is determined based on the received stream identifier of the second data stream and the target data table ,include:

For each data storage space in the target data table, based on the first eigenvalue calculation function corresponding to the data row where the data storage space is located, the data storage The stream identifier of the second data stream corresponding to the space, and based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located, determine the accumulation coefficient corresponding to the stream identifier of each second data stream corresponding to the data storage space ;

The statistical measurement value corresponding to the stream identifier of each second data stream corresponding to the data storage space is multiplied by the corresponding accumulation coefficient as an unknown, and the accumulation is performed, and the measurement value of the data stream recorded in the data storage space is used as the accumulation result. , obtain the target equation corresponding to the data storage space;

Based on the target equation corresponding to each data storage space in the target data table, a target equation group is formed;

Based on a linear optimization algorithm, taking the minimum sum of the statistical measurement values corresponding to the received flow identifiers of the second data stream as the optimization objective, solving the objective equation system to obtain the statistical measurement values corresponding to the flow identifiers of each second data stream .
The method of claim 15, wherein the method further comprises:

In the obtained statistical measurement values corresponding to the flow ID of the first data stream and the statistical measurement values corresponding to the flow ID of the second data stream, the measurement values corresponding to the same flow ID are accumulated to obtain the total measurement value corresponding to each flow ID value.
A method for data flow measurement, wherein the method is applied to a controller, and the method comprises:

receiving the stream identifier of the first data stream, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the stream identifier of the second data stream, and the target data table sent by the network device;

In the measurement values of the first data stream recorded in the target data table, the measurement values of the first data streams corresponding to the same flow identifier are filtered to obtain the first data stream of each first data stream recorded in the target data table. Measurements;

In the received measured values of the first data stream, the measured values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain a second received stream identifier corresponding to each first data stream. Measurements;

In the received second measurement value corresponding to the flow identifier of each first data stream, the second measurement value corresponding to the stream identifier of the first data stream corresponding to the same flow identifier is screened to obtain each received first data stream. A third measurement value corresponding to a stream identifier of a data stream;

In the first measurement value of each first data stream recorded in the target data table and the third measurement value corresponding to the received stream identifier of each first data stream, the first measurement value corresponding to the same stream identifier is Accumulate with the second measurement value to obtain the statistical measurement value corresponding to the stream identifier of each first data stream;

Based on the received flow identifier of the second data stream, the measured value of the data stream recorded in each data storage space in the target data table, and the obtained statistical measurement value corresponding to the stream identifier of each first data stream, determine each The flow identifier of the second data flow corresponds to the statistical measurement value.
The method according to claim 18, characterized in that, based on the received flow identifier of the second data flow, the measured value of the data flow recorded in each of the data storage spaces in the target data table, and the obtained The statistical measurement value corresponding to the flow identifier of each first data stream, and determining the statistical measurement value corresponding to the flow identifier of each second data stream, including:

For each data storage space of the target data table, based on the second eigenvalue calculation function corresponding to the data row where the data storage space is located, determine the corresponding flow identifier of each first data stream corresponding to the data storage space cumulative coefficient;

Multiply the statistical measurement value corresponding to the stream identifier of each first data stream corresponding to the data storage space by the corresponding accumulation coefficient to obtain the adjustment statistics corresponding to the stream identifier of each first data stream corresponding to the data storage space Measurements;

Based on the first eigenvalue calculation function corresponding to the data row where the data storage space is located, in the received flow identifier of the second data stream, the stream identifier of the second data stream corresponding to the data storage space is determined, and based on the received stream identifier of the second data stream The second eigenvalue calculation function corresponding to the data row where the data storage space is located, to determine the accumulation coefficient corresponding to the stream identifier of each second data stream corresponding to the data storage space;

Multiply the statistical measurement value corresponding to the stream identifier of each second data stream corresponding to the data storage space by the corresponding accumulation coefficient as an unknown, and adjust the adjustment corresponding to the stream identifier of each first data stream corresponding to the data storage space Statistical measurement values are added, and the measurement values of the data streams recorded in the data storage space are used as an accumulation result to obtain a target equation corresponding to the data storage space pair;

Based on the target equation corresponding to each data storage space in the target data table, a target equation group is formed;

Based on a linear optimization algorithm, taking the minimum sum of the statistical measurement values corresponding to the received flow identifiers of the second data stream as the optimization objective, solving the objective equation system to obtain the statistical measurement values corresponding to the flow identifiers of each second data stream .
A device for data flow measurement, characterized in that the device comprises:

A transceiver module, configured to receive a target data packet, and for each data row of the target data table, determine the target data packet based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet the data storage space corresponding to the target data stream to which it belongs in the data row;

A processing module, configured to, if the target data stream is the first data stream corresponding to the data storage space, determine the measurement value of the target data stream based on the measurement value of the target data packet, and record it in the data In the storage space; if the target data stream is not the first data stream corresponding to the data storage space, based on the measurement value of the target data packet, record the measurement value of the target data stream in the target data table.
A device for data flow measurement, characterized in that the device comprises:

A transceiver module, configured to receive a target data packet, and for each data row of the target data table, determine the target data packet based on the first feature value calculation function corresponding to the data row and the flow identifier corresponding to the target data packet The target data stream to which it belongs is in the data storage space corresponding to the data row;

a processing module, configured to record the measurement value of the target data stream in the data storage space based on the measurement value corresponding to the target data packet; if the target data stream is the first corresponding to the data storage space data flow, the measurement value of the target data flow is determined based on the measurement value of the target data packet and recorded in the data storage space.
A device for data flow measurement, characterized in that the device comprises:

a receiving module, configured to receive the flow identifier and the corresponding measurement value of the first data stream sent by the network device, the stream identifier of the second data stream, and the target data table;

The processing module is used to filter the measured values of the first data stream corresponding to the same stream identifier in the measured values of the first data stream recorded in the target data table, and obtain each first data stream recorded in the target data table. The first measurement value of the data stream; among the received measurement values of the first data stream, the measurement values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain each received first data stream. The second measurement value corresponding to the flow identifier of the stream; in the received second measurement value corresponding to the stream identifier of each first data stream, the second measurement value of the first data stream corresponding to the same stream identifier is screened, Obtain the third measurement value corresponding to the stream identifier of each first data stream received; the first measurement value of each first data stream recorded in the target data table and the received value of each first data stream In the third measurement value corresponding to the flow identification, the first measurement value and the third measurement value corresponding to the same flow identification are accumulated to obtain the statistical measurement value corresponding to the flow identification of each first data flow; The stream identifier of the data stream and the target data table determine the statistical measurement value corresponding to the stream identifier of each second data stream.
A device for data flow measurement, characterized in that the device comprises:

a receiving module, configured to receive the stream identifier of the first data stream, the corresponding measurement value and the storage space identifier of the corresponding data storage space, the stream identifier of the second data stream, and the target data table sent by the network device;

The processing module is used to filter the measured values of the first data stream corresponding to the same stream identifier in the measured values of the first data stream recorded in the target data table, and obtain each first data stream recorded in the target data table. The first measurement value of the data stream; among the received measurement values of the first data stream, the measurement values of the first data streams corresponding to the same storage space identifier and the same stream identifier are accumulated to obtain each received first data stream. The second measurement value corresponding to the flow identifier of the stream; in the received second measurement value corresponding to the stream identifier of each first data stream, the second measurement value corresponding to the stream identifier of the first data stream corresponding to the same stream identifier value is screened to obtain a third measurement value corresponding to the stream identifier of each first data stream received; the first measurement value of each first data stream recorded in the target data table and each received In the third measurement value corresponding to the flow ID of a data stream, the first measurement value and the second measurement value corresponding to the same flow ID are accumulated to obtain the statistical measurement value corresponding to the flow ID of each first data stream; The flow identification of the obtained second data flow, the measured value of the data flow recorded in each data storage space in the target data table, and the obtained statistical measurement value corresponding to the flow identification of each first data flow, determine each The statistical measurement values corresponding to the flow identifiers of the two data streams.
A network device, characterized in that the network device includes a processor and a memory, wherein:

The memory stores instructions that are executed by the processor to implement the method of data flow measurement as claimed in any one of claims 1 to 13.
A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and the instructions are loaded and executed by a processor, so as to realize any one of claims 1 to 13. method of data flow measurement.
A controller, characterized in that the controller includes a processor and a memory, wherein:

The memory stores instructions that are executed by the processor to implement the method of data flow measurement as claimed in any one of claims 14 to 19.
A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and the instructions are loaded and executed by a processor, so as to realize any one of claims 14 to 19. method of data flow measurement.
A system for data flow measurement, characterized in that the system includes a network device and a controller, wherein:

The network device is used for performing the data flow measurement method according to any one of claims 1 to 7; the controller is used for performing the data flow according to any one of claims 14 to 17 method of flow measurement.
A system for data flow measurement, characterized in that the system includes a network device and a controller, wherein:

The network device is configured to perform the data flow measurement method according to any one of claims 8 to 13 ; the controller is configured to perform the data flow measurement method according to claim 18 or 19 .