CN113726591A - Secondary sampling method suitable for NetFlow message distribution - Google Patents
Secondary sampling method suitable for NetFlow message distribution Download PDFInfo
- Publication number
- CN113726591A CN113726591A CN202110856588.0A CN202110856588A CN113726591A CN 113726591 A CN113726591 A CN 113726591A CN 202110856588 A CN202110856588 A CN 202110856588A CN 113726591 A CN113726591 A CN 113726591A
- Authority
- CN
- China
- Prior art keywords
- message
- netflow
- secondary sampling
- subsampling
- extracted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/18—Protocol analysers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a secondary sampling method suitable for NetFlow message distribution, wherein the method comprises the following steps: receiving a NetFlow message sent by slave equipment by a NetFlow secondary sampling device; the NetFlow secondary sampling device carries out secondary sampling processing on the NetFlow message according to a secondary sampling rule; the subsampling rule contains the NetFlow source device address and the subsampling ratio. The method can reduce the quantity of the NetFlow messages of the equipment within an acceptable error range.
Description
Technical Field
The invention relates to the field of configured NetFlow flow statistics, in particular to a secondary sampling method suitable for NetFlow message distribution.
Background
The current core network equipment has huge flow, even if the sampling ratio of configured NetFlow flow statistics is 1:1000, the quantity of NetFlow messages output by the equipment is not a small quantity, and the acquisition machine has an upper limit on the NetFlow message processing quantity in the same time, so that the analysis of the large quantity of NetFlow messages of a single equipment cannot be met.
Disclosure of Invention
In order to overcome the technical problems, the invention provides a secondary sampling method suitable for NetFlow message distribution, which can effectively reduce the message quantity of NetFlow and reduce the load of a collector.
In order to achieve the purpose, the invention adopts the following technical scheme:
in an embodiment of the present invention, a secondary sampling method suitable for NetFlow message distribution is provided, where the method includes:
receiving a NetFlow message sent by slave equipment by a NetFlow secondary sampling device;
the NetFlow secondary sampling device carries out secondary sampling processing on the NetFlow message according to a secondary sampling rule; the subsampling rule contains the NetFlow source device address and the subsampling ratio.
Further, performing secondary sampling processing on the NetFlow message according to a secondary sampling rule, including:
analyzing the NetFlow message, and judging whether the message is a template message or not;
if the template message is the message, the message is not subjected to secondary sampling processing, and the message is output after the serial number in the message is refreshed according to the new serial number maintained by the Source ID;
if the message is a non-template message, judging whether the message is extracted according to the secondary sampling ratio, if the message is allowed to be extracted, refreshing the serial number in the message according to a new serial number maintained by the SourceID, and then outputting the message, otherwise, the message is not allowed to be extracted and is directly discarded.
Further, determining whether the message is a template message includes:
and judging the message according to the characteristics that the version of the NetFlow protocol and the flow template number are 0.
Further, judging whether the message is extracted according to the secondary sampling ratio comprises:
judging that the current message is the second message according to the secondary sampling ratio of 1: N;
if the current message is a multiple of the Nth message, the message is allowed to be extracted, otherwise, the message is not allowed to be extracted.
Further, the NetFlow subsampling device comprises two interfaces: a NetFlow message interface and a secondary sampling rule interface.
In an embodiment of the present invention, a computer device is further provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the processor implements the above-mentioned subsampling method suitable for NetFlow message distribution.
In an embodiment of the present invention, a computer-readable storage medium is further provided, where the computer-readable storage medium stores a computer program for executing a subsampling method suitable for NetFlow message distribution.
Has the advantages that:
the invention can reduce the quantity of the NetFlow messages of the equipment within an acceptable error range by the NetFlow secondary sampling device.
Drawings
Fig. 1 is a flow chart of a secondary sampling method model suitable for NetFlow message distribution according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a secondary sampling method applied to NetFlow message distribution according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The principles and spirit of the present invention will be described below with reference to several exemplary embodiments, which should be understood to be presented only to enable those skilled in the art to better understand and implement the present invention, and not to limit the scope of the present invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
According to the embodiment of the invention, the secondary sampling method suitable for distribution of the NetFlow message is provided, and the NetFlow message is subjected to secondary sampling processing mainly through a NetFlow secondary sampling device to obtain the NetFlow message after secondary sampling.
The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the invention.
Fig. 1 is a flow chart of a subsampling method model suitable for NetFlow message distribution according to an embodiment of the present invention. As shown in fig. 1, the NetFlow subsampling device mainly includes two interfaces: a NetFlow message interface and a secondary sampling rule interface. And performing secondary sampling processing on the NetFlow message through a NetFlow secondary sampling device.
In the invention, the NetFlow message is network flow data based on equipment statistics, and the equipment outputs the flow data through a NetFlow protocol.
In the present invention, the subsampling rule contains the NetFlow source device address and the subsampling ratio. For example, table 1 below:
TABLE 1
| Source device address | Sub-sampling ratio |
| 1.1.1.1 | 1:4 |
| 1.1.1.2 | 1:2 |
Fig. 2 is a schematic flow chart of a subsampling method suitable for NetFlow message distribution according to an embodiment of the present invention. As shown in fig. 2, in the present invention, after receiving the NetFlow message, the processing is performed according to the following rules:
1. analyzing the NetFlow message, judging whether the message is a template message according to the characteristics that the NetFlow protocol version and FlowSet (flow template number) are 0, if so, executing the step 2, otherwise, executing the step 3;
2. if the message is a template message, the message is not subjected to secondary sampling processing, and the message is output after the serial number in the message is refreshed according to a new serial number maintained by a Source Engine code.
3. If the current message is a multiple of the Nth message, the message is allowed to be extracted, otherwise, the extraction is not allowed; and directly discarding the extracted message which is not allowed to be extracted, and outputting the message after refreshing the serial number in the message according to the new serial number maintained by the Source ID.
It should be noted that although the operations of the method of the present invention have been described in the above embodiments and the accompanying drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the operations shown must be performed, to achieve the desired results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
For a clearer explanation of the above sub-sampling method applied to NetFlow message distribution, a specific embodiment is described below, but it should be noted that the embodiment is only for better explaining the present invention, and does not limit the present invention unduly.
Example (c):
1. when a NetFlow message is received, it is known to be a template message according to the FlowSet attribute, and the sequence number of the message is maintained according to SourceID 17039106, for example, the current sequence number is 1.
NetFlow message:
2. after the Flowsequence of the current message is modified into a sequence number 1, the message is output, and the current sequence number is 2;
3. after receiving a NetFlow message, if it is known to be a data message according to the FlowSet attribute, judging whether the message needs to be forwarded according to a secondary sampling rule, for example, if the message needs to be forwarded currently, checking that the current serial number is 2 according to the SourceID, modifying FlowSqeuence of the current message into the serial number 2, and then outputting the message.
NetFlow message:
based on the aforementioned inventive concept, as shown in fig. 3, the present invention further provides a computer device 200, which includes a memory 210, a processor 220, and a computer program 230 stored on the memory 210 and operable on the processor 220, wherein the processor 220 implements the aforementioned sub-sampling method suitable for NetFlow message distribution when executing the computer program 230.
Based on the foregoing inventive concept, the present invention further provides a computer-readable storage medium storing a computer program for executing the foregoing subsampling method suitable for NetFlow message distribution.
The secondary sampling method suitable for the distribution of the NetFlow message provided by the invention reduces the quantity of the NetFlow message of the equipment within an acceptable error range.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
The limitation of the protection scope of the present invention is understood by those skilled in the art, and various modifications or changes which can be made by those skilled in the art without inventive efforts based on the technical solution of the present invention are still within the protection scope of the present invention.
Claims (7)
1. A subsampling method suitable for NetFlow message distribution is characterized by comprising the following steps:
receiving a NetFlow message sent by slave equipment by a NetFlow secondary sampling device;
the NetFlow secondary sampling device carries out secondary sampling processing on the NetFlow message according to a secondary sampling rule; the subsampling rule contains the NetFlow source device address and the subsampling ratio.
2. The method of claim 1, wherein the performing the subsampling processing on the NetFlow packet according to the subsampling rule comprises:
analyzing the NetFlow message, and judging whether the message is a template message or not;
if the template message is the message, the message is not subjected to secondary sampling processing, and the message is output after the serial number in the message is refreshed according to the new serial number maintained by the Source ID;
if the message is a non-template message, judging whether the message is extracted according to the secondary sampling ratio, if the message is allowed to be extracted, refreshing the serial number in the message according to a new serial number maintained by the SourceID, and then outputting the message, otherwise, the message is not allowed to be extracted and is directly discarded.
3. The method according to claim 2, wherein the determining whether the message is a template message comprises:
and judging the message according to the characteristics that the version of the NetFlow protocol and the flow template number are 0.
4. The method according to claim 2, wherein the determining whether the packet is extracted according to the subsampling ratio comprises:
judging that the current message is the second message according to the secondary sampling ratio of 1: N;
if the current message is a multiple of the Nth message, the message is allowed to be extracted, otherwise, the message is not allowed to be extracted.
5. The method for performing subsampling during NetFlow packet distribution according to claim 1, wherein the NetFlow subsampling device comprises two interfaces: a NetFlow message interface and a secondary sampling rule interface.
6. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1-5 when executing the computer program.
7. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110856588.0A CN113726591B (en) | 2021-07-28 | 2021-07-28 | Secondary sampling method suitable for NetFlow message distribution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110856588.0A CN113726591B (en) | 2021-07-28 | 2021-07-28 | Secondary sampling method suitable for NetFlow message distribution |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113726591A true CN113726591A (en) | 2021-11-30 |
| CN113726591B CN113726591B (en) | 2023-02-21 |
Family
ID=78674103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110856588.0A Active CN113726591B (en) | 2021-07-28 | 2021-07-28 | Secondary sampling method suitable for NetFlow message distribution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113726591B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104579810A (en) * | 2013-10-23 | 2015-04-29 | 中兴通讯股份有限公司 | Flow sampling method and system for software-defined network |
| CN107332732A (en) * | 2017-06-26 | 2017-11-07 | 迈普通信技术股份有限公司 | A kind of method of sampling of message flow, device and routing device |
| CN108183864A (en) * | 2018-01-29 | 2018-06-19 | 中国人民解放军国防科技大学 | IDS feedback-based software-defined network flow sampling method and system |
| CN110545199A (en) * | 2019-07-24 | 2019-12-06 | 浪潮思科网络科技有限公司 | SDN network flow statistical device and method based on Netflow |
| CN111143554A (en) * | 2019-12-10 | 2020-05-12 | 中盈优创资讯科技有限公司 | Data sampling method and device based on big data platform |
-
2021
- 2021-07-28 CN CN202110856588.0A patent/CN113726591B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104579810A (en) * | 2013-10-23 | 2015-04-29 | 中兴通讯股份有限公司 | Flow sampling method and system for software-defined network |
| CN107332732A (en) * | 2017-06-26 | 2017-11-07 | 迈普通信技术股份有限公司 | A kind of method of sampling of message flow, device and routing device |
| CN108183864A (en) * | 2018-01-29 | 2018-06-19 | 中国人民解放军国防科技大学 | IDS feedback-based software-defined network flow sampling method and system |
| CN110545199A (en) * | 2019-07-24 | 2019-12-06 | 浪潮思科网络科技有限公司 | SDN network flow statistical device and method based on Netflow |
| CN111143554A (en) * | 2019-12-10 | 2020-05-12 | 中盈优创资讯科技有限公司 | Data sampling method and device based on big data platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113726591B (en) | 2023-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2018532171A (en) | SQL examination method, server and storage device | |
| CN111585344B (en) | Substation intelligent checking method and device based on total station IED simulation | |
| CN111008246B (en) | Database log synchronization method, device, computer equipment and readable storage medium | |
| EP4170514A1 (en) | Data association query method and apparatus, and device and storage medium | |
| CN108664635B (en) | Method, device, equipment and storage medium for acquiring database statistical information | |
| CN107665170B (en) | Flow testing method and device | |
| CN108549688B (en) | Data operation optimization method, device, equipment and storage medium | |
| CN112751788A (en) | Double-plane switching method supporting multi-type frame mixed transmission | |
| CN111427784A (en) | Data acquisition method, device, equipment and storage medium | |
| CN112040001A (en) | Request processing method and device based on distributed storage | |
| CN105933208A (en) | Message processing method and device | |
| CN113726591B (en) | Secondary sampling method suitable for NetFlow message distribution | |
| CN113157904B (en) | Sensitive word filtering method and system based on DFA algorithm | |
| CN112614002A (en) | Data acquisition system, method, device, electronic equipment and computer storage medium | |
| CN110109672B (en) | Analysis processing method and device for expression | |
| CN113032341A (en) | Log processing method based on visual configuration | |
| CN111211939A (en) | Device and method for realizing efficient flow table counting based on network processor | |
| CN114443032A (en) | Form processing method, device, terminal and storage medium based on JSON schema | |
| CN115510104A (en) | Distributed database-based most-valued information extraction method and related equipment | |
| CN112818183B (en) | Data synthesis method, device, computer equipment and storage medium | |
| CN113918566A (en) | NetFlow data storage method and device based on column storage | |
| CN112217896A (en) | JSON message conversion method and related device | |
| CN116361586B (en) | Method for realizing HTTP protocol request data highlighting in webpage | |
| CN115277881B (en) | Network message analysis method and device | |
| CN114710237B (en) | Data processing method and device of communication interface, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: 200000 room 702-2, No. 4811 Cao'an Road, Jiading District, Shanghai Patentee after: CHINA UNITECHS Address before: Room 1004-4, 10 / F, 1112 Hanggui Road, Anting Town, Jiading District, Shanghai Patentee before: CHINA UNITECHS |