CN110865965A - Method and device for realizing flow table bidirectional data synchronization based on hardware - Google Patents

Method and device for realizing flow table bidirectional data synchronization based on hardware Download PDF

Info

Publication number
CN110865965A
CN110865965A CN201911109547.4A CN201911109547A CN110865965A CN 110865965 A CN110865965 A CN 110865965A CN 201911109547 A CN201911109547 A CN 201911109547A CN 110865965 A CN110865965 A CN 110865965A
Authority
CN
China
Prior art keywords
ipfix
data
egress
processing module
count field
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911109547.4A
Other languages
Chinese (zh)
Other versions
CN110865965B (en
Inventor
姚飞
周伟
单哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Sheng Ke Science And Technology Co Ltd
Original Assignee
Suzhou Sheng Ke Science And Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Sheng Ke Science And Technology Co Ltd filed Critical Suzhou Sheng Ke Science And Technology Co Ltd
Priority to CN201911109547.4A priority Critical patent/CN110865965B/en
Publication of CN110865965A publication Critical patent/CN110865965A/en
Application granted granted Critical
Publication of CN110865965B publication Critical patent/CN110865965B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a method and a device for realizing flow table bidirectional data synchronization based on hardware, wherein the method comprises the following steps: the ingress IPFIX processing module sends the data message, the key word index and the ingress flow modification count field to the egress IPFIX processing module, the egress IPFIX processing module judges whether the ingress flow modification count field is equal to the egress flow modification count field, and if so, the data message continues to be processed in the egress direction by IPFIX. The invention realizes that the statistical data of the flow in the outlet direction and the key representing the flow in the inlet direction can be synchronized without generating mismatching.

Description

Method and device for realizing flow table bidirectional data synchronization based on hardware
Technical Field
The present invention relates to a flow table bidirectional data synchronization technology, and in particular, to a method and an apparatus for implementing flow table bidirectional data synchronization based on hardware.
Background
When the network has been laid, the network administrator cannot analyze the traffic status transmitted in the network by an effective means. IPFIX (IP Flow Information Export) exists to solve the problem, and IPFIX filters traffic in a network by keys (key words), and one key represents a certain data Flow in the network. For each data flow, statistics is performed on information such as the number, size, delay, destination, etc. of the messages in the data flow in the ingress direction and the egress direction, and the statistics is referred to as ingress data and egress data.
In actual hardware implementation, there are two ways to handle the relation between data in the two directions of key and data flow, one of the ways is to use two sets of key and data, that is, two sets of key and data, namely, entry key + entry data and exit key + exit data, but this scheme has a problem that the entry key and the exit key cannot match the same data flow because there may be editing behavior in the process of a message from entry to exit, so that an administrator cannot view the data condition of a certain data flow in the entry direction and the exit direction.
The other scheme is that keys exist only in the inlet direction, and inlet data and outlet data exist in the inlet direction and the outlet direction of the chip respectively. This approach addresses the shortcomings of the above approach, allowing an administrator to accurately see the information and status of a data stream from egress to egress. However, this scheme also causes a problem that the relationship between the egress data and the key cannot be correlated because the key and the data exist in a separate state, and the synchronization problem of the egress data is caused by the time delay and the aging of the data stream during the transmission of the actual data.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method and a device for realizing flow table bidirectional data synchronization based on hardware.
In order to achieve the purpose, the invention provides the following technical scheme: a method for realizing bidirectional data synchronization of a flow table based on hardware comprises the following steps:
s1, generating key index and adding inlet flow modifying count field in the inlet IPFIX processing module of the chip according to the data message in advance, and adding outlet flow modifying count field in the outlet IPFIX processing module;
s2, the entrance IPFIX processing module sends the data message, the key word index and the entrance flow modification count field to the exit IPFIX processing module;
s3, the egress IPFIX processing module receives the data packet and the corresponding bus data, extracts the key index and the ingress flow modification count field from the bus data, reads egress data from an egress data store using the key index, and extracts the egress flow modification count field from the egress data;
and S4, the outlet IPFIX processing module continuously judges whether the inlet flow modification count field and the outlet flow modification count field are equal, if so, the data message is continuously processed by IPFIX in the outlet direction.
Preferably, the ingress IPFIX processing module extracts a corresponding field of the data packet as an IPFIX keyword, and then calculates the keyword index by using a hash algorithm for the IPFIX keyword.
Preferably, the initial values of the ingress flow modification count field and the egress flow modification count field are both set to 0.
Preferably, in S4, if the data packets are not equal to each other, ignoring IPFIX processing in the subsequent egress direction of the data packet.
Preferably, when the data stream is aged or the stream information corresponding to the IPFIX key is deleted, the values of the ingress stream modification count field and the egress stream modification count field are incremented at the same time.
The invention also discloses another technical scheme: an apparatus for implementing bidirectional data synchronization of a flow table based on hardware, the apparatus comprising: the system comprises an inlet IPFIX processing module and an outlet IPFIX processing module, wherein the inlet IPFIX processing module generates a keyword index and is added with an inlet flow modification counting field according to a data message in advance, and the outlet IPFIX processing module is added with an outlet flow modification counting field;
the inlet IPFIX processing module is used for transmitting bus data formed by a data message, the key word index and the inlet flow modification count field to the outlet IPFIX processing module;
the egress IPFIX processing module is configured to extract the key index and the ingress flow modification count field from the bus data after receiving the data packet and the corresponding bus data, read egress data from an egress data storage of the egress IPFIX processing module using the key index, and extract the egress flow modification count field from the egress data;
and the outlet IPFIX processing module is further used for continuously judging whether the inlet flow modification count field is equal to the outlet flow modification count field, and if so, continuously processing the data message by IPFIX in the outlet direction.
Preferably, the ingress IPFIX processing module extracts a corresponding field of the data packet as an IPFIX keyword, and then calculates the keyword index by using a hash algorithm for the IPFIX keyword.
Preferably, if the determination result is not equal, the egress IPFIX processing module ignores IPFIX processing in the subsequent egress direction of the data packet.
Preferably, when the data stream is aged or the stream information corresponding to the IPFIX key is deleted, the ingress data processing module increments the ingress stream modification count field and the egress data processing module increments the value of the egress stream modification count field synchronously.
The invention has the beneficial effects that: according to the method, the flow modification count field is inserted into the flow data, so that the statistical data of the flow in the outlet direction and the key representing the flow in the inlet direction can be synchronized, and mismatching cannot be generated.
Drawings
FIGS. 1 and 2 are schematic flow diagrams of the method of the present invention;
fig. 3 is a schematic diagram of an ASIC chip of the present invention.
Detailed Description
The technical solution of the embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings of the present invention.
According to the method and the device for realizing the bidirectional data synchronization of the flow table based on the hardware, the flow modification count field is inserted into the flow data, so that the statistical data of the flow in the outlet direction and the key representing the flow in the inlet direction can be synchronized, and mismatching cannot be generated.
Referring to fig. 1 and fig. 2, a method for implementing bidirectional data synchronization of a flow table based on hardware disclosed in the present invention includes:
s1, generating key index in the entrance IPFIX processing module of the chip according to the data message, adding entrance flow modification count field in the entrance data processing module, and adding exit flow modification count field in the exit data processing module.
Specifically, in the ingress direction of a chip (e.g., ASIC chip), the ingress IPFIX processing module extracts a corresponding field in the data packet as a flow key of the IPFIX according to a flow rule. The flow key is compressed into a key Index by a hash algorithm. Key indexes in the portal IPFIX processing module have three functions, one of which is the Index stored in the Key memory of the portal IPFIX processing module as a Key; its two uses are as the index of the flow data store (i.e. the entry data Memory) of the entry direction; and the flow ID is taken as the flow identifier (flow ID) to the outlet IPFIX processing module in the outlet direction.
Meanwhile, an ingress flow modification count field (ingressFlowChangeCount) for recording an ingress flow modification count is added to the ingress IPFIX processing module, and an initial value of the ingressFlowChangeCount is set to 0. And adding an egress flow modification count field (egressFlowChangeCount) in the egress IPFIX processing module for recording the egress flow modification count, the initial value of the egressFlowChangeCount being set to 0.
And S2, the entrance IPFIX processing module sends the data message, the key word index and the entrance flow modification count field to the exit IPFIX processing module.
Specifically, as shown in fig. 3, the bus data sequentially passes through the ingress data processing module, the packet scheduling processing module, and the egress data processing module of the chip, and reaches the egress IPFIX processing module.
S3, the egress IPFIX processing module receives the data packet and the corresponding bus data, extracts the key index and the ingress flow modification count field from the bus data, reads egress data from the egress data storage using the key index, and extracts the egress flow modification count field from the egress data.
Specifically, the egress IPFIX processing module receives the data packet and the corresponding bus data, extracts a key index and an ingress flow modification count field from the bus data, uses the key index to store (memory) the egress data of the egress IPFIX processing module, indexes the egress data, and extracts the egress flow modification count field from the egress data.
And S4, the outlet IPFIX processing module continuously judges whether the inlet flow modification count field and the outlet flow modification count field are equal, if so, the data message is continuously processed by IPFIX in the outlet direction.
Specifically, the outlet IPFIX processing module determines whether the values of the inlet flow modification count field of the inlet and the outlet flow modification count field of the outlet are equal, if so, considers that the current outlet data is matched with the IPFIX keyword key of the current inlet IPFIX processing module, and continues to perform IPFIX processing in the outlet direction, otherwise, considers that the current outlet data is not the same flow, and ignores the subsequent processing of the outlet IPFIX. It should be clear that the inequality indicates that the key of the Flow has been deleted, specifically, the key still exists when a certain packet in the Flow is processed by the ingress IPFIX, and the key has been deleted in the time before the packet is transmitted to the processing engine of the egress IPFIX. There are many reasons for deletion, which may be aging (aging) or active deletion of the CPU or other reasons.
In addition, when the data stream is aged or the stream information corresponding to the IPFIX key is deleted, the values of the ingress stream modification count field and the egress stream modification count field are incremented at the same time to maintain synchronization. It should be noted that there are two places where the delete key operation usually occurs in deleting the flow information, one is flow natural aging, which is handled by an aging engine (aging engine) of the ipfix, and the other is deletion processing by issuing an instruction by the CPU. Both the agingengine of the ipfix and the delete instruction processing engine of the cpu need to be able to access the store (memory) for flow statistics in the ingress direction (ingress) ipfix and the egress direction (egress) ipfix simultaneously and update the flow modification count fields in both memories.
In addition, the values of the entry flow modification count field and the exit flow modification count field do not need to be infinitely incremented, a large number needs to be stored in the infinite increment, and only circulation is actually needed. For example, 0, 1, 2, 3, 4, 5, 6.
The invention discloses a device for realizing bidirectional data synchronization of a flow table based on hardware, which comprises: the system comprises an inlet IPFIX processing module and an outlet IPFIX processing module, wherein the inlet IPFIX processing module generates a keyword index and is added with an inlet flow modification counting field according to a data message in advance, and the outlet IPFIX processing module is added with an outlet flow modification counting field;
the inlet IPFIX processing module is used for transmitting bus data formed by a data message, the key word index and the inlet flow modification count field to the outlet IPFIX processing module;
the egress IPFIX processing module is configured to extract the key index and the ingress flow modification count field from the bus data after receiving the data packet and the corresponding bus data, read egress data from an egress data storage of the egress IPFIX processing module using the key index, and extract the egress flow modification count field from the egress data;
and the outlet IPFIX processing module is further used for continuously judging whether the inlet flow modification count field is equal to the outlet flow modification count field, and if so, continuously processing the data message by IPFIX in the outlet direction.
The implementation principle of the inlet IPFIX processing module and the outlet IPFIX processing module may refer to the description of the above steps S1 to S3, which is not described herein again.
Therefore, the scope of the present invention should not be limited to the disclosure of the embodiments, but includes various alternatives and modifications without departing from the scope of the present invention, which is defined by the claims of the present patent application.

Claims (10)

1. A method for implementing bidirectional data synchronization of a flow table based on hardware, the method comprising:
s1, generating key index and adding inlet flow modifying count field in the inlet IPFIX processing module of the chip according to the data message in advance, and adding outlet flow modifying count field in the outlet IPFIX processing module;
s2, the entrance IPFIX processing module sends the data message, the key word index and the entrance flow modification count field to the exit IPFIX processing module;
s3, the egress IPFIX processing module receives the data packet and the corresponding bus data, extracts the key index and the ingress flow modification count field from the bus data, reads egress data from an egress data store using the key index, and extracts the egress flow modification count field from the egress data;
and S4, the outlet IPFIX processing module continuously judges whether the inlet flow modification count field and the outlet flow modification count field are equal, if so, the data message is continuously processed by IPFIX in the outlet direction.
2. The method of claim 1, wherein the ingress IPFIX processing module extracts a corresponding field of a data packet as an IPFIX key, and calculates the key index using a hash algorithm on the IPFIX key.
3. The method of claim 1, wherein the initial value of the ingress flow modification count field and the initial value of the egress flow modification count field are both set to 0.
4. The method according to claim 1, wherein in S4, if the difference is not equal, the IPFIX processing in the subsequent egress direction of the data packet is ignored.
5. The method of claim 2, wherein when data stream is aged or the flow information corresponding to the IPFIX key is deleted, the values of the ingress flow modification count field and the egress flow modification count field are incremented at the same time.
6. An apparatus for implementing bidirectional data synchronization of a flow table based on hardware, the apparatus comprising: the system comprises an inlet IPFIX processing module and an outlet IPFIX processing module, wherein the inlet IPFIX processing module generates a keyword index and is added with an inlet flow modification counting field according to a data message in advance, and the outlet IPFIX processing module is added with an outlet flow modification counting field;
the inlet IPFIX processing module is used for transmitting bus data formed by a data message, the key word index and the inlet flow modification count field to the outlet IPFIX processing module;
the egress IPFIX processing module is configured to extract the key index and the ingress flow modification count field from the bus data after receiving the data packet and the corresponding bus data, read egress data from an egress data storage of the egress IPFIX processing module using the key index, and extract the egress flow modification count field from the egress data;
and the outlet IPFIX processing module is further used for continuously judging whether the inlet flow modification count field is equal to the outlet flow modification count field, and if so, continuously processing the data message by IPFIX in the outlet direction.
7. The device according to claim 6, wherein the ingress IPFIX processing module extracts a corresponding field of a data packet as an IPFIX key, and calculates the key index using a hash algorithm on the IPFIX key.
8. The apparatus of claim 6, wherein the initial value of the ingress flow modification count field and the initial value of the egress flow modification count field are both set to 0.
9. The device according to claim 6, wherein if the egress IPFIX processing module determines that the egress IPFIX processing is not equal, the IPFIX processing in the subsequent egress direction of the data packet is ignored.
10. The apparatus of claim 7, wherein when data stream is aged or stream information corresponding to the IPFIX key is deleted, the ingress data processing module synchronously increments the ingress stream modification count field and the egress data processing module synchronously increments the value of the egress stream modification count field.
CN201911109547.4A 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware Active CN110865965B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911109547.4A CN110865965B (en) 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911109547.4A CN110865965B (en) 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware

Publications (2)

Publication Number Publication Date
CN110865965A true CN110865965A (en) 2020-03-06
CN110865965B CN110865965B (en) 2023-09-19

Family

ID=69654797

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911109547.4A Active CN110865965B (en) 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware

Country Status (1)

Country Link
CN (1) CN110865965B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112422434A (en) * 2020-11-10 2021-02-26 盛科网络（苏州）有限公司 IPFIX message processing method, application thereof and ASIC chip
CN112702232A (en) * 2020-12-21 2021-04-23 盛科网络（苏州）有限公司 IPFIX flow statistical method and device based on user-defined data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105337881A (en) * 2014-06-27 2016-02-17 华为技术有限公司 Data message processing method, service node and stream guiding point

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105337881A (en) * 2014-06-27 2016-02-17 华为技术有限公司 Data message processing method, service node and stream guiding point

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112422434A (en) * 2020-11-10 2021-02-26 盛科网络（苏州）有限公司 IPFIX message processing method, application thereof and ASIC chip
WO2022100581A1 (en) * 2020-11-10 2022-05-19 苏州盛科通信股份有限公司 Method for processing ipfix message, storage medium, network switching chip and asic chip
CN112702232A (en) * 2020-12-21 2021-04-23 盛科网络（苏州）有限公司 IPFIX flow statistical method and device based on user-defined data
CN112702232B (en) * 2020-12-21 2022-04-01 苏州盛科通信股份有限公司 IPFIX flow statistical method and device based on user-defined data

Also Published As

Publication number Publication date
CN110865965B (en) 2023-09-19

Similar Documents

Publication Publication Date Title
CN112468370B (en) High-speed network message monitoring and analyzing method and system supporting custom rules
JP4759389B2 (en) Packet communication device
DE69118454T2 (en) General encryption method for communication networks
US9356844B2 (en) Efficient application recognition in network traffic
CN105704091B (en) A kind of session analytic method and system based on SSH agreement
US9094219B2 (en) Network processor having multicasting protocol
WO2012138521A1 (en) Systems and methods for in-line removal of duplicate network packets
CN110708250A (en) Method for improving data forwarding performance, electronic equipment and storage medium
CN101426014B (en) Method and system for multicast source attack prevention
CN108810008B (en) Transmission control protocol flow filtering method, device, server and storage medium
CN110865965A (en) Method and device for realizing flow table bidirectional data synchronization based on hardware
CN111224882A (en) Message processing method and device and storage medium
Aljifri et al. IP traceback using header compression
CN115225734A (en) Message processing method and network equipment
CN107196879B (en) UDP message processing method and device and network forwarding device
CN113810337A (en) Method, device and storage medium for network message duplicate removal
CN112422434A (en) IPFIX message processing method, application thereof and ASIC chip
CN109195160B (en) Tamper-proof storage system of network equipment resource detection information and control method thereof
CN111224891A (en) Traffic application identification system and method based on dynamic learning triples
CN114157716B (en) Block chain-based data processing method and device and electronic equipment
JP2006236080A (en) Illegal access detection device and method
CN113839923B (en) Multi-node-oriented high-performance processing method
CN115622944A (en) TCP (Transmission control protocol) packet splicing method and system based on data streams under multiple parallelism degrees
CN110971565A (en) Source network load system vulnerability evaluation method and system based on malicious attack modeling
WO2016078212A1 (en) Packet processing method and device, and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant