CN110865965B - Method and device for realizing bidirectional data synchronization of flow table based on hardware - Google Patents

Method and device for realizing bidirectional data synchronization of flow table based on hardware Download PDF

Info

Publication number
CN110865965B
CN110865965B CN201911109547.4A CN201911109547A CN110865965B CN 110865965 B CN110865965 B CN 110865965B CN 201911109547 A CN201911109547 A CN 201911109547A CN 110865965 B CN110865965 B CN 110865965B
Authority
CN
China
Prior art keywords
data
ipfix
exit
processing module
count field
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911109547.4A
Other languages
Chinese (zh)
Other versions
CN110865965A (en
Inventor
姚飞
周伟
单哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Sheng Ke Science And Technology Co ltd
Original Assignee
Suzhou Sheng Ke Science And Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Sheng Ke Science And Technology Co ltd filed Critical Suzhou Sheng Ke Science And Technology Co ltd
Priority to CN201911109547.4A priority Critical patent/CN110865965B/en
Publication of CN110865965A publication Critical patent/CN110865965A/en
Application granted granted Critical
Publication of CN110865965B publication Critical patent/CN110865965B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a method and a device for realizing bidirectional data synchronization of a flow table based on hardware, wherein the method comprises the following steps: the inlet IPFIX processing module forms bus data by the data message, the key index and the inlet flow modification count field, and sends the bus data to the outlet IPFIX processing module, the outlet IPFIX processing module judges whether the inlet flow modification count field and the outlet flow modification count field are equal, if so, the data message is continuously processed by the IPFIX in the outlet direction. The invention realizes that the statistics data of the flow in the outlet direction and the key representing the flow in the inlet direction can be synchronized, and no mismatching is generated.

Description

Method and device for realizing bidirectional data synchronization of flow table based on hardware
Technical Field
The invention relates to a bidirectional data synchronization technology of a flow table, in particular to a method and a device for realizing bidirectional data synchronization of the flow table based on hardware.
Background
When the network has been laid, the network administrator cannot analyze the traffic state transmitted in the network by an effective means. IPFIX (IP Flow Information Export, IP data flow information output) is presented to solve this problem, where IPFIX passes through the traffic in the network by means of keys (keys), one of which represents a certain data flow in the network. The number, size, delay, destination, etc. of messages in the data streams are counted for each data stream pair in the ingress and egress directions, referred to herein as ingress and egress data.
In practical hardware implementation, there are two ways to process the data in two directions of the key and the data stream, and one scheme is to use two sets of keys and data, namely, an entry key+entry data and an exit key+exit data, but there is a problem that the entry key and the exit key cannot match the same data stream due to the editing behavior of the message in the process from the entry to the exit, so that an administrator cannot view the data condition of a certain data stream in the entry direction and the exit direction.
Another solution is to have keys only in the ingress direction, and ingress and egress data are in the ingress and egress directions of the chip, respectively. This approach solves the drawbacks of the above-described approach, so that an administrator can accurately see the information and status of a data stream from exit to exit. However, this scheme also causes a problem that the relationship between the egress data and the key cannot be associated because the key and the data exist in separate states, and the problem of synchronization of the egress data due to time delay and data stream aging during the transmission of the actual data.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method and a device for realizing bidirectional data synchronization of a flow table based on hardware.
In order to achieve the above purpose, the present invention proposes the following technical scheme: a method for realizing bidirectional data synchronization of a flow table based on hardware comprises the following steps:
s1, generating a keyword index and adding an inlet flow modification count field according to a data message in an inlet IPFIX processing module of a chip in advance, and adding an outlet flow modification count field in an outlet IPFIX processing module;
s2, the inlet IPFIX processing module forms bus data with the data message, the keyword index and the inlet flow modification count field and sends the bus data to the outlet IPFIX processing module;
s3, the exit IPFIX processing module receives the data message and the corresponding bus data, extracts the keyword index and the entry stream modification count field from the bus data, reads out exit data from the exit data storage by using the keyword index, and extracts the exit stream modification count field from the exit data;
s4, the exit IPFIX processing module continues to judge whether the entry flow modification count field and the exit flow modification count field are equal, if so, the data message continues to carry out IPFIX processing of the exit direction.
Preferably, the entry IPFIX processing module extracts the corresponding field of the data packet as an IPFIX key, and then calculates the key index by using a hash algorithm with respect to the IPFIX key.
Preferably, the initial values of the entry flow modification count field and the exit flow modification count field are each set to 0.
Preferably, in S4, if they are not equal, the IPFIX process of the subsequent exit direction of the data packet is ignored.
Preferably, when the data flow is aged or the flow information corresponding to the IPFIX key is deleted, the values of the entry flow modification count field and the exit flow modification count field are increased at the same time.
The invention also discloses another technical scheme: an apparatus for implementing bidirectional data synchronization of a flow table based on hardware, the apparatus comprising: an inlet IPFIX processing module and an outlet IPFIX processing module, wherein the inlet IPFIX processing module generates a keyword index and adds an inlet flow modification count field according to a data message in advance, and the outlet IPFIX processing module adds an outlet flow modification count field;
the inlet IPFIX processing module is used for forming bus data by the data message, the keyword index and the inlet flow modification count field and sending the bus data to the outlet IPFIX processing module;
the exit IPFIX processing module is used for extracting the keyword index and the entry stream modification count field from the bus data after receiving the data message and the corresponding bus data, reading out exit data in the exit data storage by using the keyword index, and extracting the exit stream modification count field from the exit data;
the exit IPFIX processing module is further configured to continuously determine whether the entry flow modification count field and the exit flow modification count field are equal, and if so, continuously perform IPFIX processing on the data packet in the exit direction.
Preferably, the entry IPFIX processing module extracts the corresponding field of the data packet as an IPFIX key, and then calculates the key index by using a hash algorithm with respect to the IPFIX key.
Preferably, if the output IPFIX processing module determines that the output IPFIX processing module is not equal, the IPFIX processing in the subsequent output direction of the data message is ignored.
Preferably, when the data flow is aged or the flow information corresponding to the IPFIX key is deleted, the ingress data processing module synchronously increases the value of the ingress flow modification count field and the egress data processing module synchronously increases the value of the egress flow modification count field.
The beneficial effects of the invention are as follows: according to the invention, the flow modification count field is inserted into the flow data, so that statistics data of the flow in the outlet direction and keys representing the flow in the inlet direction can be synchronized, and mismatching is not generated.
Drawings
FIGS. 1 and 2 are schematic flow diagrams of the method of the present invention;
fig. 3 is a schematic diagram of an ASIC chip of the present invention.
Detailed Description
The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
According to the method and the device for realizing bidirectional data synchronization of the flow table based on the hardware, the statistical data of the flow in the outlet direction and the key representing the flow in the inlet direction can be synchronized by inserting the flow modification count field into the flow data, and mismatching is avoided.
Referring to fig. 1 and fig. 2, the method for implementing bidirectional data synchronization of a flow table based on hardware disclosed in the present invention includes:
s1, generating a keyword index in an inlet IPFIX processing module of a chip in advance according to a data message, adding an inlet flow modification count field in an inlet data processing module, and adding an outlet flow modification count field in an outlet data processing module.
Specifically, in the direction of the chip (such as ASIC chip), the ingress IPFIX processing module extracts the corresponding field in the data packet as the flow key of IPFIX according to the flow rule. The flow key is compressed into key Index by a hash algorithm. The Key Index in the entry IPFIX processing module has three roles, one is Index stored as a Key in the Key memory of the entry IPFIX processing module; the second is index used as an entry direction flow data storage (i.e. entry data Memory); thirdly, the flow ID is taken to the exit IPFIX processing module in the exit direction.
Meanwhile, an ingress flow modification count field (ingresflowchangecount) is added in the ingress IPFIX processing module, and is used for recording the ingress flow modification count, and the initial value of the ingresflowchangecount is set to 0. And adding an exit flow modification count field (egresflowchangecount) in the exit IPFIX processing module for recording the exit flow modification count, the initial value of the egresflowchangecount being set to 0.
S2, the inlet IPFIX processing module transmits bus data composed of the data message, the keyword index and the inlet flow modification count field to the outlet IPFIX processing module.
Specifically, as shown in fig. 3, the bus (bus) data sequentially passes through an inlet data processing module, a message scheduling processing module and an outlet data processing module of the chip, and reaches an outlet IPFIX processing module.
S3, the exit IPFIX processing module receives the data message and the corresponding bus data, extracts the key index and the entry stream modification count field from the bus data, reads out exit data in the exit data storage by using the key index, and extracts the exit stream modification count field from the exit data.
Specifically, the exit IPFIX processing module receives the data message and the corresponding bus data, then extracts the key index and the entry flow modification count field from the bus data, uses the key index to store (memory) the exit data of the exit IPFIX processing module, indexes the exit data, and extracts the exit flow modification count field from the exit data.
S4, the exit IPFIX processing module continues to judge whether the entry flow modification count field and the exit flow modification count field are equal, if so, the data message continues to carry out IPFIX processing of the exit direction.
Specifically, the exit IPFIX processing module judges whether the values of the entry flow modification count field of the entry and the exit flow modification count field of the exit are equal, if so, the current exit data and the IPFIX key of the current entry IPFIX processing module are considered to be matched, the IPFIX processing of the exit direction is continued, otherwise, if not, the current exit data and the IPFIX key of the current entry IPFIX processing module are not considered to be the same flow, and the subsequent processing of the exit IPFIX is ignored. It should be clear here that inequality indicates that the key of the Flow has been deleted, specifically that a certain message in the Flow still exists when the ingress IPFIX is processed, and that the key has been deleted during the time before being transmitted to the processing engine of the egress IPFIX. The deletion causes are various, and may be aging (aging) or active deletion by the CPU or other causes.
In addition, when the data stream is aged or the stream information corresponding to the IPFIX key is deleted, the values of the entry stream modification count field and the exit stream modification count field are increased at the same time so as to keep synchronization. It should be noted that, deleting the flow information usually has two places where the operation of deleting the key occurs, one is natural aging of the flow, which is processed by an aging engine (aging engine) of the ipfix, and the other is that the CPU issues an instruction to perform the deletion processing. It is necessary to be able to access the flow-counting store (memory) in both the ingress (ingress) ipfix and egress (egress) ipfix and update the flow modification count fields in both memories at the same time in the processing engine of the processing engines and the cpu of the ipfix.
In addition, the values of the entry flow modification count field and the exit flow modification count field do not need to be infinitely incremented, and the infinite increment needs to be stored with a large number, and only a loop is actually needed. For example 0,1,2,3,4,5,6,0,1,2,3,4,5,6.
The invention discloses a bidirectional data synchronization device based on a hardware realization flow table, which comprises: an inlet IPFIX processing module and an outlet IPFIX processing module, wherein the inlet IPFIX processing module generates a keyword index and adds an inlet flow modification count field according to a data message in advance, and the outlet IPFIX processing module adds an outlet flow modification count field;
the inlet IPFIX processing module is used for forming bus data by the data message, the keyword index and the inlet flow modification count field and sending the bus data to the outlet IPFIX processing module;
the exit IPFIX processing module is used for extracting the keyword index and the entry stream modification count field from the bus data after receiving the data message and the corresponding bus data, reading out exit data in the exit data storage by using the keyword index, and extracting the exit stream modification count field from the exit data;
the exit IPFIX processing module is further configured to continuously determine whether the entry flow modification count field and the exit flow modification count field are equal, and if so, continuously perform IPFIX processing on the data packet in the exit direction.
The implementation principles of the inlet IPFIX processing module and the outlet IPFIX processing module can be referred to the description of the steps S1 to S3, and the description thereof will not be repeated here.
While the foregoing has been disclosed in the specification and drawings, it will be apparent to those skilled in the art that various substitutions and modifications may be made without departing from the spirit of the invention, and it is intended that the scope of the invention be limited not by the specific embodiments disclosed, but by the appended claims.

Claims (6)

1. A method for implementing bidirectional data synchronization of a flow table based on hardware, the method comprising:
s1, generating a keyword index and adding an inlet flow modification count field according to a data message in an inlet IPFIX processing module of a chip in advance, and adding an outlet flow modification count field in an outlet IPFIX processing module; the key index is calculated by using an IPFIX key as an IPFIX key through extracting corresponding fields of a data message by an inlet IPFIX processing module and adopting a hash algorithm;
s2, the inlet IPFIX processing module forms bus data with the data message, the keyword index and the inlet flow modification count field and sends the bus data to the outlet IPFIX processing module;
s3, the exit IPFIX processing module receives the data message and the corresponding bus data, extracts the keyword index and the entry stream modification count field from the bus data, reads out exit data from the exit data storage by using the keyword index, and extracts the exit stream modification count field from the exit data;
s4, the exit IPFIX processing module continues to judge whether the entry flow modification count field and the exit flow modification count field are equal, if so, the data message continues to carry out IPFIX processing of the exit direction; if not, the IPFIX processing of the subsequent outlet direction of the data message is ignored.
2. The method of claim 1, wherein the initial values of the entry flow modification count field and the exit flow modification count field are each set to 0.
3. The method for implementing bidirectional data synchronization of a flow table based on hardware according to claim 1, wherein when the data flow is aged or the flow information corresponding to the IPFIX key is deleted, the values of the entry flow modification count field and the exit flow modification count field are incremented at the same time.
4. An apparatus for implementing bidirectional data synchronization of a flow table based on hardware, the apparatus comprising: an inlet IPFIX processing module and an outlet IPFIX processing module, wherein the inlet IPFIX processing module generates a keyword index and adds an inlet flow modification count field according to a data message in advance, and the outlet IPFIX processing module adds an outlet flow modification count field;
the entry IPFIX processing module is used for forming bus data by the data message, the keyword index and the entry stream modification count field and sending the bus data to the exit IPFIX processing module, wherein the keyword index is obtained by extracting corresponding fields of the data message as IPFIX keywords by the entry IPFIX processing module, and then the keyword index is calculated by adopting a hash algorithm;
the exit IPFIX processing module is used for extracting the keyword index and the entry stream modification count field from the bus data after receiving the data message and the corresponding bus data, reading out exit data in the exit data storage by using the keyword index, and extracting the exit stream modification count field from the exit data;
the exit IPFIX processing module is further configured to continuously determine whether the entry flow modification count field and the exit flow modification count field are equal, if so, continuously perform IPFIX processing on the data packet in the exit direction, and if not, ignore IPFIX processing on the data packet in the subsequent exit direction.
5. The apparatus for bidirectional data synchronization of a flow table based on hardware as recited in claim 4 wherein the initial values of the entry flow modification count field and the exit flow modification count field are each set to 0.
6. The apparatus for bidirectional data synchronization of a flow table based on hardware as set forth in claim 4, wherein said ingress data processing module synchronously increments the value of said ingress flow modification count field and said egress data processing module when aging of the data flow occurs or when deleting flow information corresponding to said IPFIX key.
CN201911109547.4A 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware Active CN110865965B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911109547.4A CN110865965B (en) 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911109547.4A CN110865965B (en) 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware

Publications (2)

Publication Number Publication Date
CN110865965A CN110865965A (en) 2020-03-06
CN110865965B true CN110865965B (en) 2023-09-19

Family

ID=69654797

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911109547.4A Active CN110865965B (en) 2019-11-13 2019-11-13 Method and device for realizing bidirectional data synchronization of flow table based on hardware

Country Status (1)

Country Link
CN (1) CN110865965B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112422434A (en) * 2020-11-10 2021-02-26 盛科网络（苏州）有限公司 IPFIX message processing method, application thereof and ASIC chip
CN112702232B (en) * 2020-12-21 2022-04-01 苏州盛科通信股份有限公司 IPFIX flow statistical method and device based on user-defined data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105337881A (en) * 2014-06-27 2016-02-17 华为技术有限公司 Data message processing method, service node and stream guiding point

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105337881A (en) * 2014-06-27 2016-02-17 华为技术有限公司 Data message processing method, service node and stream guiding point

Also Published As

Publication number Publication date
CN110865965A (en) 2020-03-06

Similar Documents

Publication Publication Date Title
CN110197234B (en) Encrypted flow classification method based on dual-channel convolutional neural network
Miner et al. Graph-based authentication of digital streams
CN110865965B (en) Method and device for realizing bidirectional data synchronization of flow table based on hardware
CN110401642A (en) A kind of acquisition of industry control flow and protocol analysis method
Wang et al. Dictionary based secure provenance compression for wireless sensor networks
WO2012138521A1 (en) Systems and methods for in-line removal of duplicate network packets
CN105357137B (en) Message filtering method and the FPGA being applicable in, intelligent substation
CN104661042A (en) Method, device and system for transmitting transport stream
CN104486243A (en) Data transmission method, equipment and system
CN113507483B (en) Instant messaging method, device, server and storage medium
CN103475657B (en) The treating method and apparatus of anti-SYN extensive aggression
CN102714652A (en) Supervision of a communication session comprising several flows over a data network
CN113645233A (en) Wind control intelligent decision method and device for flow data, electronic equipment and medium
Mishra et al. TCP Flow Control in Lossy Networks: Analysis and Enhancement.
CN115361455B (en) Data transmission storage method and device and computer equipment
CN106850153B (en) Data retransmission method and system
CN113839923B (en) Multi-node-oriented high-performance processing method
JPH05191454A (en) Data transfer system
Yasuda et al. A probabilistic interest packet aggregation for content-centric networking
Christensen et al. Reduction of self-similarity by application-level traffic shaping
Fraczek et al. Steg Blocks: Ensuring perfect undetectability of network steganography
CN112291350A (en) File transmission method, system, device and medium
WO2021047612A1 (en) Packet processing method, device, and computer storage medium
CN114826602B (en) Security-aware time-sensitive network scheduling method, system, device and medium
CN116126910A (en) Method, system, equipment and medium for rolling and aggregating mass session data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant