CN111988271A - Communication flow processing method and device - Google Patents
Communication flow processing method and device Download PDFInfo
- Publication number
- CN111988271A CN111988271A CN202010621518.2A CN202010621518A CN111988271A CN 111988271 A CN111988271 A CN 111988271A CN 202010621518 A CN202010621518 A CN 202010621518A CN 111988271 A CN111988271 A CN 111988271A
- Authority
- CN
- China
- Prior art keywords
- flow
- message
- stream
- identifier
- ticket
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention discloses a communication flow processing method and a communication flow processing device. The method comprises the following steps: when the message is analyzed, generating a flow identification according to the flow data of the message, and caching the flow data related to the service in the message; then merging the cached streaming data of the plurality of messages into a streaming message and sending the streaming message to a ticket synthesis module when a certain condition is met; after receiving the flow message, the ticket synthesizing module obtains the flow identifier and the flow information required by synthesizing the ticket, stores the flow information into an array with the flow identifier as a subscript, and then accesses the flow information corresponding to the flow by positioning the flow identifier as the subscript to the array unit. The information access method of the following calibration digit group unit has higher speed and higher searching efficiency than the information access method of obtaining values through keys of a key value mapping table (Map), so the processing speed and throughput of the stream can be greatly improved through the communication stream processing method, the time for synthesizing the call ticket is correspondingly shortened, and the processing capacity of the call ticket is improved.
Description
Technical Field
The present invention relates to the field of mobile communications technologies, and in particular, to a communication flow processing method and apparatus.
Background
In recent years, with the increasing development of networks and communication technologies, the amount of communication data increases explosively, and under the circumstance, how to realize the acquisition, analysis and ticket synthesis of ultra-large amount of communication data becomes a difficult point gradually.
In the existing solutions, a flow information table is established by using a data structure of a key value mapping table (Map), and a flow information table entry corresponding to a flow is accessed by using a key of the Map to obtain a value. In the case of a large increase in the number of communication flows, the Map search efficiency is drastically reduced, and more servers have to be added to deal with the Map search efficiency, which greatly increases the networking cost and the maintenance cost.
Therefore, how to improve the processing speed and throughput of communication flow as much as possible without increasing more servers is an urgent technical problem to be solved.
Disclosure of Invention
In view of the above problems, embodiments of the present invention provide a communication flow processing method and apparatus.
According to a first aspect of an embodiment of the present invention, a communication flow processing method is applied to a packet parsing module, and the method includes: receiving a first message and analyzing the first message to obtain stream data; acquiring a stream identifier corresponding to stream data, wherein the stream identifier is a digital sequence capable of uniquely identifying the stream; adding stream data into a stream data set corresponding to the stream identifier according to the stream identifier, wherein the stream data set is a set of all stream data corresponding to the stream identifier obtained within a period of time; and detecting whether the flow data set meets a first condition, if so, generating a second message with the flow identification according to the flow identification and the flow data set, and sending the second message to a ticket synthesis module.
According to an embodiment of the present invention, the first condition includes: the number of pieces of stream data in the stream data set is greater than the number-of-pieces threshold or the stream data in the stream data set satisfies the aging condition.
According to an embodiment of the present invention, generating the second packet with the flow identifier according to the flow identifier and the flow data set includes: setting aging information according to whether the stream data in the stream data set meet the aging condition; and generating a second message with the flow identification according to the flow identification, the flow data set and the aging information, so that the second message carries the aging information.
According to an embodiment of the present invention, the first message is received through an optical port of a programmable hardware accelerator card, the method is executed in a processor of the programmable hardware accelerator card, and a stream data set is stored in a double data rate synchronous dynamic random access memory (DDR SDRAM) of the programmable hardware accelerator card.
According to a second aspect of the embodiments of the present invention, there is provided a communication flow processing method, which is applied to a ticket synthesizing module, and includes: receiving a second message sent by the message analysis module and acquiring a flow identifier and a flow data set corresponding to the flow identifier from the second message; acquiring flow information required by the synthesis of the call ticket according to the flow data set; acquiring a flow information table for synthesizing the call ticket, wherein the flow information table stores flow information in an array form, and each flow information table entry is stored in an array unit positioned by a subscript; and positioning the array unit by taking the stream identifier as an array subscript and adding the stream information into the stream information table entry stored in the array unit.
According to an embodiment of the present invention, the method further includes: acquiring a flow information table and positioning an array unit by taking the flow identifier as an array subscript; acquiring a flow information table entry from an array unit; and synthesizing the ticket according to the flow information list item.
According to an embodiment of the present invention, the second packet carries aging information, and accordingly, the method further includes: and judging whether to age the flow information table entry corresponding to the flow identifier according to the flow aging information, and if so, aging the flow information table entry corresponding to the flow identifier.
According to an embodiment of the present invention, the ticket synthesizing module has at least two ticket synthesizing sub-modules, and after receiving the second message sent by the message analyzing module and acquiring the flow identifier and the flow data set corresponding to the flow identifier from the second message, the method further includes: determining a first call ticket synthesizing submodule for processing the stream data set from at least two call ticket synthesizing submodules according to the stream data in the stream data set and the number of the call ticket synthesizing submodules, so that all the stream data corresponding to the stream identifier can be sent to the same call ticket synthesizing submodule and the load balance of the call ticket synthesizing submodule is ensured; and sending the flow identification and the flow data set to a first call ticket synthesizing submodule, and executing subsequent operation by the first call ticket synthesizing submodule.
According to a third aspect of the embodiments of the present invention, there is provided a communication flow processing apparatus, applied to a packet parsing module, the apparatus including: the first message analysis unit is used for receiving the first message and analyzing the first message to obtain stream data; a first flow identifier acquiring unit, configured to acquire a flow identifier corresponding to flow data, where the flow identifier is a digital sequence that can uniquely identify a flow; the stream data merging unit is used for adding the stream data into a stream data set corresponding to the stream identifier according to the stream identifier, wherein the stream data set is a set of all stream data corresponding to the stream identifier obtained within a period of time; a first condition detecting unit for detecting whether the stream data set satisfies a first condition; and the second message sending unit is used for generating a second message with the flow identification according to the flow identification and the flow data set and sending the second message to the ticket synthesis module.
According to a fourth aspect of the embodiments of the present invention, there is provided a communication flow processing apparatus, applied to a ticket synthesizing module, the apparatus including: the second message analysis unit is used for receiving a second message sent by the message analysis module and acquiring the flow identifier and the flow data set corresponding to the flow identifier from the second message; the flow information acquisition unit is used for acquiring flow information required by the synthesis ticket according to the flow data set; the system comprises a flow information table acquisition unit, a flow information table processing unit and a call ticket processing unit, wherein the flow information table is used for acquiring a flow information table for synthesizing call tickets, the flow information table stores flow information in an array form, and each flow information table entry is stored in an array unit positioned through subscripts; and the flow information table entry updating unit is used for positioning the array unit by taking the flow identifier as the array subscript and adding the flow information into the flow information table entry stored in the array unit.
The embodiment of the invention provides a communication flow processing method and device. The method comprises the following steps: when the message is analyzed, generating a flow identification according to the flow data of the message, and caching the flow data related to the service in the message; then merging the cached streaming data of the plurality of messages into a streaming message and sending the streaming message to a ticket synthesis module when a certain condition is met; after receiving the flow message, the ticket synthesizing module obtains the flow identifier and the flow information required by synthesizing the ticket, stores the flow information into an array with the flow identifier as a subscript, and then accesses the flow information corresponding to the flow by positioning the flow identifier as the subscript to the array unit. The information access method of the following calibration digit group unit has higher speed and higher searching efficiency than the information access method of obtaining values through keys of a key value mapping table (Map), so the processing speed and throughput of the stream can be greatly improved through the communication stream processing method, the time for synthesizing the call ticket is correspondingly shortened, and the processing capacity of the call ticket is improved.
It is to be understood that the teachings of the present invention need not achieve all of the above-described benefits, but rather that specific embodiments may achieve specific technical results, and that other embodiments of the present invention may achieve benefits not mentioned above.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
in the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
Fig. 1 is a schematic view of an application scenario of a communication flow processing method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of the implementation of the communication flow processing method at the message parsing module end according to the embodiment of the present invention;
fig. 3 is a schematic view of an implementation flow of the communication flow processing method at the ticket synthesizing module end according to the embodiment of the present invention;
fig. 4 is a schematic diagram of a component structure of a device of a communication flow processing device at a message parsing module according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a device of a communication flow processing device at a call ticket synthesizing module end according to an embodiment of the present invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
Fig. 1 shows an application scenario of the communication flow processing method according to the embodiment of the present invention. As shown in fig. 1, this is an application scenario in which a ticket synthesis system generates a ticket according to an original message. The call ticket synthesizing system mainly comprises a message analyzing module 10 and a call ticket synthesizing module 20, wherein the message analyzing module 10 is realized by codes executed on a programmable hardware accelerator card (such as an FPGA card), and the call ticket synthesizing module 20 is realized by software running on a server operating system.
It should be noted that the arrows in fig. 1 represent the data flow and the direction of data flow transmission, and each block in fig. 1 represents a functional sub-module, a functional unit, a memory or a flow table. The functional sub-modules and functional units herein generally correspond to a processing program for implementing the functions, and do not necessarily correspond to a hardware device.
The following describes how to use the trial communication flow processing method of the present invention to complete the main process from receiving the original message to synthesizing the call ticket with reference to fig. 1: firstly, an original message enters the message analysis module 10 through the optical port 101 on the programmable hardware accelerator card, key information required by a synthesis ticket, namely stream data, is obtained through the message analysis processing of the message analysis submodule 102, and the stream data is transmitted to the stream table management submodule 103. After receiving the stream data, the flow table management sub-module 103 searches for a flow table entry corresponding to the packet in the flow table, and if the flow table entry is found, obtains a flow identifier corresponding to the flow identifier from the flow table, where the flow identifier is a digital sequence that can uniquely identify the flow, and if the flow identifier is not found, generates the flow identifier according to the stream data, and creates a new flow table entry to store the flow identifier in the flow table entry. Flow table management submodule 103 then also caches the flow data in memory 105 connected to the programmable hardware accelerator card. When the number of the stream data to be cached reaches a preset threshold value or the data stream reaches an aging condition, the stream table reporting sub-module 104 merges the plurality of stream data to generate a stream message, and sends the stream message to the stream message difference and distribution sub-module 201 of the ticket synthesizing module 20.
In the ticket synthesizing module 20, there are a plurality of ticket synthesizing sub-modules, for example, a ticket synthesizing sub-module 202 and a ticket synthesizing sub-module 203, and the flow message splitting and distributing sub-module 201 determines which ticket synthesizing sub-module processes the flow message according to the flow identifier and the number of the ticket synthesizing sub-modules, and sends the flow message to the corresponding ticket synthesizing sub-module after determining. If the determined call ticket synthesizing submodule is the call ticket synthesizing submodule 202, the flow information table management unit 2021 of the call ticket synthesizing submodule 202 will extract corresponding flow information according to the flow data in the flow message and store the flow information into the flow information table entry in the flow information table 2022, and when it needs to be described, the flow information table 2022 adopts an array as a data storage structure, and the table entry is stored in an array unit with the flow identifier as a subscript. Then, the flow information table management unit 2021 may obtain corresponding information from the flow information table in a manner of positioning the array unit with the flow identifier as the subscript, and send the corresponding information to the ticket generating and sending unit 2023 for ticket synthesis, and send the synthesized ticket to the upper application module.
It should be noted that the application scenarios described above are only exemplary and are not a limitation on the application scenarios of the embodiments of the present invention, and an implementer may try the present invention in any applicable scenarios.
According to a first aspect of the embodiments of the present invention, a communication flow processing method is applied to a packet parsing module, and as shown in fig. 2, the method includes: operation 210, receiving the first message and analyzing the first message to obtain stream data; operation 220, obtaining a stream identifier corresponding to the stream data, where the stream identifier is a number sequence that can uniquely identify the stream; operation 230, adding the stream data to a stream data set corresponding to the stream identifier according to the stream identifier, where the stream data set is a set of all stream data corresponding to the stream identifier obtained within a period of time; operation 240, detecting whether the flow data set meets the first condition, if so, generating a second message with the flow identifier according to the flow identifier and the flow data set, and sending the second message to the ticket synthesizing module.
In operation 210, the first message is an original message received from a message receiving port, such as a network port of a server network card or an optical port of a programmable hardware accelerator card, and the like, such as an IP data message, a TCP data message, a UDP data message, and the like.
The original message usually includes two parts, i.e., data and message, wherein the data is data that needs to be transmitted by a higher layer, and the header is control information added for correctly transmitting the higher layer data. Taking the IP datagram as an example, the header includes information such as version, header length, service type, total length, identifier, flag, segment offset, lifetime, total number of times of passing through the router, protocol type, header, source IP address, source port, destination IP address, and destination port.
The message is analyzed to extract stream data from the master-slave header and the data. The stream data mainly comprises key information such as a source address, a source port, a destination address, a destination port, a protocol type, a total length, partial message data extracted through a regular expression and the like of the message. The key information can be used for distinguishing communication flows, and related information required by a synthesized call ticket, such as an application name, user information and the like, can be acquired from the communication flows through a Deep Packet Inspection (DPI) technology.
In operation 220, a flow identifier corresponding to the flow data may be obtained, generally by: acquiring a flow table; detecting whether a flow table entry corresponding to the flow data exists in the flow table, if so, acquiring a flow identifier corresponding to the flow data from the flow table entry, if not, generating a flow identifier corresponding to the flow data according to the flow data, creating a flow table entry corresponding to the flow data, storing the flow identifier in the flow table entry, and adding the flow table entry to the flow table.
The flow identifier may be generated by using five-tuple information of source address, source port, destination address, destination port and protocol type, or by other information that can uniquely identify the flow, for example, seven-tuple information consisting of interface index, source address, destination address, source port, destination port, protocol number and ToS. The embodiment of the present invention does not limit the specific implementation of how to generate the identifier according to the key information, such as the quintuple or the heptatuple, and may use an algorithm, such as Hash or MD5, to perform an operation on the quintuple or the heptatuple information to generate a digital sequence, and use the digital sequence as the stream identifier of the communication stream, or may use any other suitable method.
Generally, a communication flow is divided into a plurality of packets for transmission, so that a plurality of packets corresponding to a certain communication flow are received within a period of time, and accordingly, a plurality of pieces of stream data corresponding to the same flow identifier are obtained within a period of time. If the stream data is packed into a new message and sent to the ticket synthesizing module after the stream data is obtained every time, the communication traffic between the message analyzing module and the ticket synthesizing module is increased. In addition, some key information may be acquired through multiple messages, for example, a message gap, and thus, multiple pieces of stream data corresponding to the mobile phone stream identifier may be acquired.
Therefore, in operation 230, the flow data obtained after the first packet is analyzed is not immediately sent to the ticket synthesizing module, but is added to the flow data set corresponding to the flow identifier. In specific implementation, the stream data may be cached in a local memory, for example, a dynamic memory (SRAM) of the server, a static memory (RRAM), or a double data rate synchronous dynamic random access memory (DDR SDRAM) connected to the programmable hardware accelerator card through the EM. After a plurality of pieces of stream data are cached, the stream data are merged to generate a stream message, and the stream message is sent to the ticket synthesizing module.
In operation 240, it is detected whether the set of stream data satisfies a first condition that is a trigger condition to merge stream data and generate a stream packet. And the second packet refers to the merged stream packet.
The flow data carried in the second message is some key information extracted from the original message rather than full information, so that the data volume to be processed by the call ticket synthesizing module is greatly reduced, and the processing capacity of the call ticket synthesizing module is greatly improved. In addition, the stream data in the second message is the stream data combined with a plurality of stream data, and the communication traffic between the message analysis module and the ticket synthesis module is greatly reduced.
According to an embodiment of the present invention, the first condition includes: the number of pieces of stream data in the stream data set is greater than the number-of-pieces threshold or the stream data in the stream data set satisfies the aging condition.
In this embodiment, the operation of merging the stream data and generating the stream packet may be triggered in any one of the following two cases, where the number of pieces of stream data in the stream data set is greater than the number threshold, and the stream data in the stream data set satisfies the aging condition.
The number threshold value defines a value representing the merging of several pieces of stream data into a second message, which is initially set based on empirical values, and which may be adjusted later based on implementation goals and implementation results.
The stream data meeting the aging condition usually indicates that this transmission of the stream data has ended or that there is no new message for a while. The embodiment of the present invention does not limit how to detect whether the aging condition is satisfied, and may be determined by a timing task periodically detecting whether the current system time exceeds the expiration time of the stream data, or may be determined by identifying a stream receiving flag, or may be implemented by any other applicable method.
According to an embodiment of the present invention, generating the second packet with the flow identifier according to the flow identifier and the flow data set includes: setting aging information according to whether the stream data in the stream data set meet the aging condition; and generating a second message with the flow identification according to the flow identification, the flow data set and the aging information, so that the second message carries the aging information.
Generally, the call ticket synthesizing module also needs to perform aging processing on the relevant information of the communication flow, for example, aging a flow information table entry corresponding to the communication flow in the flow information table. In this embodiment, the information of whether the aging process is required is transmitted to the ticket synthesizing module in a manner of carrying the aging information in the second message. Therefore, the call ticket synthesizing module does not need to repeatedly detect whether the stream data of the communication stream is overdue or not, the workload of the call ticket synthesizing module is saved to a certain extent, and the processing capacity of the call ticket synthesizing module is indirectly improved.
According to an embodiment of the present invention, the first message is received through an optical port of a programmable hardware accelerator card, the method is executed in a processor of the programmable hardware accelerator card, and a stream data set is stored in a double-rate synchronous dynamic random access memory of the programmable hardware accelerator card.
The communication flow processing method applied to the message analysis module can be applied to the message analysis module on the server and can also be applied to the message analysis module on a programmable hardware accelerator card connected with the server. However, compared with a message analysis module implemented by software on a server, the message analysis module can be implemented on a programmable hardware accelerator card by programming a programmable chip, and the message analysis module implemented in this way does not need to analyze or transmit and process instructions through an operating system, and has higher message deep detection capability. In addition, the optical port on the programmable hardware accelerator card is a network port connected with the optical fiber transmission channel, and the transmission speed is very high. And if the stream data set is stored in the double-rate synchronous dynamic random access memory of the programmable hardware accelerator card, the data storage speed can be further improved. Therefore, in this embodiment, the communication flow method is executed by using the optical port on the programmable hardware accelerator card connected to the server to receive the first packet and using the packet analysis module implemented on the programmable hardware accelerator card, thereby further improving the packet analysis speed and throughput.
According to a second aspect of the embodiments of the present invention, there is provided a communication flow processing method, which is applied to a ticket synthesizing module, as shown in fig. 3, the method includes: operation 310, receiving a second message sent by the message parsing module, and obtaining a flow identifier and a flow data set corresponding to the flow identifier from the second message; operation 320, acquiring flow information required by the synthesized ticket according to the flow data set; operation 330, acquiring a flow information table for synthesizing the ticket, where the flow information table stores the flow information in an array form, and each flow information table entry is stored in an array unit located by the subscript; operation 340 locates the array unit with the stream identification as the array index and adds the stream information to the stream information entry stored in the array unit.
In operation 310, after receiving the second message, the ticket synthesizing module parses the second message to obtain the flow identifier carried by the second message and the flow data set corresponding to the flow identifier. Therefore, the transmission of the flow identification and the flow data set from the message analysis module to the ticket synthesis module is realized.
In operation 320, the flow information required for synthesizing the ticket mainly refers to a part of service information in the original message and a part of information in communication signaling sent between communication devices in the communication process, for example, an application name, a traffic size, a user access address, and the like of the originating communication flow. The stream data obtained from the message parsing process is usually only a small part of the stream information required by the synthesized ticket, but the small part of the information is a more critical part of the information, and more other stream information associated with the small part of the information can be obtained through the critical information in the stream data, for example, through the collection and association of firewall data, signaling related to the communication stream can be obtained, and stream information such as a user access address can be obtained from the signaling.
The flow information table is the flow information needed to store the synthesized ticket in operation 330. In most existing schemes, the flow information table stores the flow information by using a Map data structure, and in the embodiment of the present invention, the flow information table stores the flow information by using a data structure of an array, and locates an array unit by using a flow identifier as an array subscript, and accesses a flow information entry stored in the array unit.
In operation 340, the method for processing a communication flow according to this embodiment of the present invention adds the flow information to the flow information entry stored in the array unit by using the flow identifier as the array subscript to locate the array unit, so as to implement the correspondence between the flow identifier and the flow information entry. Then, a series of operations such as reading, updating, searching and aging can be performed on the entries in the flow information table in a manner that the flow identifier is used as the array subscript to locate the array unit.
Because the searching efficiency of the array unit positioned by the subscript is far higher than that of the value obtained by using the Map key, the operation speed of the flow information table can be greatly improved by the communication flow processing method, and the synthesis capacity of the call ticket is greatly improved.
According to an embodiment of the present invention, the method further includes: acquiring a flow information table and positioning an array unit by taking the flow identifier as an array subscript; acquiring a flow information table entry from an array unit; and synthesizing the ticket according to the flow information list item.
As described above, not only when storing the flow information, the flow identifier can be used as the array subscript to position the array unit to obtain the flow information table entry, but also when synthesizing the ticket, the flow information table entry can be obtained in this way, thereby greatly improving the processing speed of the synthesized ticket and the throughput of the synthesized ticket module.
According to an embodiment of the present invention, the second packet carries aging information, and accordingly, the method further includes: and judging whether to age the flow information table entry corresponding to the flow identifier according to the flow aging information, and if so, aging the flow information table entry corresponding to the flow identifier.
Because the second message carries the aging information of whether the communication flow needs to be aged or not, the call ticket synthesis module does not need to repeatedly detect whether the flow data of the communication flow is overdue or not, so that the workload of the call ticket synthesis module is saved to a certain extent, and the processing capacity of the call ticket synthesis module is indirectly improved.
According to an embodiment of the present invention, the ticket synthesizing module has at least two ticket synthesizing sub-modules, and after receiving the second message sent by the message analyzing module and acquiring the flow identifier and the flow data set corresponding to the flow identifier from the second message, the method further includes: determining a first call ticket synthesizing submodule for processing the stream data set from at least two call ticket synthesizing submodules according to the stream data in the stream data set and the number of the call ticket synthesizing submodules, so that all the stream data corresponding to the stream identifier can be sent to the same call ticket synthesizing submodule and the load balance of the call ticket synthesizing submodule is ensured; and sending the flow identification and the flow data set to a first call ticket synthesizing submodule, and executing subsequent operation by the first call ticket synthesizing submodule.
In this embodiment, the following method may be adopted to determine the first ticket synthesizing submodule: calculating hash values of the inner-layer source IP address and the inner-layer destination IP address of the stream data to obtain a first key value; acquiring the number of the call ticket synthesis sub-modules; obtaining a second key value by using the first key value to perform modular operation on the number of the ticket synthesis sub-modules; and determining the call ticket synthesis submodule corresponding to the second key value as a first call ticket synthesis submodule.
According to a third aspect of the embodiments of the present invention, there is provided a communication flow processing apparatus, which is applied to a packet parsing module, and as shown in fig. 4, the apparatus 40 includes: a first message parsing unit 401, configured to receive a first message and parse the first message to obtain stream data; a first flow identifier acquiring unit 402, configured to acquire a flow identifier corresponding to flow data, where the flow identifier is a digital sequence that can uniquely identify a flow; a stream data merging unit 403, configured to add, according to the stream identifier, stream data to a stream data set corresponding to the stream identifier, where the stream data set is a set of all stream data corresponding to the stream identifier obtained within a period of time; a first condition detection unit 404 for detecting whether the stream data set satisfies a first condition; and a second message sending unit 405, configured to generate a second message with a flow identifier according to the flow identifier and the flow data set, and send the second message to the ticket synthesizing module.
According to an embodiment of the present invention, the first condition detecting unit 404 is specifically configured to detect whether the number of pieces of stream data in the stream data set is greater than a number threshold or whether the stream data in the stream data set satisfies an aging condition.
According to an embodiment of the present invention, the second packet generating unit includes 405: an aging information setting subunit configured to set aging information according to whether the stream data in the stream data set satisfies an aging condition; and the second message production subunit is used for generating a second message with the flow identifier according to the flow identifier, the flow data set and the aging information, so that the second message carries the aging information.
According to a fourth aspect of the embodiments of the present invention, there is provided a communication flow processing apparatus, applied to a ticket synthesizing module, as shown in fig. 5, the apparatus 50 includes: a second packet parsing unit 501, configured to receive a second packet sent by the packet parsing module and obtain a flow identifier and a flow data set corresponding to the flow identifier from the second packet; a flow information obtaining unit 502, configured to obtain flow information required for synthesizing a ticket according to a flow data set; a flow information table obtaining unit 503, configured to obtain a flow information table for synthesizing a ticket, where the flow information table stores flow information in an array form, and each flow information table entry is stored in an array unit located by a subscript; and a flow information entry updating unit 504, configured to locate an array unit with the flow identifier as an array subscript and add the flow information to the flow information entry stored in the array unit.
According to an embodiment of the present invention, the apparatus 50 further includes: a flow information table acquisition unit for acquiring a flow information table; a flow information table entry obtaining unit, configured to locate an array unit using the flow identifier as an array subscript and obtain a flow information table entry from the array unit; and the ticket generating unit is used for synthesizing the ticket according to the flow information list item.
According to an embodiment of the present invention, the apparatus 50 further includes: and the aging processing unit is used for judging whether to perform aging processing on the flow information table entry corresponding to the flow identifier according to the flow aging information, and if so, performing aging processing on the flow information table entry corresponding to the flow identifier.
According to an embodiment of the present invention, the apparatus 50 further includes: a first ticket synthesizing submodule determining unit, configured to determine, from the at least two ticket synthesizing submodules, a first ticket synthesizing submodule that processes the stream data set according to the stream data in the stream data set and the number of the ticket synthesizing submodules, so that all the stream data corresponding to the stream identifier can be sent to the same ticket synthesizing submodule and load balance of the ticket synthesizing submodules is ensured; and the stream data set sending unit is used for sending the stream identifier and the stream data set to the first ticket synthesizing submodule, and the first ticket synthesizing submodule executes subsequent operations.
Here, it should be noted that: the above description of the embodiment of the communication flow processing apparatus is similar to that of the foregoing method embodiment, and has similar beneficial effects to the foregoing method embodiment, and therefore, the description thereof is omitted. For technical details that have not been disclosed in the description of the embodiment of the communication flow processing apparatus of the present invention, please refer to the description of the foregoing method embodiment of the present invention for understanding, and therefore, for brevity, will not be described again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of a unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another device, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media capable of storing program codes, such as a removable storage medium, a Read Only Memory (ROM), a magnetic disk, and an optical disk.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods of the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage medium, a ROM, a magnetic disk, an optical disk, or the like, which can store the program code.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A communication flow processing method is applied to a message analysis module, and comprises the following steps:
receiving a first message and analyzing the first message to obtain stream data;
acquiring a stream identifier corresponding to the stream data, wherein the stream identifier is a digital sequence capable of uniquely identifying the stream;
adding the stream data into a stream data set corresponding to the stream identifier according to the stream identifier, wherein the stream data set is a set of all stream data corresponding to the stream identifier obtained within a period of time;
and detecting whether the flow data set meets a first condition, if so, generating a second message with the flow identification according to the flow identification and the flow data set, and sending the second message to a ticket synthesis module.
2. The method of claim 1, the first condition comprising:
the number of pieces of stream data in the stream data set is greater than a number threshold or
The stream data in the stream data set satisfies an aging condition.
3. The method of claim 2, the generating the second packet with the flow identification based on the flow identification and the flow data set, comprising:
setting aging information according to whether the stream data in the stream data set meet aging conditions;
and generating a second message with the flow identification according to the flow identification, the flow data set and the aging information, so that the second message carries the aging information.
4. The method of any of claims 1 to 3, wherein the first message is received via an optical port of a programmable hardware accelerator card, wherein the method is implemented in a processor of the programmable hardware accelerator card, and wherein the stream data set is stored in a double-rate synchronous dynamic random access memory of the programmable hardware accelerator card.
5. A communication flow processing method is applied to the call ticket synthesizing module, and comprises the following steps:
receiving a second message sent by the message analysis module and acquiring the flow identifier and a flow data set corresponding to the flow identifier from the second message;
acquiring flow information required by the synthesis of the call ticket according to the flow data set;
acquiring a flow information table for synthesizing the call ticket, wherein the flow information table stores flow information in an array form, and each flow information table entry is stored in an array unit positioned by a subscript;
and positioning an array unit by taking the stream identifier as an array subscript and adding the stream information into a stream information table entry stored in the array unit.
6. The method of claim 5, further comprising:
acquiring the flow information table;
positioning an array unit by taking the stream identifier as an array subscript and acquiring a stream information table entry from the array unit;
and synthesizing a ticket according to the flow information list item.
7. The method according to claim 5, wherein the second packet carries aging information, and correspondingly, the method further comprises:
and judging whether to age the flow information table entry corresponding to the flow identifier according to the flow aging information, and if so, aging the flow information table entry corresponding to the flow identifier.
8. The method according to claim 5, wherein the ticket synthesizing module has at least two ticket synthesizing sub-modules, and after receiving the second packet sent by the packet parsing module and acquiring the flow identifier and the flow data set corresponding to the flow identifier from the second packet, the method further comprises:
determining a first call ticket synthesizing submodule for processing the flow data set from the at least two call ticket synthesizing submodules according to the flow data in the flow data set and the number of the call ticket synthesizing submodules, so that all the flow data corresponding to the flow identifier can be sent to the same call ticket synthesizing submodule and the load balance of the call ticket synthesizing submodules is ensured;
and sending the flow identification and the flow data set to the first call ticket synthesizing submodule, and executing subsequent operations by the first call ticket synthesizing submodule.
9. A communication flow processing device is applied to a message parsing module, and comprises the following components:
the first message analysis unit is used for receiving a first message and analyzing the first message to obtain stream data;
a first flow identifier acquiring unit, configured to acquire a flow identifier corresponding to the flow data, where the flow identifier is a digital sequence that can uniquely identify a flow;
a stream data merging unit, configured to add the stream data to a stream data set corresponding to the stream identifier according to the stream identifier, where the stream data set is a set of all stream data corresponding to the stream identifier obtained within a period of time;
a first condition detecting unit configured to detect whether the set of stream data satisfies a first condition;
and the second message sending unit is used for generating a second message with the flow identifier according to the flow identifier and the flow data set and sending the second message to a ticket synthesis module.
10. A communication flow processing apparatus applied to a call ticket synthesizing module, the apparatus comprising:
a second packet parsing unit, configured to receive a second packet sent by the packet parsing module and obtain the flow identifier and a flow data set corresponding to the flow identifier from the second packet;
the flow information acquisition unit is used for acquiring flow information required by the synthesis of the ticket according to the flow data set;
the system comprises a flow information table acquisition unit, a flow information table processing unit and a call ticket processing unit, wherein the flow information table is used for acquiring a flow information table for synthesizing call tickets, the flow information table stores flow information in an array form, and each flow information table entry is stored in an array unit positioned through subscripts;
and the flow information table entry updating unit is used for positioning the array unit by taking the flow identifier as an array subscript and adding the flow information into the flow information table entry stored in the array unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621518.2A CN111988271B (en) | 2020-06-30 | 2020-06-30 | Communication flow processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621518.2A CN111988271B (en) | 2020-06-30 | 2020-06-30 | Communication flow processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111988271A true CN111988271A (en) | 2020-11-24 |
| CN111988271B CN111988271B (en) | 2021-11-16 |
Family
ID=73437650
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010621518.2A Active CN111988271B (en) | 2020-06-30 | 2020-06-30 | Communication flow processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111988271B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113938400A (en) * | 2021-08-27 | 2022-01-14 | 曙光网络科技有限公司 | Flow table management and maintenance method, device and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102301764A (en) * | 2011-07-01 | 2011-12-28 | 华为技术有限公司 | Terminal distributed information acquiring method, data acquiring device and communication system |
| CN103269479A (en) * | 2013-05-07 | 2013-08-28 | 华为技术有限公司 | Call ticket processing method, device and system |
| CN103701666A (en) * | 2013-12-27 | 2014-04-02 | 北京西塔网络科技股份有限公司 | Deep packet detection method and system |
| KR101400948B1 (en) * | 2012-12-31 | 2014-05-29 | 주식회사 엘지유플러스 | A method and an apparatus for data traffic accounting by using aaa server |
| CN106941517A (en) * | 2017-02-10 | 2017-07-11 | 北京浩瀚深度信息技术股份有限公司 | Five-tuple ticket synthetic method and device under a kind of asymmetric condition |
| CN108024222A (en) * | 2016-11-01 | 2018-05-11 | 中国电信股份有限公司 | Flow list generating method and device |
| CN108737177A (en) * | 2018-05-21 | 2018-11-02 | 中国联合网络通信有限公司重庆市分公司 | A kind of implementation method mobile Internet real-time streaming data acquisition and analyzed |
| CN109962832A (en) * | 2017-12-26 | 2019-07-02 | 华为技术有限公司 | The method and apparatus of Message processing |
| CN110138618A (en) * | 2013-12-31 | 2019-08-16 | 华为技术有限公司 | A kind of message processing method, apparatus and system |
| CN110532294A (en) * | 2019-09-03 | 2019-12-03 | 中国联合网络通信集团有限公司 | Ticket merging method, device, equipment and computer readable storage medium |
| CN111130946A (en) * | 2019-12-30 | 2020-05-08 | 联想(北京)有限公司 | Acceleration method and device for deep packet identification and storage medium |
| CN111147403A (en) * | 2019-12-27 | 2020-05-12 | 盛科网络（苏州）有限公司 | Message processing method and device, storage medium and electronic device |
-
2020
- 2020-06-30 CN CN202010621518.2A patent/CN111988271B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102301764A (en) * | 2011-07-01 | 2011-12-28 | 华为技术有限公司 | Terminal distributed information acquiring method, data acquiring device and communication system |
| KR101400948B1 (en) * | 2012-12-31 | 2014-05-29 | 주식회사 엘지유플러스 | A method and an apparatus for data traffic accounting by using aaa server |
| CN103269479A (en) * | 2013-05-07 | 2013-08-28 | 华为技术有限公司 | Call ticket processing method, device and system |
| CN103701666A (en) * | 2013-12-27 | 2014-04-02 | 北京西塔网络科技股份有限公司 | Deep packet detection method and system |
| CN110138618A (en) * | 2013-12-31 | 2019-08-16 | 华为技术有限公司 | A kind of message processing method, apparatus and system |
| CN108024222A (en) * | 2016-11-01 | 2018-05-11 | 中国电信股份有限公司 | Flow list generating method and device |
| CN106941517A (en) * | 2017-02-10 | 2017-07-11 | 北京浩瀚深度信息技术股份有限公司 | Five-tuple ticket synthetic method and device under a kind of asymmetric condition |
| CN109962832A (en) * | 2017-12-26 | 2019-07-02 | 华为技术有限公司 | The method and apparatus of Message processing |
| CN108737177A (en) * | 2018-05-21 | 2018-11-02 | 中国联合网络通信有限公司重庆市分公司 | A kind of implementation method mobile Internet real-time streaming data acquisition and analyzed |
| CN110532294A (en) * | 2019-09-03 | 2019-12-03 | 中国联合网络通信集团有限公司 | Ticket merging method, device, equipment and computer readable storage medium |
| CN111147403A (en) * | 2019-12-27 | 2020-05-12 | 盛科网络（苏州）有限公司 | Message processing method and device, storage medium and electronic device |
| CN111130946A (en) * | 2019-12-30 | 2020-05-08 | 联想(北京)有限公司 | Acceleration method and device for deep packet identification and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| B. BHUSHAN: ""OSS functions for flexible charging and billing of mobile services in a federated environment"", 《2005 9TH IFIP/IEEE INTERNATIONAL SYMPOSIUM ON INTEGRATED NETWORK MANAGEMENT, 2005》 * |
| 潘洁、高峰、刘栋、董昭、侯慧芳: ""基于DPI不对称流量的同源同宿解决方案"", 《电信科学》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113938400A (en) * | 2021-08-27 | 2022-01-14 | 曙光网络科技有限公司 | Flow table management and maintenance method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111988271B (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2744076C (en) | Providing customized information to a user based on identifying a trend | |
| US20200322237A1 (en) | Traffic detection method and traffic detection device | |
| CN106815112B (en) | Massive data monitoring system and method based on deep packet inspection | |
| US20150271202A1 (en) | Method, device, and system for detecting link layer hijacking, user equipment, and analyzing server | |
| CN108701187A (en) | Mixed hardware software distribution threat analysis | |
| CN111800412B (en) | Advanced sustainable threat tracing method, system, computer equipment and storage medium | |
| CN111641658A (en) | Request intercepting method, device, equipment and readable storage medium | |
| CN103179132A (en) | Method and device for detecting and defending CC (challenge collapsar) | |
| CN111817984B (en) | Message sending method, device, equipment and storage medium | |
| CN113810381B (en) | Crawler detection method, web application cloud firewall device and storage medium | |
| CN113114524B (en) | Spark streaming based DNS tunnel detection method and device and electronic equipment | |
| CN108206788B (en) | Traffic service identification method and related equipment | |
| Gutterman et al. | Requet: Real-time QoE metric detection for encrypted YouTube traffic | |
| CN111988271B (en) | Communication flow processing method and device | |
| CN111224831A (en) | Method and system for generating call ticket | |
| KR100608541B1 (en) | An apparatus for capturing Internet ProtocolIP packet with sampling and signature searching function, and a method thereof | |
| Wang et al. | Smart devices information extraction in home wi‐fi networks | |
| CN114285769B (en) | Shared internet surfing detection method, device, equipment and storage medium | |
| CN114020734A (en) | Flow statistics duplication removing method and device | |
| CN114866488A (en) | Information flow identification method, network chip and network equipment | |
| CN114301696A (en) | Malicious domain name detection method and device, computer equipment and storage medium | |
| CN112954667A (en) | Detection method and device for hotspot mobile terminal, computer equipment and storage medium | |
| CN109688204B (en) | File downloading method, node and terminal based on NDN (named data networking) | |
| CN109995731B (en) | Method and device for improving cache spitting flow, computing equipment and storage medium | |
| KR101148705B1 (en) | Signature generation apparatus for network behavior of applications, collection server, detection system for network behavior, and signature generation method for network behavior |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |