CN109495447B - Flow data integration method and device of distributed DDoS defense system and electronic equipment - Google Patents
Flow data integration method and device of distributed DDoS defense system and electronic equipment Download PDFInfo
- Publication number
- CN109495447B CN109495447B CN201811176542.9A CN201811176542A CN109495447B CN 109495447 B CN109495447 B CN 109495447B CN 201811176542 A CN201811176542 A CN 201811176542A CN 109495447 B CN109495447 B CN 109495447B
- Authority
- CN
- China
- Prior art keywords
- traffic
- defense
- peak
- system log
- flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 230000007123 defense Effects 0.000 title claims abstract description 205
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000010354 integration Effects 0.000 title claims abstract description 40
- 238000007596 consolidation process Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 11
- 230000002776 aggregation Effects 0.000 claims description 10
- 238000004220 aggregation Methods 0.000 claims description 10
- 101150034533 ATIC gene Proteins 0.000 claims description 6
- 238000007689 inspection Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims 1
- 238000012544 monitoring process Methods 0.000 abstract description 11
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000004931 aggregating effect Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241000287828 Gallus gallus Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0209—Architectural arrangements, e.g. perimeter networks or demilitarized zones
- H04L63/0218—Distributed architectures, e.g. distributed firewalls
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/1458—Denial of Service
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Virology (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A flow data integration method, an integration device and electronic equipment for a distributed DDoS defense system are provided, wherein the integration method comprises the following steps: acquiring a series of first traffic system logs generated by a first defense device in the defense system; acquiring a series of second flow system logs generated by second defense equipment in the defense system; grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the flow peak data into the same group; and summarizing the flow peak data in the first flow system log and the second flow system log in the same group. Therefore, the time of pushing the grouping standard from the flow system log to the server side is converted into the time corresponding to the peak data, so that the generation time of the flow data is relatively accurately positioned, and the error of the flow monitoring data is reduced.
Description
Technical Field
The present application relates to the field of network security, and in particular, to a method and an apparatus for integrating traffic data in a distributed DDoS defense system, and an electronic device.
Background
With the development of computer network technology, the challenges of network security are becoming more and more severe. Among the numerous network attacks, DDoS network attacks are one of the most powerful and difficult attacks to defend at present. DDoS attack (Distributed Denial of Service) refers to that an attacker uses "broiler chicken" to initiate a large number of requests to a target website in a short time, and consumes host resources of the target website in a large scale, so that the attacker cannot normally Service the target website.
With the increasing frequency of DDoS attacks and the increasing peak value of single attack, a single defense device has difficulty in meeting the defense load requirement. To cope with this situation, the common coping method is: and clustering and deploying a plurality of defense devices to form a distributed DDoS defense system, and distributing attacks aiming at the same high-defense IP on each defense device through a load balancing strategy. By the mode, the defense pressure of the single defense device is reduced, and the total defense capacity of the defense system is improved.
However, although the distributed defense system reduces the defense load of a single defense device, the flow log data reported by each defense device are independent in a distributed scenario, which also brings new difficulties for global flow monitoring and also raises many new problems. However, the importance of global traffic monitoring for distributed defense systems has not received sufficient attention, and the difficulties and problems presented have not been adequately addressed.
Content of application
The main purpose of the present application is to provide a flow data integration method, an integration apparatus, and an electronic device for a distributed DDoS defense system, wherein the flow data integration method can summarize flow data of each defense device in the distributed defense system, so as to facilitate global monitoring of a flow condition of the distributed defense system.
Another object of the present application is to provide a traffic data integration method, an integration apparatus, and an electronic device for a distributed DDoS defense system, wherein the traffic data integration method can more accurately locate an accurate time point of traffic data generation, and reduce an error in time of traffic monitoring.
Another objective of the present application is to provide a traffic data integration method, an integration apparatus, and an electronic device for a distributed DDoS defense system, wherein a protection policy can be adjusted more timely based on a traffic data integration result obtained by the traffic data integration method, and a customer is warned.
Another object of the present application is to provide an attack data integration method, an integration apparatus and an electronic device for a distributed DDoS defense system, wherein a real and accurate raw material is provided for showing a traffic situation to a customer based on a traffic data integration result obtained by the traffic data integration method, so as to ensure that a user can clearly and intuitively know the defense service provided by the distributed defense system.
Other advantages and features of the present application will become apparent from the following description and may be realized by means of the instrumentalities and combinations particularly pointed out in the appended claims.
To achieve at least one of the above objects or advantages, the present application provides a traffic data consolidation method for a distributed DDoS defense system, including:
acquiring a series of first traffic system logs generated by a first defense device in the defense system;
acquiring a series of second flow system logs generated by second defense equipment in the defense system;
grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the flow peak data into the same group; and
and summarizing the traffic peak data in the first traffic system log and the second traffic system log in the same group to obtain a total traffic data result of the distributed defense system in the time interval.
In an embodiment of the present application, the traffic peak data includes a peak incoming traffic and a peak drop traffic, and the traffic data integration method further includes: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value inflow into the same group; aggregating peak incoming flows in the first and second traffic system logs within the same group to obtain a total peak incoming flow of the distributed defense system within the time interval; grouping the first traffic system log and the second traffic system log which belong to the same time interval at the time corresponding to the peak value discarded traffic into the same group; and aggregating peak drop traffic in the first and second system traffic logs within the same group to obtain a total peak drop traffic for the distributed defense system over the time interval.
In an embodiment of the present application, the traffic peak data includes a peak incoming packet number and a peak discarded packet number, and the traffic data integration method further includes: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value message entering number into the same group; summarizing the peak value message entering numbers in the first flow system log and the second flow system log in the same group to obtain the total peak value message entering number of the distributed defense system in the time interval; grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak discarded message number into the same group; and summarizing the peak discarded message numbers in the first flow system log and the second flow system log in the same group to obtain the total peak discarded message number of the distributed defense system in the time interval.
In an embodiment of the present application, the first defense device and the second defense device are stic (abstract Traffic Inspection & Control System) defense devices, where the stic defense devices are configured to push out a Traffic System log at a frequency of 10 seconds/time during operation.
In an embodiment of the present application, a time corresponding to the peak incoming flow rate is consistent with a time corresponding to the peak discarded flow rate, and a time corresponding to the peak incoming message number is consistent with a time corresponding to the peak discarded message number.
According to another aspect of the present application, there is also provided a traffic data consolidation apparatus for a distributed DDoS defense system, comprising:
the flow data acquisition unit is used for acquiring a series of first flow system logs generated by first defense equipment in the defense system and acquiring a series of second flow system logs generated by second defense equipment in the defense system;
the grouping unit is used for grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the flow peak data into the same group; and
and the summarizing unit is used for summarizing the traffic peak data in the first traffic system log and the second traffic system log in the same group so as to obtain a total traffic data result of the distributed defense system in the time interval.
In an embodiment of the present application, the grouping unit is further configured to: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value inflow into the same group; and grouping the first traffic system log and the second traffic system log which belong to the same time interval at the time corresponding to the peak discarded traffic into the same group, wherein the summarizing unit is further configured to: aggregating peak incoming flows in the first and second traffic system logs within the same group to obtain a total peak incoming flow of the distributed defense system within the time interval; and aggregating peak drop traffic in the first and second system traffic logs within the same group to obtain a total peak drop traffic for the distributed defense system over the time interval.
In an embodiment of the present application, the grouping unit is further configured to: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value message entering number into the same group; and grouping the first traffic system log and the second traffic system log which belong to the same time interval at the time corresponding to the peak discarded message number into the same group, wherein the summarizing unit is further configured to: summarizing the peak value message entering numbers in the first flow system log and the second flow system log in the same group to obtain the total peak value message entering number of the distributed defense system in the time interval; and summarizing the peak discarded message numbers in the first flow system log and the second flow system log in the same group to obtain the total peak discarded message number of the distributed defense system in the time interval.
In an embodiment of the present application, the first defense device and the second defense device are stic (abstract Traffic Inspection & Control System) defense devices, where the stic defense devices are configured to push out a Traffic System log at a frequency of 10 seconds/time during operation.
In an embodiment of the present application, a time corresponding to the peak incoming flow is consistent with a time corresponding to the peak discarded flow, and a time corresponding to the peak incoming message number is consistent with a time corresponding to the peak discarded message number
According to another aspect of the present application, there is also provided an electronic device comprising a processor and a memory, wherein computer program instructions are stored in the memory, which, when executed by the processor, cause the processor to perform the flow data integration method as described above.
According to another aspect of the present application, there is also provided a computer readable storage medium having stored thereon computer program instructions operable to, when executed by a computing device, perform a traffic data integration method as described above.
Further objects and advantages of the present application will become apparent from an understanding of the ensuing description and drawings.
These and other objects, features and advantages of the present application will become more fully apparent from the following detailed description, the accompanying drawings and the claims.
Drawings
Fig. 1 is a flow chart illustrating a traffic data consolidation method for a distributed DDoS defense system according to a preferred embodiment of the present application.
Fig. 2 is a flowchart illustrating grouping and traffic data summarization of the first traffic system log and the second traffic system log based on peak traffic data in the traffic data integration method according to another preferred embodiment of the present application.
Fig. 3 is a schematic diagram illustrating a specific application of the traffic data aggregation method for a distributed DDoS defense system according to the preferred embodiment of the present application.
Fig. 4 illustrates a block diagram of a traffic data consolidation apparatus for a distributed DDoS defense system according to a preferred embodiment of the present application.
FIG. 5 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
Detailed Description
The following description is presented to disclose the application and to enable any person skilled in the art to practice the application. The preferred embodiments in the following description are given by way of example only, and other obvious variations will occur to those skilled in the art. The underlying principles of the application, as defined in the following description, may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the application.
It will be understood by those skilled in the art that in the present disclosure, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship indicated in the drawings for ease of description and simplicity of description, and do not indicate or imply that the referenced devices or components must be constructed and operated in a particular orientation and thus are not to be considered limiting.
It is understood that the terms "a" and "an" should be interpreted as meaning that a number of one element or element is one in one embodiment, while a number of other elements is one in another embodiment, and the terms "a" and "an" should not be interpreted as limiting the number.
Summary of the application
As described above, as DDoS attacks become more frequent and single attack peaks become higher, it is difficult for a single defense device to meet the defense load requirements. For this reason, a common strategy is to deploy multiple defense devices in a clustering manner to form a distributed DDoS defense system. In the distributed defense system, each defense device disperses the attack of the same high-defense IP by a load balancing strategy. However, while distributed defense systems can reduce the defense load of individual defense devices, they also present new difficulties for global traffic monitoring.
Those skilled in the art will appreciate that it is important to globally monitor the traffic conditions of the distributed defense system. For example, the statistical traffic situation can be used to understand the real-time defense situation of each defense device and each high-defense IP in the distributed defense system, so as to adjust a more appropriate defense strategy in time; providing early warning notification service for the client according to the condition whether the attack flow exceeds the defense upper limit; and, implementing transparent high-defense service billing to the client based on the traffic data.
However, the importance of global traffic monitoring for distributed defense systems has not received sufficient attention, and the difficulties and problems presented have not been adequately addressed.
First, although some enterprises complete the transition from single-device defense to multi-defense-device clustering defense, their associated traffic data monitoring strategies still only target single devices. The specific method comprises the following steps: the method includes the steps that flow data of a single defense device in a certain continuous time period in a preset time interval, for example, certain continuous flow data in 0-20 s, are obtained through a flow Syslog (System Log) generated by the single defense device. The reason for adopting such a statistical mode (counting the flow data in a certain continuous time period within a preset time interval) is that: the flow system log is pushed out from the defense device and is transmitted from the defense device to the server side for processing, a certain delay is generated in the two processes, and the time of the delay is an uncertain value.
Here, it should be particularly noted that the existence of an indelible delay from the generation of the traffic system log to the transmission to the server side for processing is the largest source of error in the distributed defense system traffic data statistics. How to eliminate or reduce the effect of this delay will be explained in detail in the following detailed description of the embodiments.
On the basis, a global flow data aggregation scheme for a distributed defense system is developed, and the core of the global flow data aggregation scheme is to optimize the aggregation mode of flow system logs. Specifically, 10 seconds are taken as a time node, flow system logs belonging to the same high defense IP are grouped by taking 10s as an interval according to log-time (an attribute field in the flow Syslog represents the time for pushing the flow Syslog to a server side), and the flow system logs from different defense devices in the 10s are collected after the 10s is finished, so that a global flow data statistical result of the distributed defense system is obtained. However, this approach still ignores the delay existing between the generation and the transmission of the traffic system log to the server, so that in the actual traffic statistics process, the traffic data originally belonging to different time periods may be added, and an error may be generated.
In view of the above technical problems, the basic idea of the present application is to select a suitable traffic data packet standard suitable for use in a distributed defense system, and in particular to create or select an actual time (not affected by delay) at which traffic can be generated that represents a traffic system log; and further, performing grouping statistics on the traffic data based on the grouping standard to obtain a traffic data aggregation result.
Based on this, the application provides a traffic data integration method for a distributed DDoS defense system, which includes the steps of firstly acquiring a series of first traffic system logs generated by first defense equipment in the defense system; however, obtaining a series of second traffic system logs generated by a second defense device within the defense system; furthermore, the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the flow peak data are grouped into the same group; finally, traffic peak data in the first traffic system log and the second traffic system log in the same group are collected to obtain a total traffic data result of the distributed defense system in the time interval. Therefore, the time of the grouping standard pushed to the server side by the flow system log is converted into the time corresponding to the peak data in the flow data, so that the generation time of the flow data is relatively accurately positioned, the error of the flow monitoring data is reduced, the protection strategy can be adjusted more timely, and early warning can be carried out on clients.
Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.
Exemplary method
Fig. 1 is a flowchart illustrating an attack data integration method for a distributed DDoS defense system according to a preferred embodiment of the present application. As shown in fig. 1, the method for integrating traffic data for a distributed DDoS distributed defense system according to the preferred embodiment of the present application includes: s110, acquiring a series of first flow system logs generated by first defense equipment in the defense system; s120, acquiring a series of second flow system logs generated by second defense equipment in the defense system; s130, grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the flow peak data into the same group; and S140, summarizing the traffic peak data in the first traffic system log and the second traffic system log in the same group to obtain a total traffic data result of the distributed defense system in the time interval.
For convenience of illustration and understanding, in the preferred embodiment of the present application, the traffic data integration method for the distributed defense system disclosed in the present application is described by taking the example that the distributed defense system includes two defense devices. That is, in the preferred embodiment of the present application, the distributed defense system includes a first defense device and a second defense device, which are deployed in a clustered manner and adopt a load balancing policy to dispersedly undertake attacks against the same high-defense IP. In particular, in the embodiment of the present application, the first defense device and the second defense device may be configured as an stic defense device (abstract Traffic Inspection & Control System), which may be other types of DDoS defense devices, but is not limited to this application.
In steps S110 and S120, a series of first traffic system logs generated by a first defense device within the defense system are acquired, and a series of second traffic system logs generated by a second defense device within the defense system are acquired. In other words, discrete traffic data system logs of each defense device within the distributed defense system are obtained separately.
It will be appreciated by those skilled in the art that each Atic defense device will generate a series of traffic data System logs (System Log, abbreviated subsequently as Syslog) when the distributed defense System is in operation under attack by attack traffic. And, the generated traffic data Syslog is accumulated on the defense device and pushed out to the server side for processing at a fixed frequency (e.g., 10 seconds/time). Correspondingly, in the embodiment of the application, the first defense device can generate a series of first traffic system logs during operation, and transmit the accumulated first traffic system logs to the server side for processing at a frequency of 10 seconds/time; and the second defense equipment can generate a series of second flow system logs during operation, and transmits the accumulated second flow system logs to the server side for processing at the frequency of 10 seconds per time.
Typically, the traffic data system log includes at least the following attribute fields: log-time (time of pushing traffic system Log to server), Zone _ IP (corresponding high defense IP), Device _ IP (corresponding defense IP), Max _ in _ pps (peak incoming message number), Max _ drop _ pps (peak dropped message number), Max _ in _ kbps (peak incoming traffic), and Max _ drop _ kbps (peak dropped traffic).
Here, it should be particularly noted that a log-time field is included in the traffic data system log, which characterizes the time that the traffic system log is pushed from the defending device to the server side. As described above, in the existing traffic data summarization scheme, 10s is selected as a time node, traffic syslogs belonging to the same high defense IP are grouped by taking 10s as an interval according to log-time (an attribute field in the traffic Syslog indicates the time when the traffic Syslog is pushed to a server), and after 10s is finished, traffic syslogs from different defense devices in the 10s are summarized to obtain a global traffic data statistical result of the distributed defense system. In this way, delay existing between generation and transmission of the traffic system log to the server is ignored, so that in the actual traffic statistical process, errors may occur due to addition of traffic data originally belonging to different time periods.
Accordingly, in the embodiment of the present application, other fields are selected as the traffic data packet criteria in the distributed defense system, and preferably, the fields are selected to represent the generation characterization time of the traffic system log without being affected by the delay. And further, performing grouping statistics on the traffic data based on the grouping standard to obtain a traffic data aggregation result.
More specifically, in step S130 and step S140, the first traffic system log and the second traffic system log which belong to the same time interval at the time corresponding to the traffic peak data are grouped into the same group, and the traffic peak data in the first traffic system log and the second traffic system log in the same group are summarized to obtain the total traffic data result of the distributed defense system in the time interval. That is, in the present application, the time corresponding to the traffic peak data is used as the traffic data statistical grouping standard of the distributed defense system, so as to summarize the traffic data of the distributed defense system based on the time.
In practical application, the items to be summarized of the flow data at least include: the defense device comprises an incoming flow, a drop flow, incoming message data, a drop message number and the like, wherein the incoming flow represents the flow (with the unit of Kbps or Gbps) reaching the defense device at the client side, and the drop flow represents the flow which is judged to be attacked and filtered by the defense device. It is worth mentioning that in case the defending device can completely clear away the attack traffic without leakage, the attack traffic is discarded. In data characterization, the peak inflow rate can be selected to represent the inflow rate of the corresponding defense device in a specific time interval; the selectable peak drop traffic represents the drop traffic corresponding to the defensive device within a particular time interval; the number of the peak incoming messages can be selected to represent the number of the incoming messages of the corresponding defense equipment in a specific time interval; and selecting the peak discarded message number to represent the discarded message number of the corresponding defense equipment in a specific time interval.
In other words, in order to group the first traffic system log and the second traffic system log generated by the first defense device and the second defense device, an attribute field needs to be newly added in the corresponding traffic system log: max _ in _ pps _ time, Max _ drop _ pps _ time, Max _ in _ kbps _ time, and Max _ drop _ kbps _ time, wherein Max _ in _ pps _ time represents the time corresponding to the peak incoming packet number, Max _ in _ pps _ time represents the time corresponding to the peak discarded packet number, Max _ in _ kbps _ time represents the time corresponding to the peak incoming flow, and Max _ drop _ kbps _ time represents the time corresponding to the peak discarded flow.
Accordingly, steps S130 and S140 further include the steps of: s210, grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value inflow into the same group; s220, summarizing peak incoming flow rates in the first flow system log and the second flow system log in the same group to obtain total peak incoming flow rate of the distributed defense system in the time interval; s230, grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value discarded flow into the same group; s240, aggregating peak drop flow in the first flow system log and the second flow system log in the same group to obtain total peak drop flow of the distributed defense system in the time interval; s250, grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak incoming message number into the same group; s260, summarizing the peak value message entering number in the first flow system log and the second flow system log in the same group to obtain the total peak value message entering number of the distributed defense system in the time interval; s270, grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak discarded message number into the same group; and S280, summarizing the peak discarded message number in the first system traffic log and the second system traffic log in the same group to obtain the total peak discarded message number of the distributed defense system in the time interval.
It is worth mentioning that the time of the packet standard pushed to the server end by the traffic Syslog is converted into the time corresponding to the peak data in the traffic data, so that the generation time of the traffic data is relatively more accurately positioned, the error of the traffic monitoring data is reduced, and the protection strategy can be adjusted more timely and the early warning can be performed on the client.
In summary, the process of integrating the traffic data of the distributed defense system by the traffic data integration method provided by the present application is illustrated. By the flow data integration method, the generation time of the flow data can be relatively accurately positioned to reduce the error of the flow monitoring data, so that the protection strategy can be adjusted more timely and early warning can be performed on customers.
It should be appreciated that, although the distributed DDoS defense system includes two defense devices as an example, those skilled in the art will understand that the traffic data consolidation method disclosed in the present application can also be applied to the traffic data consolidation of the distributed DDoS defense system including more defense devices. The present application is not limited in this respect.
Fig. 3 is a schematic diagram illustrating a specific application of the traffic data aggregation method for a distributed DDoS defense system according to the preferred embodiment of the present application. As shown in fig. 3, in this specific application, the distributed defense system includes 3 defense devices, wherein after receiving attack traffic, each defense device is in an operating state and generates a corresponding traffic data system log (Syslog), respectively. And the flow system logs are reported to an Atic equipment server end at a certain frequency after accumulating preset time on corresponding defense equipment. And at the server side, processing and summarizing the second-level flow system logs by using a pre-loaded data acquisition program to obtain a total flow data result of the distributed defense system.
Exemplary flow data integration device
Fig. 4 illustrates a block diagram of a traffic data consolidation apparatus for a distributed DDoS defense system according to a preferred embodiment of the present application.
As shown in fig. 4, the traffic data consolidation apparatus 400 for a distributed DDoS defense system according to the preferred embodiment of the present application includes: a flow data obtaining unit 410, configured to obtain a series of first flow system logs generated by a first defense device in the defense system, and obtain a series of second flow system logs generated by a second defense device in the defense system; a grouping unit 420, configured to group the first traffic system log and the second traffic system log that belong to the same time interval at a time corresponding to traffic peak data into the same group; and a summarizing unit 430, configured to summarize traffic peak data in the first traffic system log and the second traffic system log in the same group to obtain a total traffic data result of the distributed defense system in the time interval.
In an example, in the above integration apparatus 400, the grouping unit 420 is further configured to: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value inflow into the same group; and grouping the first traffic system log and the second traffic system log, which belong to the same time interval at the time corresponding to the peak discarded traffic, into the same group, wherein the summarizing unit 430 is further configured to: aggregating peak incoming flows in the first and second traffic system logs within the same group to obtain a total peak incoming flow of the distributed defense system within the time interval; and aggregating peak drop traffic in the first and second system traffic logs within the same group to obtain a total peak drop traffic for the distributed defense system over the time interval.
In one example, in the above integration apparatus 400, the grouping unit 420 is further configured to: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value message entering number into the same group; and grouping the first traffic system log and the second traffic system log, which belong to the same time interval at the time corresponding to the peak discarded message number, into the same group, wherein the summarizing unit 430 is further configured to: summarizing the peak value message entering numbers in the first flow system log and the second flow system log in the same group to obtain the total peak value message entering number of the distributed defense system in the time interval; and summarizing the peak discarded message numbers in the first flow system log and the second flow system log in the same group to obtain the total peak discarded message number of the distributed defense system in the time interval.
In one example, in the above-mentioned Traffic data integration apparatus 400, the first defense device and the second defense device are stic (abstract Traffic Inspection & Control System) defense devices, wherein the stic defense devices are configured to push out the Traffic System log at a frequency of 10 seconds/time during operation.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the above-described traffic data consolidation apparatus 400 have been described in detail in the traffic data consolidation method for a distributed DDoS defense system described above with reference to fig. 1 to 3, and thus, a repetitive description thereof will be omitted.
As described above, the traffic data consolidation apparatus according to the embodiment of the present application may be implemented in various terminal devices, for example, a server of a distributed DDoS defense system. In one example, the traffic data integration apparatus according to the embodiment of the present application may be integrated into the terminal device as a software module and/or a hardware module. For example, the traffic data integration means may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the traffic data integration device may also be one of many hardware modules of the terminal device.
Alternatively, in another example, the traffic data integration apparatus and the terminal device may be separate terminal devices, and the traffic data integration apparatus may be connected to the terminal device through a wired and/or wireless network and transmit the interaction information according to an agreed data format.
Illustrative electronic device
Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 5.
FIG. 5 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
As shown in fig. 5, the electronic device 10 includes one or more processors 11 and memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device 13 may be, for example, a keyboard, a mouse, or the like.
The output device 14 can output various information including the summarized flow data result and the like to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 5, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
Illustrative computer program product
In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in a traffic data consolidation method for a distributed DDoS defense system according to various embodiments of the present application described in the "exemplary methods" section of this specification, supra.
The computer program product may write program code for carrying out operations for embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + +, or the like, as well as conventional procedural programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in a traffic data consolidation method for a distributed DDoS defense system according to various embodiments of the present application described in the "exemplary methods" section above in this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
It will be appreciated by persons skilled in the art that the embodiments of the present application described above and illustrated in the drawings are given by way of example only and are not limiting of the present application. The objectives of the present application have been fully and effectively attained. The functional and structural principles of the present application have been shown and described in the examples, and any variations or modifications of the embodiments of the present application may be made without departing from the principles.
Claims (8)
1. A method for integrating flow data of a distributed DDoS defense system is characterized by comprising the following steps:
acquiring a series of first traffic system logs generated by a first defense device in the defense system;
acquiring a series of second flow system logs generated by second defense equipment in the defense system;
grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the flow peak data into the same group; and
summarizing the traffic peak data in the first traffic system log and the second traffic system log in the same group to obtain a total traffic data result of the distributed DDoS defense system in the time interval;
wherein the traffic peak data comprises a peak incoming traffic and a peak drop traffic;
the method for collecting the traffic peak data in the distributed DDoS defense system comprises the following steps of grouping the first traffic system logs and the second traffic system logs of which the time corresponding to the traffic peak data belongs to the same time interval into the same group, and collecting the traffic peak data in the first traffic system logs and the second traffic system logs in the same group to obtain a total traffic data result of the distributed DDoS defense system in the time interval, wherein the method comprises the following steps:
grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value inflow into the same group;
summarizing peak incoming traffic in the first traffic system log and the second traffic system log in the same group to obtain total peak incoming traffic of the distributed DDoS defense system in the time interval;
grouping the first traffic system log and the second traffic system log which belong to the same time interval at the time corresponding to the peak value discarded traffic into the same group; and
and summarizing the peak drop traffic in the first traffic system log and the second traffic system log in the same group to obtain the total peak drop traffic of the distributed DDoS defense system in the time interval.
2. The traffic data aggregation method according to claim 1, wherein the traffic peak data includes a peak incoming packet number and a peak discarded packet number, and wherein the traffic data aggregation method further includes:
grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value message entering number into the same group;
summarizing the peak value message entering number in the first flow system log and the second flow system log in the same group to obtain the total peak value message entering number of the distributed DDoS defense system in the time interval;
grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak discarded message number into the same group; and
and summarizing the peak discarded message numbers in the first traffic system log and the second traffic system log in the same group to obtain the total peak discarded message number of the distributed DDoS defense system in the time interval.
3. The Traffic data integration method according to any one of claims 1 or 2, wherein the first defense device and the second defense device are respectively an Atic (Abnormal Traffic Inspection & Control System) defense device, wherein the Atic defense device is configured to push out the Traffic System log at a frequency of 10 seconds/time during operation.
4. A traffic data consolidation apparatus for a distributed DDoS defense system, comprising:
the flow data acquisition unit is used for acquiring a series of first flow system logs generated by first defense equipment in the defense system and acquiring a series of second flow system logs generated by second defense equipment in the defense system;
the grouping unit is used for grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the flow peak data into the same group; and
the summarizing unit is used for summarizing the traffic peak data in the first traffic system log and the second traffic system log in the same group to obtain a total traffic data result of the distributed DDoS defense system in the time interval;
wherein the grouping unit is further configured to: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value inflow into the same group; the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value discarded flow are grouped into the same group;
wherein, the summarizing unit is further configured to: summarizing peak incoming traffic in the first traffic system log and the second traffic system log in the same group to obtain total peak incoming traffic of the distributed DDoS defense system in the time interval; and summarizing peak drop traffic in the first traffic system log and the second traffic system log in the same group to obtain total peak drop traffic of the distributed DDoS defense system in the time interval.
5. The traffic data consolidation apparatus according to claim 4, wherein the grouping unit is further configured to: grouping the first flow system log and the second flow system log which belong to the same time interval at the time corresponding to the peak value message entering number into the same group; and grouping the first traffic system log and the second traffic system log which belong to the same time interval at the time corresponding to the peak discarded message number into the same group, wherein the summarizing unit is further configured to: summarizing the peak value message entering number in the first flow system log and the second flow system log in the same group to obtain the total peak value message entering number of the distributed DDoS defense system in the time interval; and summarizing the peak discarded message number in the first traffic system log and the second traffic system log in the same group to obtain the total peak discarded message number of the distributed DDoS defense system in the time interval.
6. The Traffic data consolidation apparatus of claim 5, wherein the first and second defense devices are Atic (abstract Traffic Inspection & Control System) defense devices, wherein the Atic defense devices are configured to push Traffic System logs outward at a frequency of 10S/time during operation.
7. An electronic device, comprising:
a processor; and
a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform a method of traffic data integration according to any of claims 1-3.
8. A computer readable storage medium having computer program instructions stored thereon, which, when executed by a computing device, are operable to perform the traffic data consolidation method according to any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811176542.9A CN109495447B (en) | 2018-10-10 | 2018-10-10 | Flow data integration method and device of distributed DDoS defense system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811176542.9A CN109495447B (en) | 2018-10-10 | 2018-10-10 | Flow data integration method and device of distributed DDoS defense system and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109495447A CN109495447A (en) | 2019-03-19 |
| CN109495447B true CN109495447B (en) | 2021-05-07 |
Family
ID=65690197
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811176542.9A Expired - Fee Related CN109495447B (en) | 2018-10-10 | 2018-10-10 | Flow data integration method and device of distributed DDoS defense system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109495447B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114629694B (en) * | 2022-02-28 | 2024-01-19 | 天翼安全科技有限公司 | Distributed denial of service (DDoS) detection method and related device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500185A (en) * | 2013-09-13 | 2014-01-08 | 北京奇虎科技有限公司 | Data table generation method and system based on multi-platform data |
| CN103795529A (en) * | 2014-02-26 | 2014-05-14 | 东南大学 | Wireless sensor network data safety infusion method based secret key vectors |
| CN106789954A (en) * | 2016-11-30 | 2017-05-31 | 杭州迪普科技股份有限公司 | A kind of method and apparatus of the DDOS attack identification based on multi -CPU |
| CN108259426A (en) * | 2016-12-29 | 2018-07-06 | 华为技术有限公司 | A kind of ddos attack detection method and equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10616271B2 (en) * | 2017-01-03 | 2020-04-07 | Microsemi Frequency And Time Corporation | System and method for mitigating distributed denial of service attacks |
-
2018
- 2018-10-10 CN CN201811176542.9A patent/CN109495447B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500185A (en) * | 2013-09-13 | 2014-01-08 | 北京奇虎科技有限公司 | Data table generation method and system based on multi-platform data |
| CN103795529A (en) * | 2014-02-26 | 2014-05-14 | 东南大学 | Wireless sensor network data safety infusion method based secret key vectors |
| CN106789954A (en) * | 2016-11-30 | 2017-05-31 | 杭州迪普科技股份有限公司 | A kind of method and apparatus of the DDOS attack identification based on multi -CPU |
| CN108259426A (en) * | 2016-12-29 | 2018-07-06 | 华为技术有限公司 | A kind of ddos attack detection method and equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于多核CPU的DDOS检测技术研究;朱裕福;《中国优秀硕士学位论文电子期刊全文库》;20130315;139-154 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109495447A (en) | 2019-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10984010B2 (en) | Query summary generation using row-column data storage | |
| US9836600B2 (en) | Method and apparatus for detecting a multi-stage event | |
| EP4060958B1 (en) | Attack behavior detection method and apparatus, and attack detection device | |
| US20130305365A1 (en) | System and method for optimization of security traffic monitoring | |
| US9584533B2 (en) | Performance enhancements for finding top traffic patterns | |
| KR100748246B1 (en) | Multi-step integrated security monitoring system and method using intrusion detection system log collection engine and traffic statistic generation engine | |
| CN109766695A (en) | A kind of network security situational awareness method and system based on fusion decision | |
| CN103152352A (en) | Perfect information security and forensics monitoring method and system based on cloud computing environment | |
| CN110300100A (en) | The association analysis method and system of log audit | |
| US11706114B2 (en) | Network flow measurement method, network measurement device, and control plane device | |
| CN104125214A (en) | Security architecture system for realizing software definition security and security controller | |
| JP2010237975A (en) | Incident monitoring apparatus, method and program | |
| CN104869155A (en) | Data auditing method and device | |
| CN109561097B (en) | Method, device, equipment and storage medium for detecting security vulnerability injection of structured query language | |
| US9122546B1 (en) | Rapid processing of event notifications | |
| CN104092588B (en) | A kind of exception flow of network detection method combined based on SNMP with NetFlow | |
| CN111740868A (en) | Alarm data processing method and device and storage medium | |
| CN108712365B (en) | DDoS attack event detection method and system based on flow log | |
| CN109495447B (en) | Flow data integration method and device of distributed DDoS defense system and electronic equipment | |
| CN115712646A (en) | Alarm strategy generation method, device and storage medium | |
| EP4274160A1 (en) | System and method for machine learning based malware detection | |
| CN109327441B (en) | Attack data integration method and integration device of distributed DDoS defense system and electronic equipment | |
| TWI720963B (en) | System and method for high frequency heuristic data acquisition and analytics of information security events | |
| WO2016202025A1 (en) | Trap message processing method and apparatus | |
| CN114172707A (en) | Fast-Flux botnet detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210507 |