CN114244584A - Method and device for realizing automatic suppression and protection based on network equipment log - Google Patents
Method and device for realizing automatic suppression and protection based on network equipment log Download PDFInfo
- Publication number
- CN114244584A CN114244584A CN202111457582.2A CN202111457582A CN114244584A CN 114244584 A CN114244584 A CN 114244584A CN 202111457582 A CN202111457582 A CN 202111457582A CN 114244584 A CN114244584 A CN 114244584A
- Authority
- CN
- China
- Prior art keywords
- pressing
- alarm
- suppression
- task
- resource pool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/101—Access control lists [ACL]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method and a device for realizing automatic pressing and protection based on network equipment logs, wherein the method comprises the following steps: respectively acquiring network equipment log information by a front-end processor server configured with each resource pool, and classifying and preprocessing the network equipment log information; the safety protection management system receives the preprocessed alarm and flow information, analyzes and stores the information in a warehouse, and dynamically presents the information in real time; the safety protection management system executes automatic suppression logic according to the acquired IP alarm and flow information, the IP protection level, the alarm notification threshold and the automatic suppression threshold, judges whether a real-time suppression task exists at present, suppresses whether the real-time suppression task is in a waiting period or an observation period, and then implements or adjusts a corresponding protection strategy and issues the protection strategy. The method and the device flexibly configure the IP protection strategy of the resource pool according to the service characteristics and the requirements of the device, thereby reducing the working difficulty of operation and maintenance managers; and the corresponding suppression or protection rule is dynamically adjusted according to the flow value in the log of the network equipment, so that the network security protection efficiency is improved.
Description
Technical Field
The invention relates to the field of safety protection based on analysis of network equipment logs, in particular to a method and a device for realizing automatic pressing and protection based on network equipment logs.
Background
In the prior art, there are a plurality of security protection methods implemented based on analysis of network device logs, which generally configure a fixed protection policy for an IP to be protected, but cannot perform dynamic control and adjustment according to a real-time traffic attack condition, for example, once a blacklist is determined, a blacklist and a blacklist cannot be automatically updated; the IP of the resource pool which is not recorded in advance can be automatically discarded even if the IP is attacked by large flow; for example, after the automatic pressing is started, the pressing strategy cannot be adjusted by combining the current flow value and the states of whether the current flow value enters a waiting period, an observation period and the like; therefore, the prior art is not flexible enough, has relatively single function, and needs an operation and maintenance manager to manually adjust the protection strategy, which results in lower efficiency of network security protection.
Disclosure of Invention
In order to solve the above problems in the prior art, the present invention provides a method and an apparatus for implementing automatic suppression and protection based on a network device log, which implement automatic suppression and protection when the entry of an alarm and flow log and the flow exceed a threshold value by analyzing the alarm log and flow log information uploaded by the network device and combining with a security protection policy preconfigured by a user.
In order to achieve the purpose, the invention adopts the following technical scheme:
in an embodiment of the present invention, a method for implementing automatic throttling and protection based on a network device log is provided, where the method includes:
respectively acquiring network equipment log information by a front-end processor server configured with each resource pool, and classifying and preprocessing the network equipment log information;
the safety protection management system receives the preprocessed alarm and flow information, analyzes and stores the information in a warehouse, and dynamically presents the information in real time;
the safety protection management system executes automatic suppression logic according to the acquired resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold and the resource pool automatic suppression threshold, judges whether a real-time suppression task exists at present, suppresses whether the real-time suppression task is in a waiting period or an observation period, and then implements or adjusts a corresponding protection strategy and issues the protection strategy.
Further, the method comprises the steps of respectively collecting the log information of the network equipment by configuring the front-end processor server of each resource pool, and carrying out classification and pretreatment, wherein the classification comprises the following steps:
respectively acquiring network equipment logs by a front-end processor server configured with each resource pool, dividing the acquired network equipment log information into alarm log information and flow log information, dividing the alarm log information into real-time alarm log information and historical alarm log information by analyzing alarm starting and ending time in the alarm log information, and matching the flow log information with the real-time alarm log information through a resource pool IP (Internet protocol);
and deleting useless field information in the collected network equipment log information, and discarding logs lower than a flow basic threshold value.
Further, the safety protection management system receives the preprocessed alarm and flow information, analyzes and stores the information in a warehouse, and dynamically presents the information in real time, and the method comprises the following steps:
and the safety protection management system receives the preprocessed alarm and flow information, judges the protection level of the alarm IP by matching with various information of the pre-configured resource pool IP, stores the alarm and flow information in a warehouse and displays the alarm and flow information on a front-end page.
Further, the safety protection management system executes an automatic pressing logic according to the acquired resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold and the resource pool automatic pressing threshold, judges whether a real-time pressing task exists at present, presses whether the pressing task is in a waiting period or an observation period, and then implements or adjusts and issues a corresponding protection strategy, including:
the safety protection management system acquires resource pool IP alarm and flow information, the protection level of the resource pool IP, a resource pool alarm notification threshold value and a resource pool automatic suppression threshold value; the protection level of the resource pool IP comprises a white list, a VIP list, a black list and a black + list, wherein the white list only sends an alarm notice and does not start pressing, the VIP list starts overseas pressing, the black list gradually upgrades the pressing level, the overseas pressing level is greater than the inter-network pressing level, the inter-network pressing level is greater than the whole-network pressing level, and the black + list directly starts the whole-network pressing;
if the flow value is larger than the automatic suppression threshold value and no suppression task is executed, starting the corresponding suppression task according to the protection level of the resource pool IP and setting the suppression task as a waiting period;
if the flow value is larger than the automatic pressing threshold value, the pressing task which is being executed exists, and the pressing level is full-network pressing, the pressing task is continuously executed;
if the flow value is larger than the automatic suppression threshold value, and a suppression task which is being executed exists, the suppression level is overseas suppression or inter-network suppression, and the protection level of the resource pool IP is a blacklist, judging whether the suppression task is in a waiting period or an observation period, or the suppression task which is not in the waiting period or the observation period, and upgrading to a higher-level suppression task, wherein the blacklist upgraded to the whole-network suppression is automatically adjusted to be a blacklist; changing the pressing task in the observation period back to the original pressing level, and continuing to execute the pressing task;
if the timing task detects that the pressing task is in a waiting period state, the pressing task is automatically set to be in a pressing state after 15 minutes;
if the timing task detects that the pressing task is executed for more than 2 hours, setting the pressing task to be in an observation period state;
if the timing task detects that the pressing task is executed for 24 hours, calling a decompression interface to close pressing;
and after all the suppression and protection strategies are successfully issued, the safety protection management system informs the user of successful protection in a short message and mail mode.
In an embodiment of the present invention, a device for implementing automatic suppression and protection based on a network device log is further provided, where the device includes:
the network equipment log acquisition module is used for respectively acquiring the log information of the network equipment through the front-end processor server configured with each resource pool, and classifying and preprocessing the log information;
the alarm data storage and display module is used for analyzing and warehousing the alarm and flow information which is received by the safety protection management system after the preprocessing, and dynamically presenting the alarm and flow information in real time;
and the automatic pressing and protecting module is used for executing automatic pressing logic by the safety protection management system according to the acquired resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold and the resource pool automatic pressing threshold, judging whether a real-time pressing task exists at present, pressing whether the pressing task is in a waiting period or an observation period, and further implementing or adjusting a corresponding protection strategy and issuing the protection strategy.
Further, the network device log collection module is specifically configured to:
respectively acquiring network equipment logs by a front-end processor server configured with each resource pool, dividing the acquired network equipment log information into alarm log information and flow log information, dividing the alarm log information into real-time alarm log information and historical alarm log information by analyzing alarm starting and ending time in the alarm log information, and matching the flow log information with the real-time alarm log information through a resource pool IP (Internet protocol);
and deleting useless field information in the collected network equipment log information, and discarding logs lower than a flow basic threshold value.
Further, the alarm data storage and display module is specifically configured to:
and the safety protection management system receives the preprocessed alarm and flow information, judges the protection level of the alarm IP by matching with various information of the pre-configured resource pool IP, stores the alarm and flow information in a warehouse and displays the alarm and flow information on a front-end page.
Further, the automatic pressing and protection module is specifically configured to:
the safety protection management system acquires resource pool IP alarm and flow information, the protection level of the resource pool IP, a resource pool alarm notification threshold value and a resource pool automatic suppression threshold value; the protection level of the resource pool IP comprises a white list, a VIP list, a black list and a black + list, wherein the white list only sends an alarm notice and does not start pressing, the VIP list starts overseas pressing, the black list gradually upgrades the pressing level, the overseas pressing level is greater than the inter-network pressing level, the inter-network pressing level is greater than the whole-network pressing level, and the black + list directly starts the whole-network pressing;
if the flow value is larger than the automatic suppression threshold value and no suppression task is executed, starting the corresponding suppression task according to the protection level of the resource pool IP and setting the suppression task as a waiting period;
if the flow value is larger than the automatic pressing threshold value, the pressing task which is being executed exists, and the pressing level is full-network pressing, the pressing task is continuously executed;
if the flow value is larger than the automatic suppression threshold value, and a suppression task which is being executed exists, the suppression level is overseas suppression or inter-network suppression, and the protection level of the resource pool IP is a blacklist, judging whether the suppression task is in a waiting period or an observation period, or the suppression task which is not in the waiting period or the observation period, and upgrading to a higher-level suppression task, wherein the blacklist upgraded to the whole-network suppression is automatically adjusted to be a blacklist; changing the pressing task in the observation period back to the original pressing level, and continuing to execute the pressing task;
if the timing task detects that the pressing task is in a waiting period state, the pressing task is automatically set to be in a pressing state after 15 minutes;
if the timing task detects that the pressing task is executed for more than 2 hours, setting the pressing task to be in an observation period state;
if the timing task detects that the pressing task is executed for 24 hours, calling a decompression interface to close pressing;
and after all the suppression and protection strategies are successfully issued, the safety protection management system informs the user of successful protection in a short message and mail mode.
In an embodiment of the present invention, a computer device is further provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the foregoing method for implementing automatic throttling and protection based on a network device log.
In an embodiment of the present invention, a computer-readable storage medium is further provided, where a computer program for executing the method for implementing automatic throttling and protection based on a network device log is stored in the computer-readable storage medium.
Has the advantages that:
1. the invention flexibly configures the protection strategy of the resource pool IP according to the service characteristics and requirements of the system, and reduces the working difficulty of operation and maintenance managers.
2. The invention dynamically adjusts the corresponding pressing or protection rule according to the flow value in the network equipment log, and improves the efficiency of network safety protection.
Drawings
FIG. 1 is a flow chart of a method for implementing automatic throttling and protection based on a network device log according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an automatic pressing process according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of an apparatus for implementing automatic throttling and protection based on a network device log according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The principles and spirit of the present invention will be described below with reference to several exemplary embodiments, which should be understood to be presented only to enable those skilled in the art to better understand and implement the present invention, and not to limit the scope of the present invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
According to the embodiment of the invention, the invention provides a method and a device for realizing automatic pressing and protection based on network equipment logs, wherein a front-end processor server of each resource pool respectively collects the network equipment logs, classifies and preprocesses the network equipment logs to form alarm and flow information, and distributes the alarm and flow information to a safety protection management system; the safety protection management system stores and analyzes the alarm and flow information in a warehouse and dynamically presents the alarm and flow information in real time; and the safety protection management system judges whether an automatic pressing and protection strategy needs to be started or not according to the protection IP alarm and flow information, the protection level of the resource pool IP, the alarm notification threshold and the automatic pressing threshold, so that the safety protection of the resource pool IP is realized.
The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the invention.
Fig. 1 is a flowchart illustrating a method for implementing automatic throttling and protection based on a network device log according to an embodiment of the present invention. As shown in fig. 1, the method includes:
1. respectively acquiring network equipment log information by a front-end processor server configured with each resource pool, and classifying and preprocessing the network equipment log information;
respectively collecting network equipment logs by a front-end processor server configured with each resource pool, dividing the collected network equipment log information into alarm log information and flow log information, dividing the alarm log information into real-time alarm log information and historical alarm log information by analyzing alarm starting and ending time in the alarm log information, matching the flow log information with the real-time alarm log information through a resource pool IP, and filling a flow value in the flow log information into the alarm log information;
and deleting useless field information in the collected network equipment log information, and discarding the log which is lower than a flow basic threshold value in consideration of huge data volume of the log and limited load capacity of a data analysis module.
2. The safety protection management system receives the preprocessed alarm and flow information, analyzes and stores the information in a warehouse, and dynamically presents the information in real time;
the safety protection management system receives the preprocessed alarm and flow information, judges the protection level of the alarm IP by matching with various information of the pre-configured resource pool IP, stores the alarm and flow information into a storage layer and displays the alarm and flow information on a front-end page (display layer);
the various types of information of the resource pool IP comprise: the IP protection level of the resource pool, the alarm notification threshold value of the resource pool and the compression threshold value of the resource pool;
the protection level of the alarm IP comprises: white list, VIP list, black + list.
3. The safety protection management system executes automatic suppression logic according to the acquired resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold and the resource pool automatic suppression threshold, judges whether a real-time suppression task exists at present, suppresses whether the real-time suppression task is in a waiting period or an observation period, and then implements or adjusts a corresponding protection strategy and issues the protection strategy;
the security protection management system obtains the resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold, and the resource pool automatic suppression threshold, and performs the following processing as shown in fig. 2:
if the flow value is larger than the automatic suppression threshold value and no suppression task is executed, starting the corresponding suppression task according to the protection level of the resource pool IP and setting the suppression task as a waiting period; wherein the white list only sends an alarm notice and does not start suppression, the VIP list starts overseas suppression, the black list gradually upgrades the suppression level (overseas suppression- > inter-network suppression- > whole-network suppression), and the black + list directly starts whole-network suppression;
if the flow value is larger than the automatic pressing threshold value, the pressing task which is being executed exists, and the pressing level is full-network pressing, the pressing task is continuously executed;
if the flow value is larger than the automatic suppression threshold value, and a suppression task which is being executed exists, the suppression level is overseas suppression or inter-network suppression, and the protection level of the resource pool IP is a blacklist, judging whether the suppression task is in a waiting period or an observation period, or the suppression task which is not in the waiting period or the observation period, and upgrading to a higher-level suppression task, wherein the blacklist upgraded to the whole-network suppression is automatically adjusted to be a blacklist; changing the pressing task in the observation period back to the original pressing level, and continuing to execute the pressing task;
the safety protection management system can detect the pressing task at regular time, and if the timing task detects that the pressing task is in a waiting period state, the pressing task is automatically set to be in a pressing state after 15 minutes;
if the timing task detects that the pressing task is executed for more than 2 hours, setting the pressing task to be in an observation period state;
if the timing task detects that the pressing task is executed for 24 hours, calling a decompression interface to close pressing;
and after all the suppression and protection strategies are successfully issued, the safety protection management system informs the user of successful protection in a short message and mail mode.
It should be noted that although the operations of the method of the present invention have been described in the above embodiments and the accompanying drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the operations shown must be performed, to achieve the desired results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
For a clearer explanation of the above method for implementing automatic throttling and protection based on network device logs, a specific embodiment is described below, however, it should be noted that the embodiment is only for better explaining the present invention and is not to be construed as an undue limitation to the present invention.
The implementation scenario one is as follows:
1. the Guizhou resource pool receives the large-flow alarm and attack logs as follows:
Aug 8 04:31:14 2021-08-08 04:31:02 10.255.156.6％％01SEC/5/ATCKDF(l):log_type＝ip_attack device_ip＝10.255.156.6device_type＝CLEAN direction＝inbound zone_id＝11zone_name＝CLEAN zone_ip＝175.6.107.242start_time_alert＝"2021-08-0804:27:17"start_time_attack＝""end_time＝""duration＝214attack_type＝25protocol＝0port＝0attack_status＝ALERT drop_packets＝0drop_kbits＝0attacker＝attacker_pps＝attacker_kbps＝current_attack_flow＝13223860limit_attack_value＝5242880attack_unit＝"kbps"max_drop_kbps＝0max_drop_pps＝0
Aug 8 04:31:16 2021-08-08 04:31:0510.255.156.6％％01SEC/5/ATCKDF(l):log_type＝ip_flow time＝"2021-08-08 04:31:05"device_ip＝10.255.156.6device_type＝CLEAN direction＝inbound zone_id＝11zone_name＝CLEAN zone_ip＝175.6.107.242 biz_id＝1is_deszone＝false is_ipLocation＝false ipLocation_id＝0total_pps＝3321076total_kbps＝132238560tcp_pps＝7tcp_kbps＝13tcpfrag_pps＝0 tcpfrag_kbps＝0 udp_pps＝3320584sip_invite_pps＝0 sip_invite_kbps＝0 tcp_increase_con＝2udp_increase_con＝0icmp_increase_con＝0other_increase_con＝0tcp_concur_con＝9 udp_concur_con＝0 icmp_concur_con＝0other_concur_con＝0 total_average_pps＝1541678total_average_kbps＝6135683
the method comprises the steps that a front-end processor server of a Guizhou resource pool is configured with a corresponding log acquisition component, syslog alarm logs and flow logs are received, a corresponding flash (log collection system provided by Cloudera) process is started to process the logs, jar packets of the logs are pre-processed in the flash in a self-defined mode, key field information in the logs is analyzed primarily through technologies of conversion of regular expressions and key values, key information such as protection IP (Internet protocol), alarm starting time, alarm ending time and flow size is mainly acquired, useless fields are deleted, logs with low flow are discarded, overload and processing delay caused by large data volume are prevented and are sent to a safety protection management system.
2. The Flume process forwards the processed alarm and traffic information to a safety protection management system through kafka (a high-throughput distributed publish-subscribe message system); the safety protection management system receives the processed alarm and flow log, judges the protection level of the alarm IP by matching with various information of a resource pool IP pre-configured in the system, stores the alarm and flow information into an elastic search (a search and analysis engine based on Lucene) index library, and displays the alarm and flow information on a front-end page for a user to inquire in real time.
3. Inquiring the IP which is not recorded in the Guizhou resource pool: 175.6.107.242, because the IP traffic attack is large, the IP is automatically logged into the Guizhou resource pool, and the protection level is set as a blacklist. And the alarm threshold value and the suppression threshold value of the resource pool in the Guizhou are inquired to be 3G and 5G respectively.
4. The flow attack value of the resource pool IP is larger than a suppression threshold value, no suppression task exists in the library during execution, the protection level of the IP is a blacklist, and the level of starting suppression is overseas suppression; and calling and starting an overseas pressing interface, executing a pressing task, setting the state of the pressing task as a waiting period state, and automatically entering a pressing state within 15 minutes.
5. And after the overseas pressing is successfully started, the system informs the user of the related information of successful protection.
And a second implementation scenario:
1. the suzhou resource pool receives the large-flow alarm and attack logs as follows:
Aug 18 09:10:08 2021-08-18 09:10:08 180.100.214.103％％01SEC/5/ATCKDF(l):log_type＝ip_attackdevice_ip＝222.93.107.180 device_type＝CLEAN zone_id＝14zone_name＝DETECT zone_ip＝180.100.214.103start_time_alert＝"2021-03-18 09:10:08"start_time_attack＝"2021-03-18 09:10:08"end_time＝""duration＝196 attack_type＝11 protocol＝0 port＝0attack_status＝ALERT drop_packets＝0drop_kbits＝0attacker＝attacker_pps＝attacker_kbps＝current_attack_flow＝4885510limit_attack_value＝2188250attack_unit＝"kbps"max_drop_kbps＝0max_drop_pps＝0
Aug 8 04:31:16 2021-08-08 04:31:0510.255.156.6％％01SEC/5/ATCKDF(l):log_type＝ip_flow time＝"2021-08-08 04:31:05"device_ip＝222.93.107.180device_type＝CLEAN direction＝inbound zone_id＝11zone_name＝CLEAN zone_ip＝180.100.214.103 biz_id＝1is_deszone＝false is_ipLocation＝false ipLocation_id＝0total_pps＝6548237total_kbps＝258205420tcp_pps＝7tcp_kbps＝13tcpfrag_pps＝0 tcpfrag_kbps＝0 udp_pps＝3320584sip_invite_pps＝0 sip_invite_kbps＝0 tcp_increase_con＝2udp_increase_con＝0icmp_increase_con＝0other_increase_con＝0tcp_concur_con＝9 udp_concur_con＝0 icmp_concur_con＝0other_concur_con＝0 total_average_pps＝2589895total_average_kbps＝9528466
the method comprises the steps that a front-end processor server of a Suzhou resource pool is configured with a corresponding log acquisition component, syslog alarm logs and flow logs are received, corresponding flash (log collection system provided by Cloudera) processes are started to process the logs, jar packets of the pre-processed logs are customized in the flash, key field information in the logs is primarily analyzed through technologies of conversion and the like by regular expressions and key values, key information such as protection IP (Internet protocol), alarm starting time, alarm ending time, flow size and the like is mainly acquired, useless fields are deleted, logs with low flow are discarded, overload and processing delay caused by large data volume are prevented and are sent to a safety protection management system.
2. The Flume process forwards the processed alarm and traffic information to a safety protection management system through kafka (a high-throughput distributed publish-subscribe message system); the safety protection management system receives the processed alarm and flow log, judges the protection level of the alarm IP by matching with various information of a resource pool IP pre-configured in the system, stores the alarm and flow information into an elastic search (a search and analysis engine based on Lucene) index library, and displays the alarm and flow information on a front-end page for a user to inquire in real time.
3. Inquiring the recorded IP in the Suzhou resource pool: 180.100.214.103, the alarm threshold and the suppressing threshold of the Suzhou resource pool are 5G and 7G respectively, and the protection level of the IP is blacklist, namely the starting suppressing level is overseas suppressing- > inter-network suppressing- > full-network suppressing.
4. And inquiring that the flow attack value of the IP of the resource pool is greater than a pressing threshold value, and the library has a pressing task in execution, wherein the pressing level is inter-network pressing, and if the pressing is started for not more than 2 hours, calling an upgrading pressing interface, and adjusting the pressing level to be full-network pressing.
5. After the whole network is successfully upgraded, the system informs the user of the information related to the successful protection, sets the IP protection level as a blacklist, and directly executes the whole network suppression when a large flow attack exists next time.
Based on the same invention concept, the invention also provides a device for realizing automatic pressing and protection based on the network equipment logs. The implementation of the device can be referred to the implementation of the method, and repeated details are not repeated. The term "module," as used below, may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 3 is a schematic structural diagram of an apparatus for implementing automatic throttling and protection based on a network device log according to an embodiment of the present invention. As shown in fig. 3, the apparatus includes:
the network equipment log acquisition module 101 is used for respectively acquiring the log information of the network equipment through the front-end processor servers configured with the resource pools, and carrying out classification and pretreatment;
respectively acquiring network equipment logs by a front-end processor server configured with each resource pool, dividing the acquired network equipment log information into alarm log information and flow log information, dividing the alarm log information into real-time alarm log information and historical alarm log information by analyzing alarm starting and ending time in the alarm log information, and matching the flow log information with the real-time alarm log information through a resource pool IP (Internet protocol);
and deleting useless field information in the collected network equipment log information, and discarding logs lower than a flow basic threshold value.
The alarm data storage and display module 102 is used for analyzing and warehousing the alarm and flow information which is received by the security protection management system after the preprocessing, and dynamically presenting the alarm and flow information in real time;
and the safety protection management system receives the preprocessed alarm and flow information, judges the protection level of the alarm IP by matching with various information of the pre-configured resource pool IP, stores the alarm and flow information in a warehouse and displays the alarm and flow information on a front-end page.
The automatic pressing and protecting module 103 is used for the safety protection management system to execute an automatic pressing logic according to the acquired resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold and the resource pool automatic pressing threshold, judge whether a real-time pressing task exists at present, press whether the pressing is in a waiting period or an observation period, and further implement or adjust a corresponding protection strategy and issue the protection strategy;
the safety protection management system acquires resource pool IP alarm and flow information, the protection level of the resource pool IP, a resource pool alarm notification threshold value and a resource pool automatic suppression threshold value; the protection level of the resource pool IP comprises a white list, a VIP list, a black list and a black + list, wherein the white list only sends an alarm notice and does not start pressing, the VIP list starts overseas pressing, the black list gradually upgrades the pressing level, the overseas pressing level is greater than the inter-network pressing level, the inter-network pressing level is greater than the whole-network pressing level, and the black + list directly starts the whole-network pressing;
if the flow value is larger than the automatic suppression threshold value and no suppression task is executed, starting the corresponding suppression task according to the protection level of the resource pool IP and setting the suppression task as a waiting period;
if the flow value is larger than the automatic pressing threshold value, the pressing task which is being executed exists, and the pressing level is full-network pressing, the pressing task is continuously executed;
if the flow value is larger than the automatic suppression threshold value, and a suppression task which is being executed exists, the suppression level is overseas suppression or inter-network suppression, and the protection level of the resource pool IP is a blacklist, judging whether the suppression task is in a waiting period or an observation period, or the suppression task which is not in the waiting period or the observation period, and upgrading to a higher-level suppression task, wherein the blacklist upgraded to the whole-network suppression is automatically adjusted to be a blacklist; changing the pressing task in the observation period back to the original pressing level, and continuing to execute the pressing task;
if the timing task detects that the pressing task is in a waiting period state, the pressing task is automatically set to be in a pressing state after 15 minutes;
if the timing task detects that the pressing task is executed for more than 2 hours, setting the pressing task to be in an observation period state;
if the timing task detects that the pressing task is executed for 24 hours, calling a decompression interface to close pressing;
and after all the suppression and protection strategies are successfully issued, the safety protection management system informs the user of successful protection in a short message and mail mode.
It should be noted that although several modules of the apparatus implementing automatic throttling and guarding based on network device logs are mentioned in the above detailed description, such partitioning is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the modules described above may be embodied in one module according to embodiments of the invention. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.
Based on the aforementioned inventive concept, as shown in fig. 4, the present invention further provides a computer device 200, which includes a memory 210, a processor 220, and a computer program 230 stored in the memory 210 and executable on the processor 220, wherein the processor 220 implements the aforementioned method for implementing automatic throttling and protection based on the network device log when executing the computer program 230.
Based on the foregoing inventive concept, the present invention further provides a computer-readable storage medium storing a computer program for executing the foregoing method for implementing automatic throttling and protection based on network device logs.
The method and the device for realizing automatic suppression and protection based on the network equipment log flexibly set the corresponding suppression or protection rules by configuring different protection levels of the alarm IP, and can dynamically upgrade the suppression and protection levels of the IP to be protected according to the attack flow, thereby reducing the maintenance difficulty of operation and maintenance managers and improving the accuracy and the scene adaptability of a network safety protection system.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
The limitation of the protection scope of the present invention is understood by those skilled in the art, and various modifications or changes which can be made by those skilled in the art without inventive efforts based on the technical solution of the present invention are still within the protection scope of the present invention.
Claims (10)
1. A method for realizing automatic suppression and protection based on network device logs is characterized by comprising the following steps:
respectively acquiring network equipment log information by a front-end processor server configured with each resource pool, and classifying and preprocessing the network equipment log information;
the safety protection management system receives the preprocessed alarm and flow information, analyzes and stores the information in a warehouse, and dynamically presents the information in real time;
the safety protection management system executes automatic suppression logic according to the acquired resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold and the resource pool automatic suppression threshold, judges whether a real-time suppression task exists at present, suppresses whether the real-time suppression task is in a waiting period or an observation period, and then implements or adjusts a corresponding protection strategy and issues the protection strategy.
2. The method of claim 1, wherein the method for implementing automatic suppression and protection based on the network device log comprises the steps of respectively collecting the network device log information by a front-end processor server configured with each resource pool, classifying and preprocessing the network device log information, and including:
respectively acquiring network equipment logs by a front-end processor server configured with each resource pool, dividing the acquired network equipment log information into alarm log information and flow log information, dividing the alarm log information into real-time alarm log information and historical alarm log information by analyzing alarm starting and ending time in the alarm log information, and matching the flow log information with the real-time alarm log information through a resource pool IP (Internet protocol);
and deleting useless field information in the collected network equipment log information, and discarding logs lower than a flow basic threshold value.
3. The method of claim 1, wherein the safety protection management system receives the preprocessed alarm and flow information, analyzes and stores the information in a warehouse, and dynamically presents the information in real time, and the method comprises:
and the safety protection management system receives the preprocessed alarm and flow information, judges the protection level of the alarm IP by matching with various information of the pre-configured resource pool IP, stores the alarm and flow information in a warehouse and displays the alarm and flow information on a front-end page.
4. The method of claim 1, wherein the safety protection management system executes an automatic pressing logic according to the acquired resource pool IP alarm and traffic information, the protection level of the resource pool IP, the resource pool alarm notification threshold, and the resource pool automatic pressing threshold, determines whether a real-time pressing task exists at present, and whether pressing is in a waiting period or an observation period, and then implements or adjusts a corresponding protection policy and issues the protection policy, including:
the safety protection management system acquires resource pool IP alarm and flow information, the protection level of the resource pool IP, a resource pool alarm notification threshold value and a resource pool automatic suppression threshold value; the protection level of the resource pool IP comprises a white list, a VIP list, a black list and a black + list, wherein the white list only sends an alarm notice and does not start pressing, the VIP list starts overseas pressing, the black list gradually upgrades the pressing level, the overseas pressing level is greater than the inter-network pressing level, the inter-network pressing level is greater than the whole-network pressing level, and the black + list directly starts the whole-network pressing;
if the flow value is larger than the automatic suppression threshold value and no suppression task is executed, starting the corresponding suppression task according to the protection level of the resource pool IP and setting the suppression task as a waiting period;
if the flow value is larger than the automatic pressing threshold value, the pressing task which is being executed exists, and the pressing level is full-network pressing, the pressing task is continuously executed;
if the flow value is larger than the automatic suppression threshold value, and a suppression task which is being executed exists, the suppression level is overseas suppression or inter-network suppression, and the protection level of the resource pool IP is a blacklist, judging whether the suppression task is in a waiting period or an observation period, or the suppression task which is not in the waiting period or the observation period, and upgrading to a higher-level suppression task, wherein the blacklist upgraded to the whole-network suppression is automatically adjusted to be a blacklist; changing the pressing task in the observation period back to the original pressing level, and continuing to execute the pressing task;
if the timing task detects that the pressing task is in a waiting period state, the pressing task is automatically set to be in a pressing state after 15 minutes;
if the timing task detects that the pressing task is executed for more than 2 hours, setting the pressing task to be in an observation period state;
if the timing task detects that the pressing task is executed for 24 hours, calling a decompression interface to close pressing;
and after all the suppression and protection strategies are successfully issued, the safety protection management system informs the user of successful protection in a short message and mail mode.
5. An apparatus for implementing automatic suppression and protection based on network device log, the apparatus comprising:
the network equipment log acquisition module is used for respectively acquiring the log information of the network equipment through the front-end processor server configured with each resource pool, and classifying and preprocessing the log information;
the alarm data storage and display module is used for analyzing and warehousing the alarm and flow information which is received by the safety protection management system after the preprocessing, and dynamically presenting the alarm and flow information in real time;
and the automatic pressing and protecting module is used for executing automatic pressing logic by the safety protection management system according to the acquired resource pool IP alarm and flow information, the protection level of the resource pool IP, the resource pool alarm notification threshold and the resource pool automatic pressing threshold, judging whether a real-time pressing task exists at present, pressing whether the pressing task is in a waiting period or an observation period, and further implementing or adjusting a corresponding protection strategy and issuing the protection strategy.
6. The method for implementing automatic throttling and protection based on the network device log according to claim 5, wherein the network device log collection module is specifically configured to:
respectively acquiring network equipment logs by a front-end processor server configured with each resource pool, dividing the acquired network equipment log information into alarm log information and flow log information, dividing the alarm log information into real-time alarm log information and historical alarm log information by analyzing alarm starting and ending time in the alarm log information, and matching the flow log information with the real-time alarm log information through a resource pool IP (Internet protocol);
and deleting useless field information in the collected network equipment log information, and discarding logs lower than a flow basic threshold value.
7. The method for implementing automatic throttling and protection based on the network device log according to claim 5, wherein the alarm data storage and presentation module is specifically configured to:
and the safety protection management system receives the preprocessed alarm and flow information, judges the protection level of the alarm IP by matching with various information of the pre-configured resource pool IP, stores the alarm and flow information in a warehouse and displays the alarm and flow information on a front-end page.
8. The method for implementing automatic throttling and guarding based on network device logs according to claim 5, wherein the automatic throttling and guarding module is specifically configured to:
the safety protection management system acquires resource pool IP alarm and flow information, the protection level of the resource pool IP, a resource pool alarm notification threshold value and a resource pool automatic suppression threshold value; the protection level of the resource pool IP comprises a white list, a VIP list, a black list and a black + list, wherein the white list only sends an alarm notice and does not start pressing, the VIP list starts overseas pressing, the black list gradually upgrades the pressing level, the overseas pressing level is greater than the inter-network pressing level, the inter-network pressing level is greater than the whole-network pressing level, and the black + list directly starts the whole-network pressing;
if the flow value is larger than the automatic suppression threshold value and no suppression task is executed, starting the corresponding suppression task according to the protection level of the resource pool IP and setting the suppression task as a waiting period;
if the flow value is larger than the automatic pressing threshold value, the pressing task which is being executed exists, and the pressing level is full-network pressing, the pressing task is continuously executed;
if the flow value is larger than the automatic suppression threshold value, and a suppression task which is being executed exists, the suppression level is overseas suppression or inter-network suppression, and the protection level of the resource pool IP is a blacklist, judging whether the suppression task is in a waiting period or an observation period, or the suppression task which is not in the waiting period or the observation period, and upgrading to a higher-level suppression task, wherein the blacklist upgraded to the whole-network suppression is automatically adjusted to be a blacklist; changing the pressing task in the observation period back to the original pressing level, and continuing to execute the pressing task;
if the timing task detects that the pressing task is in a waiting period state, the pressing task is automatically set to be in a pressing state after 15 minutes;
if the timing task detects that the pressing task is executed for more than 2 hours, setting the pressing task to be in an observation period state;
if the timing task detects that the pressing task is executed for 24 hours, calling a decompression interface to close pressing;
and after all the suppression and protection strategies are successfully issued, the safety protection management system informs the user of successful protection in a short message and mail mode.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1-4 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for performing the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111457582.2A CN114244584B (en) | 2021-12-02 | 2021-12-02 | Method and device for realizing automatic suppression and protection based on network equipment log |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111457582.2A CN114244584B (en) | 2021-12-02 | 2021-12-02 | Method and device for realizing automatic suppression and protection based on network equipment log |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114244584A true CN114244584A (en) | 2022-03-25 |
| CN114244584B CN114244584B (en) | 2023-07-25 |
Family
ID=80752713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111457582.2A Active CN114244584B (en) | 2021-12-02 | 2021-12-02 | Method and device for realizing automatic suppression and protection based on network equipment log |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114244584B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140269319A1 (en) * | 2013-03-15 | 2014-09-18 | International Business Machines Corporation | Network per-flow rate limiting |
| CN106161333A (en) * | 2015-03-24 | 2016-11-23 | 华为技术有限公司 | DDOS attack means of defence based on SDN, Apparatus and system |
| CN107743118A (en) * | 2017-09-25 | 2018-02-27 | 北京奇安信科技有限公司 | A kind of stagewise network safety protection method and device |
| CN111224924A (en) * | 2018-11-27 | 2020-06-02 | 北京金山云网络技术有限公司 | Traffic processing method and device, electronic equipment and storage medium |
-
2021
- 2021-12-02 CN CN202111457582.2A patent/CN114244584B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140269319A1 (en) * | 2013-03-15 | 2014-09-18 | International Business Machines Corporation | Network per-flow rate limiting |
| CN106161333A (en) * | 2015-03-24 | 2016-11-23 | 华为技术有限公司 | DDOS attack means of defence based on SDN, Apparatus and system |
| CN107743118A (en) * | 2017-09-25 | 2018-02-27 | 北京奇安信科技有限公司 | A kind of stagewise network safety protection method and device |
| CN111224924A (en) * | 2018-11-27 | 2020-06-02 | 北京金山云网络技术有限公司 | Traffic processing method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 黄谋江: "抗拒绝服务攻击系统中的流量控制技术研究与实现", 《中国优秀硕士学位论文全文数据库（电子期刊）》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114244584B (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10432650B2 (en) | System and method to protect a webserver against application exploits and attacks | |
| CN109688097B (en) | Website protection method, website protection device, website protection equipment and storage medium | |
| CN112671807B (en) | Threat processing method, threat processing device, electronic equipment and computer readable storage medium | |
| US11621971B2 (en) | Low-complexity detection of potential network anomalies using intermediate-stage processing | |
| CN112783725B (en) | Index collection method and device | |
| CN112350854B (en) | Flow fault positioning method, device, equipment and storage medium | |
| CN110351237B (en) | Honeypot method and device for numerical control machine tool | |
| CN112788035B (en) | Network attack warning method of UPF terminal under 5G and terminal | |
| CA2791317C (en) | Application level admission overload control | |
| CN111651754A (en) | Intrusion detection method and device, storage medium and electronic device | |
| CN112929376A (en) | Flow data processing method and device, computer equipment and storage medium | |
| CN101409654A (en) | Method for processing SNMP information in network management system | |
| CN111813627A (en) | Application auditing method, device, terminal, system and readable storage medium | |
| CN110806924B (en) | Network processing method and device based on CPU occupancy rate | |
| CN114244584A (en) | Method and device for realizing automatic suppression and protection based on network equipment log | |
| CN111309442B (en) | Method, device, system, medium and equipment for adjusting number of micro-service containers | |
| CN107454043A (en) | The monitoring method and device of a kind of network attack | |
| CN109327433B (en) | Threat perception method and system based on operation scene analysis | |
| CN111258712B (en) | Method and system for protecting safety of virtual machine under virtual platform network isolation | |
| CN106612241B (en) | Service control method and device | |
| CN114079619A (en) | Port flow sampling method and device | |
| CN112994934A (en) | Data interaction method, device and system | |
| CN112134845A (en) | Rejection service system | |
| CN113127853A (en) | Method and device for safely processing virtual machine file | |
| CN112311765B (en) | Message detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |