CN112637171A - Data traffic processing method, device, equipment, system and storage medium - Google Patents
Data traffic processing method, device, equipment, system and storage medium Download PDFInfo
- Publication number
- CN112637171A CN112637171A CN202011481469.3A CN202011481469A CN112637171A CN 112637171 A CN112637171 A CN 112637171A CN 202011481469 A CN202011481469 A CN 202011481469A CN 112637171 A CN112637171 A CN 112637171A
- Authority
- CN
- China
- Prior art keywords
- data request
- attribute information
- marked
- preset
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 12
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000001514 detection method Methods 0.000 claims description 55
- 230000000903 blocking effect Effects 0.000 claims description 15
- 238000004458 analytical method Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 11
- 230000015654 memory Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012550 audit Methods 0.000 description 2
- ZPUCINDJVBIVPJ-LJISPDSOSA-N cocaine Chemical compound O([C@H]1C[C@@H]2CC[C@@H](N2C)[C@H]1C(=O)OC)C(=O)C1=CC=CC=C1 ZPUCINDJVBIVPJ-LJISPDSOSA-N 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000013486 operation strategy Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000007789 sealing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/102—Entity profiles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application provides a data traffic processing method, a device, equipment, a system and a storage medium, wherein the method comprises the following steps: when a data request sent by a terminal is received, analyzing the data request to obtain attribute information of the data request, wherein the attribute information at least comprises a network address of the data request and an equipment identifier of the terminal; judging whether the attribute information is marked as a white list by a preset database; if the attribute information is marked as a white list by a preset database, sending the data request to a specified application server; and if the attribute information is not marked as a white list by the preset database, limiting the flow direction of the data request based on preset marking information. According to the method and the device, the data requests are screened based on the preset database, and the conditions of request delay, packet loss, request failure and the like caused by data flow peak are effectively reduced.
Description
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a data traffic processing method, apparatus, device, system, and storage medium.
Background
Conventional WAF (Web Application Firewall) solutions typically decode all Web requests uniformly before connecting the WAF directly to the Application server. And performing text matching on the decoded request packet through a regular rule carried on the WAF, and if the matching rule is hit, directly blocking the request, and if the request is not hit, forwarding the request to an application server.
In the existing firewall technology, after a load balancing server receives a request, a request packet of a user is directly analyzed, and malicious attack flow is matched according to rules to block the request packet. This traditional firewall approach has several problems:
1. affecting the business performance problem. Analyzing the matching rule in real time for each request of the user consumes a large amount of computing performance, and when the traffic of the normal user is high, the conditions of request delay, packet loss, request failure and the like are caused.
2. Malicious attacks of non-WEB requests cannot be detected. With the continuous development of the hacker technology, more and more attackers try to attack the server basic middleware services such as redis, mysql and the like first, and continue to attack the WEB application service by taking the server basic middleware services as a springboard. The traditional WAF is carried on a WEB gateway such as nginx (a high-performance HTTP and reverse proxy WEB server) and the like, can only detect WEB requests, and cannot defend against the attack.
Disclosure of Invention
An object of the embodiments of the present application is to provide a data traffic processing method, apparatus, device, system, and storage medium, which are used to implement screening of data requests based on a preset database, and effectively reduce situations such as request delay, packet loss, and request failure when a data traffic peak occurs.
A first aspect of the embodiments of the present application provides a data traffic processing method, including: when a data request sent by a terminal is received, analyzing the data request to obtain attribute information of the data request; judging whether the attribute information is marked as a white list by a preset database; if the attribute information is marked as a white list by a preset database, sending the data request to a specified application server; and if the attribute information is not marked as a white list by the preset database, limiting the flow direction of the data request based on preset marking information.
In an embodiment, if the attribute information is not marked as a white list by the preset database, the method for restricting the flow direction of the data request based on preset marking information further includes: if the attribute information is not marked as a white list by the preset database, judging whether the attribute information is marked as a black list by the preset database; and if the attribute information is marked as a blacklist by a preset database, blocking the flow direction of the data request and recording an attack log.
In an embodiment, the attribute information includes a network address of the data request and a device identifier of the terminal.
In an embodiment, the attribute information further includes: browser information of the data request; if the attribute information is not marked as a white list by the preset database, limiting the flow direction of the data request based on preset marking information, further comprising: if the attribute information is not marked as a blacklist by the preset database, judging whether the network address and the browser information are marked by a preset crawler library or not; if the network address and the browser information are marked by the preset crawler library, extracting a request threshold value of the data request from the preset crawler library, and limiting the flow of the data request based on the request threshold value.
In an embodiment, if the attribute information is not marked as a white list by the preset database, the method for restricting the flow direction of the data request based on preset marking information further includes: if the attribute information is not marked as a blacklist by the preset database, judging whether the data request hits a preset detection rule or not; and if the data request hits the preset detection rule, blocking the flow direction of the data request, and recording an attack log, otherwise, sending the data request to the application server.
In one embodiment, the method further comprises: and receiving blacklist mark information sent by a log server, and establishing the preset database according to the blacklist mark information.
A second aspect of the embodiments of the present application provides a data traffic processing apparatus, including: the analysis module is used for analyzing the data request to obtain attribute information of the data request when receiving the data request sent by the terminal, wherein the attribute information at least comprises a network address of the data request and an equipment identifier of the terminal; the judging module is used for judging whether the attribute information is marked as a white list by a preset database; the sending module is used for sending the data request to a specified application server if the attribute information is marked as a white list by a preset database; and the limiting module is used for limiting the flow direction of the data request based on preset marking information if the attribute information is not marked as a white list by the preset database.
In one embodiment, the restriction module is configured to: if the attribute information is not marked as a white list by the preset database, judging whether the attribute information is marked as a black list by the preset database; and if the attribute information is marked as a blacklist by a preset database, blocking the flow direction of the data request and recording an attack log.
In an embodiment, the attribute information includes a network address of the data request and a device identifier of the terminal.
In an embodiment, the attribute information further includes: browser information of the data request; the restriction module is further configured to: if the attribute information is not marked as a blacklist by the preset database, judging whether the network address and the browser information are marked by a preset crawler library or not; if the network address and the browser information are marked by the preset crawler library, extracting a request threshold value of the data request from the preset crawler library, and limiting the flow of the data request based on the request threshold value.
In one embodiment, the restriction module is further configured to: if the attribute information is not marked as a blacklist by the preset database, judging whether the data request hits a preset detection rule or not; and if the data request hits the preset detection rule, blocking the flow direction of the data request, and recording an attack log, otherwise, sending the data request to the application server.
In one embodiment, the method further comprises: and the receiving module is used for receiving the blacklist marking information sent by the log server and establishing the preset database according to the blacklist marking information.
A third aspect of the embodiments of the present application provides a firewall system, including: the intrusion detection cluster comprises a plurality of detection nodes and is used for receiving a data request and generating an attack event log after detecting the data request; the log server is connected with the intrusion detection cluster and used for receiving the attack event log and generating blacklist marking information of the attack event log; the load balancing cluster comprises a plurality of load balancing nodes and is used for receiving the blacklist marking information, establishing a preset database, and executing the method of the first aspect and any embodiment of the application to process the data request flow.
A fourth aspect of the embodiments of the present application provides an electronic device, including: a memory to store a computer program; the processor is configured to perform the method of the first aspect and any embodiment thereof of the embodiments of the present application to process the data request traffic.
A fifth aspect of embodiments of the present application provides a non-transitory electronic device-readable storage medium, including: a program which, when run by an electronic device, causes the electronic device to perform the method of the first aspect of an embodiment of the present application and any embodiment thereof.
According to the data traffic processing method, the data traffic processing device, the data traffic processing equipment, the data traffic processing system and the storage medium, when a data request sent by a terminal is received, the data request is analyzed to obtain attribute information of the data request, the data request is screened based on marking information of the attribute information in a preset database, the data request marked as a white list can be directly forwarded to an application server appointed by the data request, and therefore the terminal can obtain return resources. Compared with the traditional mode of carrying out rule verification on the data requests on the whole disk, the method can effectively reduce the conditions of request delay, packet loss, request failure and the like when the data traffic is in a peak.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an applied firewall system based on intrusion detection according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating a data traffic processing method according to an embodiment of the present application;
fig. 4 is a schematic flow chart illustrating a data traffic processing method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data traffic processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the present application, the terms "first," "second," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
As shown in fig. 1, the present embodiment provides an electronic apparatus 1 including: at least one processor 11 and a memory 12, one processor being exemplified in fig. 1. The processor 11 and the memory 12 are connected by the bus 10, and the memory 12 stores instructions executable by the processor 11, and the instructions are executed by the processor 11, so that the electronic device 1 can execute all or part of the flow of the method in the embodiments described below to process the data request traffic.
In an embodiment, the electronic device 1 may be a mobile phone, a notebook computer, a desktop computer, or the like.
Please refer to fig. 2, which is a schematic view of a scenario of an application firewall system based on intrusion detection according to an embodiment of the present application, and mainly includes: the system comprises three modules of an intrusion detection cluster 21, a log server 22 and a load balancing cluster 23, wherein:
the intrusion detection cluster 21, which may be an IDS (intrusion detection system) cluster, includes a plurality of detection nodes (i.e., IDS node 1 to IDS node N, where N is a positive integer) for receiving a data request and generating an attack event log after detecting the data request. The specific execution steps can be as follows:
step 1: and leading all user flow images to an IDS cluster server network card through a flow image port of the machine room switch.
Step 2: the IDS cluster decrypts the imported mirror traffic via https (Hyper Text Transfer Protocol over secure session Layer) certificate.
And 3, carrying out rule detection on the decrypted request by using an intrusion detection model of snort (a piece of intrusion detection rule matching model software).
And 4, step 4: the request of the hit rule forms an attack event log and is transmitted to a log storage center through rsyslog (a piece of log pushing processing software) software.
The log server 22 may include two parts, namely a log analysis engine and a log storage center (elastic search), the log storage center is connected to the intrusion detection cluster 21 and is configured to receive the attack event log, and the log analysis engine is configured to generate blacklist marking information of the attack event log. The specific implementation steps of the log analysis engine may be as follows:
step 1: polling the Elasticsearch gets the latest IDS attack event log.
Step 2: and matching the rule ID hit by the attack event to the rule operation strategy of the back-end record. The rule policy contains whether to block the IP, the equipment ID and the blocking time. Different sealing durations can be given according to different IP regions and equipment environments of malicious attackers.
And 3, step 3: and pushing the forbidden IP, the equipment ID and the forbidden time length information to all WAF nodes through an API (application programming interface) of the WAF cluster nodes, adding the corresponding IP and equipment ID information into a blacklist cache by the WAF nodes, and directly blocking the blacklist detection link when an attacker accesses the blacklist detection link.
The load balancing cluster 23 may be a WAF (Web Application Firewall, website Application level intrusion prevention system) load balancing cluster 23, and includes a plurality of load balancing nodes (i.e., WAF nodes 1 to WAF node M, where M is a positive integer), where the WAF nodes may be implemented by the electronic device 1, and are configured to receive blacklist marking information, establish a preset database, and execute all or part of processes of the method in the following embodiments to process data request traffic.
And analyzing the intrusion detection based on the machine room mirror flow to form an attack event log, analyzing the attack event by using a log analysis engine, and blocking malicious attacks in real time through distributed WAF nodes.
In this embodiment, when data request traffic of a user enters, the WAF node that enters load balancing first enters normal detection, and the detected data request traffic is forwarded to the backend application server 24 to respond. The application server 24 may include a plurality of K, for example, where K is a positive integer. The server audit log generated by the application server 24 may be transmitted to the log storage center via rsyslog.
At the same time, all data request traffic for the user is passed to the IDS cluster through the mirroring operation. The IDS cluster carries detection rules of a highly complex model, and detection delay does not affect the performance of a user request due to mirroring operation. The user request log, the IDS log and the audit log are uniformly stored in a log storage center, a log analysis engine is used for polling and scheduling, and the IP and the equipment ID of the malicious attacker are transmitted to the WAF node to block the malicious attacker by analyzing log events.
In this embodiment, after all the WAF nodes are deployed in the load balancing manner, the data requests of the users are distributed to the WAF nodes in a balanced manner. After receiving the request, the WAF node firstly analyzes the request packet to obtain the real IP of the user, whether the user is the CDN or not, the equipment ID of the user and request host information. And detecting a white list and a black list after the acquisition is finished, directly releasing if the user IP or the equipment ID belongs to the white list, and forwarding the flow to a back-end server. And if the user IP or the equipment ID belongs to the blacklist, blocking the request and returning a user violation page prompt. The method comprises the steps that the flow which does not belong to the black and white list firstly enters a search engine crawler detection and flow limiting module, and the flow limitation is carried out on crawlers which request frequently according to a set threshold strategy. And finally, the user requests to enter WAF rule model detection, the request of the hit rule can be blocked, and an event log is formed and pushed to a log storage center through an rsyslog. The detected request is forwarded to the back-end application server 24 for normal resource return.
In this embodiment, the user data request traffic is transmitted to the IDS server network card through the machine room switch traffic mirror interface, and intrusion detection can be performed on all mirror traffic using a snort rule model. The detected attack events form an attack log which can be pushed to a log storage center by an rsyslog. And a background log analysis engine polls the log storage center in real time to obtain the latest IDS event log, and pushes the IP and equipment ID information of an attacker to all WAF nodes through an API (application programming interface) interface according to the rule triggered by the attack event and a preset rule strategy. The WAF node adds the IP and the equipment ID of the attacker into a blacklist for forbidding, and the attacker can be directly intercepted by the WAF when accessing.
Please refer to fig. 3, which is a data traffic processing method according to an embodiment of the present application, and the method may be executed by the electronic device 1 shown in fig. 1 and may be applied to the load balancing cluster 23 in the firewall system shown in fig. 2 to intercept data access of an attacker. The method comprises the following steps:
step 301: when a data request sent by a terminal is received, the data request is analyzed to obtain attribute information of the data request, wherein the attribute information at least comprises a network address of the data request and a device identifier of the terminal.
In this step, a data request sent by the terminal is received first, and the data request traffic is sent to the idle WAF node through the load balancing cluster 23. Performing packet body analysis on the data request by the WAF node to obtain attribute information of the data request, wherein the attribute information at least comprises: the network address of the data request (real IP) and the user equipment ID (device identification of the terminal).
Step 302: and judging whether the attribute information is marked as a white list by a preset database. If yes, go to step 303, otherwise go to step 304.
In this step, the white list refers to terminal information that can be free from detection, and the data request sent by the terminal in the white list is trusted and does not need attack detection. The inspection-free terminal information fishing village can be put into a preset database in advance. The white list can mark the network address IP and the equipment ID of the inspection-free terminal, so that whether the network address IP and the equipment ID of the requester are in the white list of the WAF system preset database can be checked to judge whether the data request needs to be subjected to attack detection.
Step 303: sending a data request to the designated application server 24.
In this step, if the attribute information is marked as a white list by the preset database, that is, in the white list, the data request is directly released without performing attack detection on the data request, and the data request is forwarded to the back-end application server 24. Such as application server 24, is specified by the HOST (requesting WEB server domain name address) of the data request.
Step 304: and limiting the flow direction of the data request based on the preset mark information.
In this step, if the attribute information is not marked as a white list by the preset database, it indicates that the source of the data request may be an attacker, and for information security, the data request is processed by using a corresponding restriction policy, such as limiting the flow direction and flow rate of the data request.
According to the data traffic processing method, when a data request sent by a terminal is received, the data request is analyzed to obtain attribute information of the data request, the data request is screened based on the marking information of the attribute information in the preset database, and the data request marked as a white list can be directly forwarded to the application server 24 specified by the data request, so that the terminal can obtain a return resource. Compared with the traditional mode of carrying out rule verification on the data requests on the whole disk, the method can effectively reduce the conditions of request delay, packet loss, request failure and the like when the data traffic is in a peak.
Please refer to fig. 4, which is a data traffic processing method according to an embodiment of the present application, and the method may be executed by the electronic device 1 shown in fig. 1 and may be applied to the load balancing cluster 23 in the firewall system shown in fig. 2 to intercept data access of an attacker. The method comprises the following steps:
step 401: when a data request sent by a terminal is received, the data request is analyzed to obtain attribute information of the data request, wherein the attribute information at least comprises a network address of the data request and a device identifier of the terminal. See the description of step 301 in the above embodiments for details.
Step 402: and judging whether the attribute information is marked as a white list by a preset database. If yes, go to step 409, otherwise go to step 403. See the description of step 302 in the above embodiments for details.
Step 403: and judging whether the attribute information is marked as a blacklist by the preset database. If yes, go to step 404, otherwise go to step 405.
In this step, the blacklist refers to terminal information that is marked as an attacker in advance, and the terminals may have an attack history record and may be automatically updated into the blacklist, and may also mark the network address IP and the device ID of the attacker terminal, so that if the network address and the device identifier are not marked as a white list by the preset database, it is indicated that the data request sent by the terminal has a potential safety hazard, and it is required to perform security detection, and it may be checked whether the network address IP and the device ID of the terminal are in the blacklist of the preset database, and step 407 is entered, otherwise, step 404 is entered.
In an embodiment, before step 403, the method may further include: and receiving blacklist marking information sent by the log analysis engine, and establishing a preset database according to the blacklist marking information.
In this step, the blacklist marking information sent by the log analysis engine is received, so as to update the preset database in real time. For example, the WAF cluster node caches blacklist marking information sent by the log analysis engine to a preset database in real time through an API interface of the WAF cluster node, where the blacklist marking information may include at least information such as a forbidden network address IP, a device ID, and a forbidden duration.
Step 404: the attribute information further includes: browser information for the data request. And judging whether the network address and the browser information are marked by a preset crawler library. If yes, go to step 405, otherwise, go to step 406.
In this step, the attribute information may further include: the browser information USER _ AGENT of the current data request (detailed information of the browser type operated by the HTTP client, and the WEB server can determine the client browser type of the current HTTP request by using the header information). If the network address and the device identifier are not marked as a blacklist by the preset database, the terminal is not a known attacker, but security detection is still required to be carried out on the terminal, so that search engine crawler detection can be carried out on the data request. For example, whether the network address IP and USER _ AGENT fields of the data request are matched with the public USER _ AGENT and crawler IP library of the search engine (i.e., the predetermined crawler library) first, if so, step 405 is executed, otherwise, step 406 is executed.
In an embodiment, the attribute information may further include: the IP of the CDN (Content Delivery Network, i.e., Content Delivery Network) of the data request, the request chain tracking ID, the request protocol version, the browser information USER _ AGENT (detailed information of the browser type operated by the HTTP client, through which the WEB server can determine the client browser type of the current HTTP request), the HOST (domain name address of the requested WEB server), and other information.
Step 405: and extracting a request threshold value of the data request from a preset crawler library, and limiting the flow of the data request based on the request threshold value. Step 406 is then entered.
In this step, if the network address and the browser information are marked by a preset crawler library, which indicates that the data request belongs to a crawler request, and crawlers frequently request, the data request is limited according to a formulated request threshold.
Step 406: and judging whether the data request hits a preset detection rule or not. If yes, go to step 407, otherwise, go to step 408.
In this step, after completing the crawler detection, rule detection may be performed on the data request whose network address and device identifier are not marked as the blacklist by the preset database. The preset detection rule may be a small number of general detection rules that are carried on the WAF node and are used for screening some general attack request features. For example, attack detection can be performed by matching different preset detection rule models built in the WAF system according to HOST information corresponding to the data request.
Step 407: and blocking the flow direction of the data request and recording an attack log.
In this step, if it is determined in step 403 that the attribute information is marked as a blacklist by the preset database, indicating that the terminal sending the data request is an attacker, the data request is directly blocked without forwarding, and an attack log may be recorded and fed back to the log server 22. Moreover, for the data request hit in the preset detection rule in step 406, the data blocking request is performed according to the policy defined by the preset detection rule, and an attack log can be recorded, an alarm is issued, and the like.
Step 408: sending a data request to the designated application server 24.
In this step, if the network address and the device identifier are marked as a white list by the preset database, or if the data request does not hit the preset detection rule, it indicates that the data request is safe, the data request may be forwarded to the back-end application server 24, and the application server 24 is specified by HOST (domain name address of the requested WEB server) information of the data request.
The data flow processing method has the following beneficial effects:
1. the WAF cluster is deployed on all load balancing gateways, and a uniform rule issuing and logic control center is realized, so that the performance of a single-node firewall is not influenced by high concurrent flow. In order to further improve the performance, the WAF node only carries a small number of common preset detection rules. All mirror image flows of the machine room are led into the IDS cluster in real time for detection, and the IDS cluster can carry a large number of rich detection rules. The data request traffic led into the IDS cluster not only contains the decrypted WEB traffic, but also contains the request traffic of other server middleware. For the detected suspicious attacks, the suspicious attacks are transmitted to each WAF node through the log analysis engine to block the attacks, so that the performance problem under the condition of high flow and high concurrency is solved, and the problem of non-WEB flow detection is solved.
2. The background administrator can select any number of preset detection rules from the rule base for collocation aiming at a certain specific domain name, and the matched rule model is transmitted to the WAF node through the API interface of the WAF node. The method for carrying the self-defined rule by the single service application is realized, and the problem of applying the personalized rule under the complex service is solved.
3. And analyzing the crawler flow by the device fingerprint and requesting the USER _ AGENT, and limiting the crawler flow behavior of the search engine.
Please refer to fig. 5, which is a data traffic processing apparatus 500 according to an embodiment of the present application, and the apparatus can be applied to the electronic device 1 shown in fig. 1, and can be applied to the load balancing cluster 23 in the firewall system shown in fig. 2 to intercept data access of an attacker. The device includes: the analysis module 501, the judgment module 502, the sending module 503 and the limitation module 504, the principle relationship of each module is as follows:
the parsing module 501 is configured to parse the data request to obtain attribute information of the data request when the data request sent by the terminal is received, where the attribute information at least includes a network address of the data request and a device identifier of the terminal. See the description of step 301 in the above embodiments for details.
The determining module 502 is configured to determine whether the attribute information is marked as a white list by the preset database. See the description of step 302 in the above embodiments for details.
The sending module 503 is configured to send a data request to the specified application server 24 if the attribute information is marked as a white list by the preset database. See the description of step 303 in the above embodiments for details.
The limiting module 504 is configured to limit a flow direction of the data request based on the preset flag information if the attribute information is not marked as a white list by the preset database. See the description of step 304 in the above embodiments for details.
In one embodiment, the restriction module 504 is configured to: and if the attribute information is not marked as a white list by the preset database, judging whether the attribute information is marked as a black list by the preset database. And if the attribute information is marked as a blacklist by the preset database, blocking the flow direction of the data request and recording an attack log. See the description of steps 403 to 404 in the above embodiments in detail.
In one embodiment, the attribute information includes a network address of the data request and a device identifier of the terminal.
In one embodiment, the attribute information further includes: browser information for the data request. The restriction module 504 is further configured to: and if the attribute information is not marked as a blacklist by the preset database, judging whether the network address and the browser information are marked by the preset crawler library. And if the network address and the browser information are marked by the preset crawler library, extracting a request threshold value of the data request from the preset crawler library, and limiting the flow of the data request based on the request threshold value. See the above embodiments for a detailed description of steps 405 through 406.
In one embodiment, the restriction module 504 is further configured to: and if the attribute information is not marked as a blacklist by the preset database, judging whether the data request hits the preset detection rule or not. If the data request hits the preset detection rule, the flow direction of the data request is blocked, and the attack log is recorded, otherwise, the data request is sent to the application server 24. See the description of step 407 to step 408 in the above embodiments in detail.
In one embodiment, the method further comprises: the receiving module 505 is configured to receive the blacklist marking information sent by the log server 22, and establish a preset database according to the blacklist marking information. See the description of the related methods in the above examples in detail.
For a detailed description of the data traffic processing apparatus 500, please refer to the description of the related method steps in the above embodiments.
An embodiment of the present invention further provides a non-transitory electronic device readable storage medium, including: a program that, when run on an electronic device, causes the electronic device to perform all or part of the procedures of the methods in the above-described embodiments. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like. The storage medium may also comprise a combination of memories of the kind described above.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (11)
1. A method for processing data traffic, comprising:
when a data request sent by a terminal is received, analyzing the data request to obtain attribute information of the data request;
judging whether the attribute information is marked as a white list by a preset database;
if the attribute information is marked as a white list by a preset database, sending the data request to a specified application server;
and if the attribute information is not marked as a white list by the preset database, limiting the flow direction of the data request based on preset marking information.
2. The method of claim 1, wherein if the attribute information is not marked as a white list by the predetermined database, restricting a flow direction of the data request based on predetermined marking information, further comprising:
if the attribute information is not marked as a white list by the preset database, judging whether the attribute information is marked as a black list by the preset database;
and if the attribute information is marked as a blacklist by a preset database, blocking the flow direction of the data request and recording an attack log.
3. The method of claim 1, wherein the attribute information comprises a network address of the data request and a device identification of the terminal.
4. The method according to any one of claim 3, wherein the attribute information further comprises: browser information of the data request;
if the attribute information is not marked as a white list by the preset database, limiting the flow direction of the data request based on preset marking information, further comprising:
if the attribute information is not marked as a blacklist by the preset database, judging whether the network address and the browser information are marked by a preset crawler library or not;
if the network address and the browser information are marked by the preset crawler library, extracting a request threshold value of the data request from the preset crawler library, and limiting the flow of the data request based on the request threshold value.
5. The method of claim 2, wherein if the attribute information is not marked as a white list by the predetermined database, restricting a flow direction of the data request based on predetermined marking information, further comprising:
if the attribute information is not marked as a blacklist by the preset database, judging whether the data request hits a preset detection rule or not;
and if the data request hits the preset detection rule, blocking the flow direction of the data request, and recording an attack log, otherwise, sending the data request to the application server.
6. The method according to any one of claims 1-3, further comprising:
and receiving blacklist mark information sent by a log server, and establishing the preset database according to the blacklist mark information.
7. A data traffic processing apparatus, comprising:
the analysis module is used for analyzing the data request to obtain the attribute information of the data request when receiving the data request sent by the terminal;
the judging module is used for judging whether the attribute information is marked as a white list by a preset database;
the sending module is used for sending the data request to a specified application server if the attribute information is marked as a white list by a preset database;
the limiting module is used for limiting the flow direction of the data request based on preset marking information if the attribute information is not marked as a white list by the preset database;
wherein the attribute information includes a network address of the data request and a device identifier of the terminal.
8. The apparatus of claim 7, wherein the restriction module is configured to:
if the attribute information is not marked as a white list by the preset database, judging whether the attribute information is marked as a black list by the preset database;
and if the attribute information is marked as a blacklist by a preset database, blocking the flow direction of the data request and recording an attack log.
9. A firewall system, comprising:
the intrusion detection cluster comprises a plurality of detection nodes and is used for receiving a data request and generating an attack event log after detecting the data request;
the log server is connected with the intrusion detection cluster and used for receiving the attack event log and generating blacklist marking information of the attack event log;
load balancing cluster comprising a plurality of load balancing nodes for receiving said blacklist marking information, establishing a predetermined database and performing the method according to any one of claims 1 to 6 for processing data request traffic.
10. An electronic device, comprising:
a memory to store a computer program;
a processor configured to perform the method of any one of claims 1 to 6 for processing data request traffic.
11. A non-transitory electronic device readable storage medium, comprising: program which, when run by an electronic device, causes the electronic device to perform the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011481469.3A CN112637171A (en) | 2020-12-15 | 2020-12-15 | Data traffic processing method, device, equipment, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011481469.3A CN112637171A (en) | 2020-12-15 | 2020-12-15 | Data traffic processing method, device, equipment, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112637171A true CN112637171A (en) | 2021-04-09 |
Family
ID=75313275
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011481469.3A Pending CN112637171A (en) | 2020-12-15 | 2020-12-15 | Data traffic processing method, device, equipment, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112637171A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113489726A (en) * | 2021-07-06 | 2021-10-08 | 中国联合网络通信集团有限公司 | Flow limiting method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007098960A1 (en) * | 2006-03-03 | 2007-09-07 | Art Of Defence Gmbh | Distributed web application firewall |
| CN106790313A (en) * | 2017-03-31 | 2017-05-31 | 杭州迪普科技股份有限公司 | Intrusion prevention method and device |
| CN108683631A (en) * | 2018-03-30 | 2018-10-19 | 厦门白山耘科技有限公司 | A kind of method and system preventing scanning authority |
| CN108777709A (en) * | 2018-05-31 | 2018-11-09 | 康键信息技术（深圳）有限公司 | Website access method, device, computer equipment and storage medium |
| CN110474890A (en) * | 2019-07-29 | 2019-11-19 | 深圳数位传媒科技有限公司 | A kind of anti-crawling method of data and device based on intelligent flow guiding switching |
-
2020
- 2020-12-15 CN CN202011481469.3A patent/CN112637171A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007098960A1 (en) * | 2006-03-03 | 2007-09-07 | Art Of Defence Gmbh | Distributed web application firewall |
| CN106790313A (en) * | 2017-03-31 | 2017-05-31 | 杭州迪普科技股份有限公司 | Intrusion prevention method and device |
| CN108683631A (en) * | 2018-03-30 | 2018-10-19 | 厦门白山耘科技有限公司 | A kind of method and system preventing scanning authority |
| CN108777709A (en) * | 2018-05-31 | 2018-11-09 | 康键信息技术（深圳）有限公司 | Website access method, device, computer equipment and storage medium |
| CN110474890A (en) * | 2019-07-29 | 2019-11-19 | 深圳数位传媒科技有限公司 | A kind of anti-crawling method of data and device based on intelligent flow guiding switching |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113489726A (en) * | 2021-07-06 | 2021-10-08 | 中国联合网络通信集团有限公司 | Flow limiting method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10491614B2 (en) | Illegitimate typosquatting detection with internet protocol information | |
| US10447730B2 (en) | Detection of SQL injection attacks | |
| CN112383546B (en) | Method for processing network attack behavior, related equipment and storage medium | |
| CN108780485B (en) | Pattern matching based data set extraction | |
| US11831420B2 (en) | Network application firewall | |
| US10200384B1 (en) | Distributed systems and methods for automatically detecting unknown bots and botnets | |
| US10601848B1 (en) | Cyber-security system and method for weak indicator detection and correlation to generate strong indicators | |
| US8949988B2 (en) | Methods for proactively securing a web application and apparatuses thereof | |
| US10270792B1 (en) | Methods for detecting malicious smart bots to improve network security and devices thereof | |
| US9055093B2 (en) | Method, system and computer program product for detecting at least one of security threats and undesirable computer files | |
| US10666680B2 (en) | Service overload attack protection based on selective packet transmission | |
| TW201824047A (en) | Attack request determination method, apparatus and server | |
| US8713674B1 (en) | Systems and methods for excluding undesirable network transactions | |
| CN111786966A (en) | Method and device for browsing webpage | |
| CN111464526A (en) | Network intrusion detection method, device, equipment and readable storage medium | |
| CN113645234A (en) | Honeypot-based network defense method, system, medium and device | |
| US11128639B2 (en) | Dynamic injection or modification of headers to provide intelligence | |
| WO2023045196A1 (en) | Access request capturing method and apparatus, computer device, and storage medium | |
| US11303670B1 (en) | Pre-filtering detection of an injected script on a webpage accessed by a computing device | |
| CN115102781A (en) | Network attack processing method, device, electronic equipment and medium | |
| KR101658450B1 (en) | Security device using transaction information obtained from web application server and proper session id | |
| KR101658456B1 (en) | Security device using transaction information obtained from web application server | |
| US10686834B1 (en) | Inert parameters for detection of malicious activity | |
| KR101650475B1 (en) | Security device using transaction information obtained from web server | |
| US10757118B2 (en) | Method of aiding the detection of infection of a terminal by malware |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210409 |