CN112491784A

CN112491784A - Request processing method and device of Web site and computer readable storage medium

Info

Publication number: CN112491784A
Application number: CN202011094977.6A
Authority: CN
Inventors: 盛洋
Original assignee: Sina Technology China Co Ltd
Current assignee: Sina Technology China Co Ltd
Priority date: 2020-10-14
Filing date: 2020-10-14
Publication date: 2021-03-12

Abstract

The application discloses a method and a device for processing a request of a Web site and a computer readable storage medium, and the scheme of the embodiment of the application comprises the following steps: acquiring service log data of a Web site corresponding to a target request; detecting the service log data according to a first safety rule to obtain a first confidence coefficient for judging that the URL in the service log data is an attack URL; acquiring security policy rule information of a Web site; detecting the service log data according to a second security rule in the security policy rule information to obtain a second confidence coefficient for judging that the URL is an attack URL; and determining whether the target request corresponds to a threat detection result of the attack request according to the first confidence coefficient and the second confidence coefficient, and executing interception operation on the target request based on the threat detection result. According to the embodiment of the application, the accuracy of detecting the attack request of the Web site can be improved, and the risk of the attack on the Web site is reduced.

Description

Request processing method and device of Web site and computer readable storage medium

Technical Field

The present application relates to the field of website security technologies, and in particular, to a method and an apparatus for processing a request of a Web website, and a computer-readable storage medium.

Background

Logs of the existing Web site service system are stored locally, and the firewall system intercepts the Web site attack according to an interception strategy set by security operation and maintenance personnel in advance. The existing firewall technology detects based on a threat interception analysis means of a single safety rule, reduces the accuracy of judging the attack threat of a firewall system to a Web site, improves the false alarm rate and the missing report rate of threat analysis and judgment, and greatly improves the risk of the Web site.

How to reduce the risk of the attack of the Web site and improve the safety of the Web site is a technical problem to be solved by the application.

Disclosure of Invention

The embodiment of the application aims to provide a method and a device for processing a request of a Web site and a computer-readable storage medium, which are used for solving the problem of low accuracy of the existing attack request detection.

In order to solve the above technical problem, the present specification is implemented as follows:

in a first aspect, a method for processing a request of a Web site is provided, which includes:

acquiring service log data of a Web site corresponding to a target request;

detecting the service log data according to a first safety rule to obtain a first confidence coefficient for judging that a Uniform Resource Locator (URL) of the target request in the service log data is an attack URL; acquiring security policy rule information of the Web website;

detecting the service log data according to a second security rule in the security policy rule information to obtain a second confidence coefficient for judging the URL as an attack URL;

and determining whether the target request corresponds to a threat detection result of an attack request or not according to the first confidence degree and the second confidence degree, and executing an interception operation on the target request based on the threat detection result.

Optionally, the method further includes:

acquiring a preset anomaly detection model;

inputting the URL into the preset abnormal detection model for detection to obtain a third confidence coefficient for judging the URL as an attack URL;

determining whether the target request is a threat detection result of an attack request according to the first confidence degree and the second confidence degree, wherein the method comprises the following steps:

and determining a threat detection result corresponding to the target request according to the first confidence degree, the second confidence degree and the third confidence degree.

Optionally, the method further includes:

acquiring the IP address of the target request in the service log data;

detecting the service log data according to a preset threat intelligence database to obtain a fourth confidence coefficient for judging the IP address to be a threat IP address;

and determining a threat detection result corresponding to the target request according to the first confidence degree and the second confidence degree and by combining the third confidence degree and the fourth confidence degree.

Optionally, the first confidence is a first score corresponding to threat scoring when the URL is determined to be an attack URL according to the first security rule;

the second confidence coefficient is a second score corresponding to the threat scoring when the URL is judged to be an attack URL according to the second safety rule;

the third confidence coefficient is a third score corresponding to threat scoring when the URL is judged to be an attack URL according to the preset abnormal detection model;

the fourth confidence is a fourth score corresponding to threat scoring performed when the IP address is judged to be the threat IP address according to the preset threat intelligence database,

determining a threat detection result corresponding to the target request according to the first confidence degree and the second confidence degree and by combining the third confidence degree and the fourth confidence degree, including:

summing the first score, the second score, the third score, and the fourth score to obtain a cumulative threat score;

and when the accumulated threat score is larger than a preset score threshold value, determining that the target request is an attack request.

Optionally, the detecting the service log data according to the first security rule includes: performing semantic analysis on the URL to extract threat semantics corresponding to the URL; comparing and analyzing the threat semantics with a plurality of threat semantics rules included in a threat semantics analyzing library to judge whether the URL is an attack URL; the second security rule comprises a plurality of threat detection regularization rules, each threat detection regularization rule having a different scoring weight according to a corresponding threat level,

and determining the second score according to the scoring weight corresponding to the threat detection regular rule for judging the URL as the attack URL.

Optionally, before obtaining the service log data of the Web site corresponding to the target request, the method includes:

reading service log information obtained from a service cluster node of the Web website from a preset queue;

and converting the service log information of the Web site service cluster node into service log data in a Json format, wherein the service log data comprises an IP address and a URL (uniform resource locator) of a request for accessing the Web site.

Optionally, before obtaining the security policy rule information of the Web site, the method includes:

creating an Application Program Interface (API) of the HTTP service;

reading the security policy rule information pushed by a security policy rule system through the API;

and converting the security policy rule information into security policy rule log data in a Json format, wherein the security policy rule log data comprises the second security rule.

Optionally, when it is determined that the target request is an attack request according to the threat detection result, the method further includes: and taking the URL of the target request as a training sample, and training and updating the preset anomaly detection model.

In a second aspect, a request processing apparatus for a Web site is provided, which includes a processor and a processor electrically connected to the memory, where the memory stores a computer program that is executable by the processor, and the computer program implements the steps of the request processing method for a Web site according to the first aspect when executed by the processor.

In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, implements the steps of the request processing method for a Web site according to the first aspect.

In the embodiment of the application, the service log data of the Web site corresponding to the target request is acquired; detecting the service log data according to a first safety rule to obtain a first confidence coefficient for judging that a Uniform Resource Locator (URL) of the target request in the service log data is an attack URL; acquiring security policy rule information of the Web website; detecting the service log data according to a second security rule in the security policy rule information to obtain a second confidence coefficient for judging the URL as an attack URL; and determining a threat detection result corresponding to the target request according to the first confidence degree and the second confidence degree, and executing an interception operation on the target request based on the threat detection result, so that the accuracy of attack request detection can be improved, the risk of the Web site being attacked is greatly reduced, and the safety of the Web site is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a flowchart illustrating a request processing method for a Web site according to an embodiment of the present application.

Fig. 2 is a schematic application scenario diagram of a request processing method of a Web site according to an embodiment of the present application.

Fig. 3 is a diagram illustrating an example of a method for processing a request of a Web site according to an embodiment of the present application.

Fig. 4 is a schematic structural diagram of a request processing device of a Web site according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. The reference numbers in the present application are only used for distinguishing the steps in the scheme and are not used for limiting the execution sequence of the steps, and the specific execution sequence is described in the specification.

In order to solve the problems in the prior art, an embodiment of the present application provides a method for processing a request of a Web site, and fig. 1 is a schematic flow diagram of the method for processing a request of a Web site according to the embodiment of the present application. As shown in fig. 1, the method comprises the following steps:

and 102, acquiring service log data of the Web site corresponding to the target request.

The service log data of the Web site is obtained by collecting Web service log information generated by the Web cluster and performing corresponding processing.

Optionally, before obtaining the service log data of the Web site corresponding to the target request, the method includes: reading service log information obtained from a service cluster node of the Web website from a preset queue; and converting the service log information of the Web site service cluster node into service log data in a Json format, wherein the service log data comprises an IP address and a Uniform Resource Locator (URL) of a request for accessing the Web site.

The collecting of the Web service log information generated by the Web cluster is website Web service log information aggregation, and the website Web service log information aggregation is to convert the service log information generated by the Web cluster into service log data in a Json text form for storage.

Specifically, an application scenario of the request processing method of the Web site in the embodiment of fig. 2 may be referred to. A Kafka consumption process is created by the Web threat log collection system 30, the Web service system 22 for reading each node of the Web service cluster sends the service log information of the Web site on the Kafka queue 28 through Kafkcat software, the Web threat log collection system 30 obtains the corresponding service log information from the Kafka queue 28, and formats and converts the service log information according to the Key and Value definition of the Json format to obtain the service log data. Then, the fields of the service log data are split and stored in a database, for example, a cluster of a log collection database shown in fig. 2. Optionally, the table format of the Web service log data correspondingly obtained from the Web service system is as follows:

[ ACCESSORY IP ] [ TIMEST ] [ URL ]

The visitor IP is an IP address of the access request, the timestamp is the time when the current request is to access the Web site, and the URL is the URL of the access request.

In the above embodiment, the Web service log information is sent to the Kakfa service through Kafakcat software, and the Web threat log collection system 30 consumes data pushed into the Kafka queue by each Web service node through KafkaCat. The Web threat log collection system 30 creates a Kafka handler, reads Kafka queue data, performs text field splitting, and stores the Web service log data in the Json format into the table of the Elastic Search log collection database cluster 32. The stored table format is as follows:

[ INDEX NAME ] [ ACCESSOR IP ] [ TIME-STAMP ] [ URL ]

Wherein [ index name ] refers to a table index name of the Elastic Search log collection database cluster.

According to the method and the system, the Kafka service is created, the data collection agent client service with low dependency is achieved, a large traditional data collection agent service program with high dependency or a third-party plug-in does not need to be installed on the collected server, and the influence on the system service stability in the log data collection process is reduced.

And 104, detecting the service log data according to a first safety rule to obtain a first confidence coefficient for judging that the URL of the target request in the service log data is an attack URL request.

As described above, step 102 obtains the service log data corresponding to the target request. The service log data includes the IP address, URL, and access timestamp of the target request.

In step 104, the service log data of the target request may be detected according to a predetermined security rule, i.e. a first security rule, to preliminarily determine whether the target request is a threat request. Specifically, the first security rule may be a semantic analysis rule, and performs semantic determination on a URL in the service log data corresponding to the target request. Extracting threat semantics corresponding to the URL by analyzing the semantics of the URL, determining Script data contained in the URL, and comparing and analyzing the extracted threat semantics with a plurality of threat semantics rules included in a threat semantics analysis library to determine whether the URL requested by the target is a malicious Script, for example, if the URL contains a Structured Query Language (SQL) injection and Cross Site Script (xss) attack. It is determined that a malicious script exists in the URL.

However, since there may be a certain probability of misjudgment if the attack request is determined only by the first security rule, in the present application, if it is determined that a malicious script exists in the URL, the target request is preliminarily determined to be a threat request, and the first confidence level for determining the URL of the target request to be the attack URL is determined.

Specifically, the first confidence level is a threat score performed when the URL of the target request is determined to be the attack URL according to the first security rule, where the threat score corresponding to the first confidence level is a first score. For example, the URL of the target request hits the threat decision rule corresponding to the first security rule for a score of 1.

Therefore, in step 104, by using the semantic analysis rule of the semantic threat analysis library to perform attack threat determination on the URL in the service log data of the target request, it may be determined that the URL of the target request is the first confidence of the attack URL.

And step 106, acquiring the security policy rule information of the Web site.

The security policy rule information of the Web site is a website security policy rule issued by the security operation and maintenance personnel of the Web site, and is issued or changed by the security policy rule system 26 shown in fig. 2, for example. The security operation and maintenance personnel create and edit security detection policy rules, such as threat detection regular rules, through the security policy rule system 26 and maintain the security policy rule system 26. The security policy rule information of the Web site may be collected and obtained from the security policy rule system 26 by the Web threat log collection system 30 shown in fig. 2.

Optionally, before obtaining the security policy rule information of the Web site, the method includes: creating an Application Program Interface (API) for the HTTP service; reading the security policy rule information pushed by a security policy rule system through the API; and converting the security policy rule information into security policy rule log data in a Json format, wherein the security policy rule log data comprises a second security rule.

The above steps can be implemented by the WEB threat log collection system 30, an HTTP service interface of a REST (Representational State Transfer) API can be created, when a security operation and maintenance worker issues or changes a security policy through the security policy rule system 26, the security policy rule system 26 will push a message to the API interface, and the WEB threat log collection system 30 processes the service, and performs Json format conversion on the security policy rule information sent by the security policy rule system 26 to obtain corresponding security policy rule log data. And dividing fields of the log data, wherein the table format is as follows:

[ START ACTIVE STAMP ] [ END EFFECTIVE STAMP ] [ REGULATING RULE ]

The regular rule is a second security rule, and is used for threat attack analysis of the target request. The start valid timestamp and the end valid timestamp are correspondingly set with valid time periods corresponding to the regular rules, that is, the valid time periods are not within the time periods, and the rules are invalid by the adjustment. The operation and maintenance personnel manage the validity period of the regular rule through the timestamp, and regularly issue the push rule information to the WEB threat log collection system 30. The REST API interface created by the WEB threat log collection system 30 is used to receive the security policy rule information issued by the security policy rule system 26 at different times. And then field division is carried out, and the corresponding security policy rule log data is stored in a base table of the ElasticSearch log collection database cluster 32.

[ INDEX NAME ] [ START ACTIVATE STAMP ] [ END EFFECT STAMP ] [ REGULATION RULE ]

And 108, detecting the service log data according to a second security rule in the security policy rule information to obtain a second confidence coefficient for judging the URL as an attack URL request.

And the second confidence coefficient is a second score corresponding to the threat scoring when the URL is judged to be the attack URL request according to the second safety rule, the second safety rule is the regular rule for threat detection or analysis, and the regular rule is created and edited by safety operation and maintenance personnel. The second security rule comprises a plurality of threat detection regular rules, and the URL of the target request can be preliminarily judged as the confidence coefficient of the attack URL by comparing the URL with the regular rules. That is, when the URL requested by the determination target is the attack URL, the second score obtained by the threat scoring is performed.

In one embodiment, the second security rule includes a plurality of threat detection regular rules having different scoring weights according to corresponding threat degrees, that is, when a requested URL conforms to a certain regular rule, the URL may be determined as an attack URL with a higher probability, and when the requested URL conforms to another regular rule, the URL may be determined as an attack URL with a lower probability. Thus, a regularization rule corresponding to a higher probability has a higher scoring weight, and a regularization rule corresponding to a lower probability has a lower scoring weight. And determining the second score according to the scoring weight corresponding to the threat detection regular rule for judging the URL as the attack URL request.

For example, the URL of the target request hits a corresponding threat determination rule of the second security rule with a score in the range of 2-3, where the highest probability rule may have a score of 3 and the lowest probability rule may have a score of 2, and the intermediate probability rules may have a score of between greater than 2 and less than 3 based on the respective probability magnitudes.

Step 110, determining whether the target request corresponds to a threat assessment result of an attack request according to the first confidence level and the second confidence level, and executing an interception operation on the target request based on the threat assessment result.

As described above, in

steps

104 and 108, by detecting the target request, a first confidence corresponding to the first security rule and a second confidence corresponding to the second security rule may be obtained. The detection steps of step 104 and step 108 may be performed independently and in parallel, or may be performed sequentially, and the present application is not limited thereto.

In step 110, it is determined whether the target request is a threat assessment result of the attack request according to the first confidence degree and the second confidence degree, which may be a cumulative threat score obtained by summing a first score corresponding to the first confidence degree and a second score corresponding to the second confidence degree. And when the cumulative threat score is larger than a preset score threshold value, determining that the target request is an attack request, namely determining that the URL in the website Web service log data corresponding to the target request is an attack URL. The interception operation of the target request is executed, and the interception operation can be an operation of alarming and informing a firewall to intercept, limit the speed, shield and the like the target request to prevent attacks.

The first score and the second score participate in a threat analysis algorithm count, such as the Web attack threat analysis 34 described in fig. 2, resulting in a cumulative threat score.

And finally, after the threat detection analysis judgment is carried out, the threat count is greater than a specified score, the UR of the target request is judged to be an attack type URL, and then the attack blocking operation is executed.

Therefore, by acquiring the service log data and the security policy rule information of different types of Web sites and performing associated threat detection analysis on the target request, the accuracy of threat analysis and judgment can be improved, and the risk of the Web site being attacked is reduced.

In order to further improve the accuracy of detecting the target request threat, optionally, the request processing method of the application Web site further includes: acquiring a preset anomaly detection model; inputting the URL into the preset abnormal detection model for detection to obtain a third confidence coefficient for judging the URL as an attack URL request; determining whether the target request is a threat detection result of an attack request according to the first confidence degree and the second confidence degree, wherein the method comprises the following steps: and determining a threat assessment result corresponding to the target request according to the first confidence degree, the second confidence degree and the third confidence degree.

The predetermined anomaly detection model is obtained by performing artificial neural network modeling training of a Long Short-Term Memory network (LSTM) by using an attack request of abnormal access as a sample. For example, a training sample set and an initial anomaly detection model are obtained firstly, wherein the training sample set comprises a plurality of URL data samples of attack requests and URL data samples of normal requests; and training the initial anomaly detection model by using the training sample set to obtain the preset anomaly detection model.

In addition, LSTM artificial neural network modeling can be performed by using the URL data sample set of the normal request, and a normal detection model is obtained through training. And then, storing the normal detection model and the abnormal detection model in a warehouse.

After determining that the target request is an attack request in step 110, the target request is further input into a trained predetermined anomaly detection model for decision analysis, resulting in a ratio that the request is an attack request. The result of the model calculation is a proportional number that approaches the number 1. In the anomaly detection model, the closer the calculation result is to 1, the greater the probability that the target request is an attack request is represented.

The third confidence degree is a third score corresponding to the threat scoring when the URL is determined to be the attack URL according to the predetermined anomaly detection model, that is, after the URL of the target request is output to the predetermined anomaly detection model and the corresponding calculation result is close to 1, the target request is considered to be the attack request, and the score corresponding to the third confidence degree is, for example, 1.

At this time, the first score of the first security rule, the second score of the second security rule, and the cumulative score corresponding to the third score of the anomaly detection model may be combined and compared with a predetermined score threshold, so as to determine whether the current URL is an attack behavior.

According to the method and the system, the safety detection strategy is dynamically and automatically established through machine learning by means of an intelligent algorithm trained by a model, the false alarm rate and the missing report rate of attack interception are reduced, and the safety operation and maintenance labor cost is reduced.

In order to automatically improve or improve the accuracy of the anomaly detection model, optionally, when the target request is determined to be an attack request according to the threat detection result, the method further includes: and taking the URL of the target request as a training sample, and training and updating the preset anomaly detection model.

The method comprises the steps that new attack URLs are increased along with newly generated external attacks, after the number of the attack URL entries reaches a preset number, old unrepeated sample data can be iterated by using URL sample data of new attack requests, the requests determined as the attack requests serve as threat sample data to be stored in a warehouse, updated anomaly detection models are generated again according to the new threat sample data and threat models stored in the warehouse before, and the accuracy is improved automatically along with the accumulation of attack request data. Through detection and analysis, normal request data and abnormal request data in the historical log are divided, a detection system is enabled to automatically learn through historical attack data, and a detection model is automatically optimized and adjusted, so that the detection model has the capability of automatically performing association threat analysis, and the detection precision is improved.

In order to further improve the accuracy of detecting the target request threat, optionally, the request processing method of the application Web site further includes: acquiring an IP address of the target request in service log data of a Web site; detecting the service log data according to a preset threat intelligence database to obtain a fourth confidence coefficient for judging the IP address to be a threat IP address; determining whether the target request is a threat detection result of an attack request according to the first confidence degree and the second confidence degree, wherein the method comprises the following steps: and determining a threat detection result corresponding to the target request according to the first confidence degree and the second confidence degree and by combining the third confidence degree and the fourth confidence degree. And the fourth confidence coefficient is a fourth score corresponding to threat scoring when the IP address is judged to be the threat IP address according to the preset threat information database.

As described above, the service log data of the Web site includes the URL and the IP address, and when the Web threat log collection system 30 collects the service log of the Web site, the visitor IP of the client port that has received the Web request is received. The IP address is compared with a predetermined threat intelligence database, such as threat intelligence query database 38 of fig. 2, and if the IP address can be found in threat intelligence query database 38, the threat type and risk level result attributes of the IP address threat intelligence can be obtained.

The table format in threat intelligence query repository 38 is as follows:

[ visitor IP ] [ timestamp ] [ URL ] [ threat class ] [ risk class ]

If no threat intelligence query is found in threat intelligence query repository 38, the fourth score corresponding to the fourth confidence level for the IP address is 0, and the IP address is not considered to be a threat. If available in threat intelligence query repository 38, subsequent threat scoring processes are performed according to the level of risk, with different levels of scoring differing.

At this time, the cumulative score corresponding to the first score of the first security rule, the second score of the second security rule, the third score of the anomaly detection model, and the fourth score of the threat IP may be compared with a predetermined score threshold value, so as to determine whether the current URL is an attack behavior.

Of course, the third score of the abnormality detection model may not be considered, and the scores may be accumulated only based on the first score, the second score, and the fourth score. Or, according to the accumulated scores obtained by calculation of at least two of the first score, the second score, the third score and the fourth score, at least two attack request detection modes are associated to determine whether the target request is an attack request, so that the accuracy of attack request detection is greatly improved, and the safety of the Web site is improved.

In an embodiment, the request processing method for the Web site of the present application may further collect logs in a firewall system, for example, the Web threat log collection system 30 collects and aggregates logs in the firewall system 24 to obtain an alarm log of a Web attack event therein, and splits an alarm log text into meaningful fields according to a certain format requirement, and stores the meaningful fields in a library table of the Elastic Search log collection database cluster 32.

Specifically, by creating a network monitoring service of a User Datagram Protocol (UDP), the log data sent by the firewall system 24 is received, and according to the definition of the relevant fields of the firewall threat event, the field of the whole firewall system log is split, and the whole firewall system log is divided into a plurality of fields and stored in a database table, where the table format is as follows:

[ types of attacks ] [ timestamps ] [ attack URLs ] [ threat level ]

The table format of the database is as follows:

[ INDEX NAME ] [ ATTACK TYPE ] [ TIME-STAMP ] [ ATTACK URL ] [ ATTACK LEVEL ]

The existing firewall log collection is to collect log data by supporting a firewall sending function, but the collection of the firewall logs causes high data packet loss rate, and the reliability and integrity of log data transmission cannot be ensured under high-load work. According to the method and the system, the network monitoring service is established, the firewall log output can be directly supported, the log data loss is avoided, and the influence of collection of the firewall log by an installation data agent on the service stability of the system is also avoided.

For the embodiment of fig. 2, in the present application, the service log information of the WEB service system, the system log information of the firewall system 24, and the security policy rule information of the security policy rule system 26 are obtained in advance through the WEB threat log collection system 30, and are correspondingly converted into fields in a predetermined format, and the WEB threat analysis associated logs are stored in a storage in the Elastic Search log collection database cluster 32.

The Web attack threat analysis 34 detects a target request of current Web access by using a database 36 and a threat intelligence query library 38, which store threat sample data and threat models, stores corresponding threat analysis results in a database 40, and finally displays the threat analysis results to a front end 42 through attack threat analysis display service.

Next, a description will be made of an example of a request processing method of a Web site according to an embodiment of the present application, with reference to the embodiment of fig. 3.

As shown in fig. 3, the method comprises the following steps:

step 302: web business service log information aggregation;

step 304: formatting the log;

step 306: and storing the log database.

Step 308: firewall log information aggregation;

step 310: formatting the log;

step 312: and storing the log database.

Step 314: aggregating security policy rule information;

step 316: formatting the log;

step 318: and storing the log database.

Through

steps

306, 312 and 318, the text log related to the website service is stored in the full-text retrieval database, and a set of data processing flow method in the REST API form is provided, so that log processing standardization is easy to maintain, and reading is convenient and efficient.

Step 320: and analyzing the Web threat attack.

In step 320, after the currently received target request, the REST API interface queries the index table of the Elastic Search log collection database cluster to store the above various log data, and detects the target request according to the threat association analysis of the following rules:

and acquiring the visitor IP in the Web service log data (visitor IP) and URL (uniform resource locator), and inquiring the visitor IP in a threat information database. And obtaining the information of data (threat level) returned from the threat information database of the (visitor IP), and accumulating a certain score for the (threat score) variable according to the threat level.

And acquiring URL in Web service log data, namely visitor IP, URL, and judging the URL through a threat semantic analysis library. And if the obtained semantic analysis result is that the URL is not the attack request URL, if the URL is the attack request URL, accumulating a certain score for the variable [ threat count ].

The method comprises the steps of obtaining URL in Web service log data, visitor IP, URL, obtaining safety strategy rule data, beginning effective timestamp, ending effective timestamp and regular rule, matching URL with regular rule, defining URL meeting regular rule as attack URL, and accumulating certain scores according to scoring weight of matched regular rule if the URL is attack request URL.

And acquiring URL in Web service log data (visitor IP) and URL, counting the URL and a preset abnormal detection model, and analyzing and judging whether the URL is an abnormal attack request by using the abnormal detection model. If the calculated ratio of the anomaly detection model approaches 1, it is indicated as an anomalous request, and approaches 0, it is indicated as a normal request. The URL and the normal detection model can be counted, if the result of the normal detection model approaches to 1, the URL is represented as a normal request, and if the result of the normal detection model approaches to 0, the URL does not belong to the normal request. And comprehensively judging whether the current URL request is an attack request or not according to the analysis results of the two types of detection models, and if the current URL request is the attack request, accumulating a certain score for the variable of the URL pair threat count.

Comparing the visitor IP in the Web service log data with a threat information database, performing associated threat analysis, performing semantic attack threat analysis, regular strategy attack threat analysis and attack threat analysis with a preset abnormal detection model on URL, accumulating a certain score for the threat count when judging that the threat attack is established in each step, and finally judging that the URL in the Web service log of the current website is an attack URL after judging that the threat count is greater than a specified threshold score through the threat analysis, alarming and informing a firewall, and intercepting, limiting speed, shielding and other attack prevention operations on a target request of the current visitor IP.

Step 322: and (5) warehousing the Web threat attack analysis result.

In this step, the analysis result of the detection target request as a normal request and/or an abnormal request may be stored in the database as a training sample of the abnormal detection model and the normal detection model, so as to update and train the detection model, and further improve the detection accuracy of the detection model.

Step 324: and displaying the analysis result of the Web threat attack.

According to the method, through acquiring the service log information of the Web site and the rule information in a safety detection rule strategy system, the URL in the service log of the Web site is subjected to regular rule detection, semantic threat attack detection and artificial neural network algorithm detection at the same time, threat information database association detection is carried out on visitor IP in the log, multi-angle multi-level analysis is carried out on the safety inspection strategy of the same log data source through multiple threat detection strategies, the potential threat attack behavior is accurately judged, the attack behavior is intercepted and blocked, the website attack threat detection accuracy is improved, the false alarm rate and the false missing report rate of a firewall are reduced, and the risk that the website service is attacked is reduced or avoided.

In addition, by creating the Kafaka service, the REST API interface and/or the UDP monitoring service, corresponding types of threat detection data can be automatically collected and aggregated, the stability of the system and the integrity of the data are ensured, and the labor cost for detection is reduced.

In order to solve the problems in the prior art, optionally, an embodiment of the present application further provides a request processing device for a Web site, as shown in fig. 4, where the request processing device 2000 includes a memory 2200 and a processor 2400 electrically connected to the memory 2200, where the memory 2200 stores a computer program that can be run by the processor 2400, and the computer program, when executed by the processor 2400, implements each process of the above-mentioned embodiment of the request processing method for a Web site, and can achieve the same technical effect, and in order to avoid repetition, it is not described here again.

The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements each process of the above-mentioned method for processing a request of a Web site, and can achieve the same technical effect, and is not described herein again to avoid repetition. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.

While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A request processing method of a Web site is characterized by comprising the following steps:

acquiring service log data of a Web site corresponding to a target request;

detecting the service log data according to a first safety rule to obtain a first confidence coefficient for judging that a Uniform Resource Locator (URL) of the target request in the service log data is an attack URL;

acquiring security policy rule information of the Web website;

2. The method of claim 1, further comprising:

acquiring a preset anomaly detection model;

inputting the URL into the preset abnormal detection model for detection to obtain a third confidence coefficient for judging the URL as an attack URL; wherein the content of the first and second substances,

determining whether the target request is a threat detection result of an attack request according to the first confidence degree and the second confidence degree, wherein the threat detection result comprises:

3. The method of claim 2, further comprising:

acquiring the IP address of the target request in the service log data;

detecting the service log data according to a preset threat intelligence database to obtain a fourth confidence coefficient for judging the IP address to be a threat IP address; wherein the content of the first and second substances,

4. The method of claim 3,

the first confidence coefficient is a first score corresponding to threat scoring when the URL is judged to be an attack URL according to the first safety rule;

the fourth confidence is a fourth score corresponding to threat scoring when the IP address is detected to be a threat IP address according to the preset threat intelligence database, wherein,

5. The method of claim 4,

the first security rule is a semantic analysis rule, and the detecting the service log data according to the first security rule includes: performing semantic analysis on the URL to extract threat semantics corresponding to the URL; comparing and analyzing the threat semantics with a plurality of threat semantics rules included in a threat semantics analyzing library to judge whether the URL is an attack URL;

the second security rule comprises a plurality of threat detection regularization rules, each threat detection regularization rule having a different scoring weight according to a corresponding threat level, wherein,

and determining the second score according to the scoring weight corresponding to the threat detection regular rule for detecting the URL as the attack URL.

6. The method of claim 1, wherein prior to obtaining the service log data of the Web site corresponding to the target request, the method comprises:

7. The method of claim 1, prior to obtaining security policy rule information for the Web site, comprising:

creating an Application Program Interface (API) of the HTTP service;

8. The method of claim 2, wherein upon determining that the target request is an attack request based on the threat detection result, further comprising:

and taking the URL of the target request as a training sample, and training and updating the preset anomaly detection model.

9. An electronic device, comprising: a memory and a processor electrically connected to the memory, the memory storing a computer program executable on the processor, the computer program, when executed by the processor, implementing the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.