CN115695043A

CN115695043A - Vulnerability scanning attack detection method, model training method and device

Info

Publication number: CN115695043A
Application number: CN202211446732.4A
Authority: CN
Inventors: 李云龙; 谭学士; 陈祚松
Original assignee: Qianxin Technology Group Co Ltd; Secworld Information Technology Beijing Co Ltd
Current assignee: Qianxin Technology Group Co Ltd; Secworld Information Technology Beijing Co Ltd
Priority date: 2022-11-18
Filing date: 2022-11-18
Publication date: 2023-02-03

Abstract

The application provides a vulnerability scanning attack detection method, a model training method and a model training device. The method comprises the following steps: acquiring an access log, and extracting key field information from the access log; generating a feature vector according to the key field information; inputting the characteristic vector into a vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating corresponding training feature vectors according to training logs in advance and performing model training by using the training feature vectors and labels for representing whether vulnerability scanning attack behaviors exist in training source IP addresses corresponding to the training logs; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior. According to the vulnerability scanning attack behavior identification method and device, the characteristic vectors of the access logs are analyzed through the machine learning model, and therefore accuracy of vulnerability scanning attack behavior identification is improved.

Description

Vulnerability scanning attack detection method, model training method and device

Technical Field

The application relates to the technical field of network security, in particular to a vulnerability scanning attack detection method, a model training method and a vulnerability scanning attack detection device.

Background

During the process of writing a website program, a worker may have a vulnerability caused by the reasons of not being considered all the time. In order to attack the website, a lawbreaker scans the programs of the website to find the existing vulnerabilities in the website.

In order to improve the security of a website, a rule matching mode is mostly adopted to find whether a vulnerability scanning attack behavior exists on the website, that is, a detection rule is preset, a generated log is matched by using the detection rule, and if the matching is successful, the behavior corresponding to the log is the vulnerability scanning attack behavior. Because the behavior of the lawless persons for vulnerability scanning is variable, the problem of low detection accuracy exists in a rule matching mode.

Disclosure of Invention

The embodiment of the application aims to provide a vulnerability scanning attack detection method, a model training method and a model training device, which are used for improving the accuracy of vulnerability scanning attack behavior identification.

In a first aspect, an embodiment of the present application provides a vulnerability scanning attack detection method, including:

acquiring an access log, and extracting key field information from the access log;

generating a feature vector according to the key field information;

inputting the characteristic vector into a vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating a corresponding training feature vector in advance according to a training log and performing model training by using the training feature vector and a label for representing whether vulnerability scanning attack behaviors exist in a training source IP address corresponding to the training log; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

According to the embodiment of the application, the key field information is extracted from the access log, the feature vector is generated according to the key field information, and the feature vector is input into the machine learning model, so that the machine learning model analyzes the feature vector of the access log, and the accuracy of vulnerability scanning attack behavior identification is improved.

In either embodiment, the key field information includes access time and source IP address; generating a feature vector according to the key field information, comprising:

storing the key field information into a set of corresponding time windows according to the access time and the source IP address;

and extracting the characteristics of the key field information in the set to obtain a characteristic vector.

According to the embodiment of the application, the time window is utilized to generate the characteristic vector corresponding to the key field information in the time window, so that the characteristic vector corresponding to the time window can be analyzed by utilizing the vulnerability scanning attack discovery model, and vulnerability scanning attack behaviors can be discovered in time.

In any embodiment, inputting the feature vector into a vulnerability scanning attack discovery model, comprising:

inputting the characteristic vector corresponding to the time window into a vulnerability scanning attack discovery model, and if the analysis result output by the vulnerability scanning attack discovery model indicates that the behavior corresponding to the time window is not vulnerability scanning attack behavior, obtaining a new characteristic vector according to the characteristic vector corresponding to the time window and the characteristic vector of the next time window corresponding to the source IP address;

and inputting the new feature vector into a vulnerability scanning attack discovery model.

In the embodiment of the application, the time window corresponding to the characteristic vector input into the vulnerability scanning attack discovery model is changed from small to large, vulnerability scanning attack behaviors can be discovered in time, and low-frequency and time-sharing vulnerability scanning attack behaviors can be identified.

In any embodiment, the features in the feature vector include: at least one of the number of access times of the source IP address, the number of URL access, the number of access ports, the number of different payload data, accumulated access time, the average length of the payload data, the average length of the URL accessed, the number of successful status code returns, the number of failed status code returns, the number of the same data and the same URL, the number of the same data and the same domain name, and the number of sensitive words in the payload data.

In the embodiment of the application, the characteristics are determined through a large number of research experiments, so that specific contents contained in the characteristic vector are determined, and the accuracy of identifying the access log is improved.

In any embodiment, the key field information further includes: at least one of returned status code, access duration, length of access header, byte number of access data packet, request method, destination port, accessed URL, accessed domain name, destination IP address, and payload data sent during access.

Since the key field information is a data basis of the feature vector, it is determined that specific information contained in the key field information is more critical, and the embodiment of the application provides a data basis for improving the accuracy of vulnerability scanning attack behaviors by extracting the key field information from the access log.

In any embodiment, after obtaining the analysis result output by the vulnerability scanning attack discovery model, the method further includes:

and if the analysis result represents that the behavior corresponding to the access log is a vulnerability scanning attack behavior, the source IP address corresponding to the access log is forbidden.

In the embodiment of the application, when the behavior corresponding to the access log is determined to be a vulnerability scanning attack behavior, the corresponding source IP address is sealed, and the security of the website is improved.

In any embodiment, the method further comprises:

acquiring a training sample, wherein the training sample comprises training logs corresponding to a plurality of training source IP addresses and labels corresponding to the training source IP addresses; the label is used for representing whether vulnerability scanning attack behaviors exist in the training source IP address or not;

extracting keywords from the training logs to obtain training keywords;

generating training characteristic vectors corresponding to the training IP addresses according to the training keywords corresponding to the training source IP addresses;

and training the model to be trained by utilizing the training characteristic vector and the corresponding label to obtain a vulnerability scanning attack discovery model.

According to the method and the device, the vulnerability scanning attack discovery model is obtained after model training, and the vulnerability scanning attack discovery model is utilized to improve the accuracy of identifying the access log.

In a second aspect, an embodiment of the present application provides a vulnerability scanning attack discovery model training method, including:

acquiring training samples, wherein the training samples comprise training logs corresponding to a plurality of training source IP addresses and labels corresponding to the training source IP addresses; the label is used for representing whether vulnerability scanning attack behaviors exist in the training source IP address or not;

extracting keywords from the training logs to obtain training keywords;

According to the vulnerability scanning attack discovery method and device, the training keywords are extracted from the training logs, the training feature vectors are generated according to the training keywords, the generated training feature vectors and the corresponding training source IP addresses are used for training the model to be trained, and the accuracy of detecting whether vulnerability scanning attack behaviors exist in the source IP addresses or not can be improved through the acquired vulnerability scanning attack discovery model.

In any embodiment, training the features in the feature vector comprises: at least one of the number of visits, the number of visited URLs, the number of visited ports, the number of different payload data, the accumulated visit duration, the average length of the payload data, the average length of the visited URLs, the number of successful status code returns, the number of failed status code returns, the number of the same data and the same URLs, the number of the same data and the same domain names, and the number of sensitive words in the payload data, which correspond to the training source IP address within a preset time period.

In the embodiment of the application, the characteristics used by the training model are determined through a large number of research experiments, so that the specific content contained in the training characteristic vector is determined, and the performance of the vulnerability scanning attack discovery model is improved.

In a third aspect, an embodiment of the present application provides a vulnerability scanning attack detection apparatus, including:

the log acquisition module is used for acquiring the access log and extracting key field information from the access log;

the characteristic extraction module is used for generating a characteristic vector according to the key field information;

the detection module is used for inputting the characteristic vector into the vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus, wherein,

the processor and the memory are communicated with each other through the bus;

the memory stores program instructions executable by the processor, the processor invoking the program instructions to be capable of performing the method of the first or second aspect.

In a fifth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium, including:

the non-transitory computer readable storage medium stores computer instructions that cause the computer to perform the method of the first or second aspect.

Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

Fig. 1 is a schematic flow chart of a vulnerability scanning attack detection method provided in an embodiment of the present application;

fig. 2 is a schematic flow chart of a model training method according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram illustrating interaction between a model and a server according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a vulnerability scanning attack detection apparatus provided in an embodiment of the present application;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only used to illustrate the technical solutions of the present application more clearly, and therefore are only used as examples, and the protection scope of the present application is not limited thereby.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions.

In the description of the embodiments of the present application, the technical terms "first", "second", and the like are used only for distinguishing different objects, and are not to be construed as indicating or implying relative importance or implicitly indicating the number, specific order, or primary-secondary relationship of the technical features indicated. In the description of the embodiments of the present application, "a plurality" means two or more unless specifically defined otherwise.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

In the description of the embodiments of the present application, the term "and/or" is only one kind of association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B, and may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

In the description of the embodiments of the present application, the term "plurality" refers to two or more (including two), and similarly, "plural sets" refers to two or more (including two), and "plural pieces" refers to two or more (including two).

In the description of the embodiments of the present application, the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", and the like, indicate the directions or positional relationships indicated in the drawings, and are only for convenience of description of the embodiments of the present application and for simplicity of description, but do not indicate or imply that the referred device or element must have a specific direction, be constructed and operated in a specific direction, and thus, should not be construed as limiting the embodiments of the present application.

In the description of the embodiments of the present application, unless otherwise explicitly stated or limited, the terms "mounted," "connected," "fixed," and the like are used in a broad sense, and for example, may be fixedly connected, detachably connected, or integrated; mechanical connection or electrical connection is also possible; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the embodiments of the present application can be understood by those of ordinary skill in the art according to specific situations.

Because a code writer may have bugs in a website program due to a poor consideration and the like when writing the website program, in order to repair the bugs, a website program maintainer scans the bugs of the website program to repair the bugs after finding the bugs. Meanwhile, lawless persons can attack the vulnerability after finding the vulnerability through vulnerability scanning, and the behavior is called vulnerability scanning attack. However, vulnerability scanning does not necessarily enable vulnerability discovery, and therefore, a lawbreaker may perform vulnerability scanning by sending multiple access requests to a server. As lawbreakers have deeper knowledge of vulnerabilities through the results returned by vulnerability scanning, more vulnerabilities may be discovered to exist in the website program.

At present, vulnerability scans from lawbreakers are often discovered using rule-matching methods, such as: common access paths, common attack loads and the like during vulnerability scanning can be matched, and once matching is successful, the vulnerability can be determined to belong to illegal behaviors. This approach requires a worker to set the matching rules empirically in advance. However, in order to evade the matching rule, lawbreakers change behavior habits of vulnerability scanning, such as: and adopting a low-frequency time-sharing method to scan the loopholes. Therefore, the accuracy of detecting the vulnerability scanning attack by adopting the rule matching method is lower.

In order to improve the accuracy of vulnerability scanning attack detection, the embodiment of the application provides a vulnerability scanning attack detection method and device based on machine learning, an electronic device and a storage medium. Before describing the embodiments of the present application, the related concepts involved in the present application will be explained:

nginx: nginx (engine x) is a high-performance HTTP and reverse proxy web server, and also provides IMAP/POP3/SMTP services. nginx is a lightweight Web server/reverse proxy server and an electronic mail (IMAP/POP 3) proxy server, and is characterized by small memory occupation and strong concurrency capability, and in fact, the concurrency capability of nginx is better in the same type of Web servers. The logs generated on nginx include an access log and an error log; the method comprises the following steps that a user region source, a skip source, a use terminal, a certain URL access amount and other related information can be obtained through an access log; through the error log, the performance bottleneck of a certain service or server of the system can be obtained.

web vulnerabilities: the web vulnerability generally refers to a vulnerability on a website program, and may be a vulnerability caused by the reason that a code writer considers the lack of the whole code writing and the like, and common web vulnerabilities include sql injection, xss vulnerability, uploading vulnerability and the like.

Vulnerability scanning attack: vulnerability scanning refers to detecting the security vulnerability of a designated remote or local computer system by means of scanning and the like based on a vulnerability database, and finding out a security detection (penetration attack) behavior of available vulnerabilities. The vulnerability scanner comprises different types such as network missing scanning, host missing scanning, database missing scanning and the like. Similarly, a hacker may first detect the vulnerability of the server through vulnerability scanning to prepare for subsequent attacks.

payload data: is the portion of data in which the information is written. In general, when data is transmitted, in order to make data transmission more reliable, original data is transmitted in batches, and certain auxiliary information, such as the size of the batch of data, check bits and the like, is added at the head and the tail of each batch of data, which is equivalent to adding some "coats" to the batch of original data, and these "coats" play a role in marking, so that the original data is not easily lost. A batch of data plus its "outer cover" forms the basic transmission unit in the transmission channel, called data frame or data packet (there is a place where data frame and data packet are not the same concept such as network transmission). The original data of the recording information in these data frames is payload data, i.e., payload data.

It can be understood that the model training method and the vulnerability scanning attack detection method provided by the embodiment of the present application may be applied to an electronic device, where the electronic device may include a terminal and a server; the terminal can be a smart phone, a tablet computer, a Personal Digital Assistant (PDA), and the like; the server may specifically be an application server, and may also be a Web server. In addition, both the model training method and the vulnerability scanning attack detection method can be executed by the same electronic device, and can also be executed by different electronic devices.

Fig. 1 is a schematic flow chart of a vulnerability scanning attack detection method provided in an embodiment of the present application, and as shown in fig. 1, the method includes:

step 101: acquiring an access log, and extracting key field information from the access log;

step 102: generating a feature vector according to the key field information;

step 103: inputting the characteristic vector into a vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating a corresponding training feature vector in advance according to a training log and performing model training by using the training feature vector and a label for representing whether vulnerability scanning attack behaviors exist in a training source IP address corresponding to the training log; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

In step 101, the access log refers to a log generated when the user accesses the target object, i.e. nginx access log. The user may be a compliance worker or customer or may be a lawbreaker. The target object refers to an object for which vulnerability scanning attack detection is performed by the present application, and may be, for example, a web site or the like.

When a user accesses a web site, a server corresponding to the web site generates an access log. If the electronic device is the web server, the access log can be directly utilized to perform subsequent step processing, and if the electronic device is a terminal, the terminal communicates with the web server to acquire the access log from the web server and utilize the acquired access log to perform subsequent step processing.

It will be appreciated that the access log includes a plurality of fields, such as: client (user) IP address (also referred to as source IP address), access time, access port, response time, request time, user address location code, URL address of the request, manner of request, status of request, requested page size, source page (i.e., from which page to which page), user browser language and other information of the user browser (e.g., browser version, browser type, etc.), etc.

Because the access log comprises a plurality of field information, some field information in the field information is required by the application, and some fields are not required by the application, after the electronic device acquires the access log, the electronic device can extract the key fields of the access log to obtain the key field information. It will be appreciated that the key fields to be extracted from the access log may be preset.

In step 102, after the electronic device obtains the key field information, the electronic device performs feature engineering on the key field information, that is, analyzes the key field information to obtain a feature vector. Specifically, the electronic device further stores a historical access log of the source IP address corresponding to the access log in a historical time period, so that a feature vector can be generated according to the historical access log and the current access log. It is understood that the feature vector may be formed by access features of the source IP address corresponding to the access log for accessing the target object, for example: the feature vector may be constituted by the number of times of accessing the URL, the number of accessing the URL, and the like. If the number of times of visiting the URL is 5 and the number of visiting URLs is 2, the feature vector may be (5,2). Of course, the feature vector may further include values corresponding to other access features, which is not specifically limited in this embodiment of the present application.

In step 103, the vulnerability scanning attack discovery model is obtained by training a model to be trained by using a training log in advance, wherein the vulnerability scanning attack discovery model may adopt a convolutional neural network model, a decision tree model, a support vector machine model, a random forest model and the like.

After the electronic device generates the feature vector, the feature vector is input into a vulnerability scanning attack discovery model obtained by pre-training, and the vulnerability scanning attack discovery model can output an analysis result corresponding to the feature vector. The analysis result may be that the access behavior of the source IP address corresponding to the access log is a vulnerability scanning attack behavior or not.

On the basis of the above embodiment, the key field information includes access time and source IP address; generating a feature vector according to the key field information, comprising:

storing the key field information into a corresponding time window set according to the access time and the source IP address;

In a specific implementation process, the key field information of the same source IP address in a time period is stored into a set corresponding to a corresponding time window. A time window refers to a period of time, such as: the time window may be 5 minutes or 10 minutes, and of course, the size of the time window may also be set according to actual situations, and the size of the time window is not specifically limited in this embodiment of the application. Taking 10 months and 10 days 0 of 2022 as an example, the time window is 10 minutes, the source IP address is 111.111.111.11, and the first time window is: 10 months 10 days 0 in 2022; second time window: 10/0 in 2022; third time window: 10/0 in 2022; …. And if the access time is 2022, 10 months, 10 days and 0. It should be noted that each appearing source IP address may obtain a set of corresponding time windows according to the above-mentioned dividing method, and delete key field information in a set after generating a corresponding feature vector according to key field information in the set corresponding to a certain time window.

In another embodiment, the starting time of each source IP address partition time window may also be different, that is, the access time corresponding to the first access of each source IP address to the target object may be divided. This has the advantage that if the access behavior of a certain source IP address is from a certain time in the middle, for example, from 12 am, then no corresponding time window needs to be set for the source IP address between 0 pm and 12 pm, which reduces the amount of computation.

A set corresponding to a time window may be empty, may include key field information corresponding to an access log, and may also include key field information corresponding to multiple access logs.

It should be noted that the key field information may further include at least one of:

the returned status code, the access duration, the length of the access header, the byte number of the access data packet, the request method, the target port, the accessed URL, the accessed domain name, the destination IP address and the payload data sent during the access.

And when the current time reaches the end time of each time window, acquiring key field information in a set corresponding to the time window, and performing feature extraction on the key field information to acquire a feature vector corresponding to the time window.

It should be noted that the features in the feature vector may comprise at least one of:

the number of access times of the source IP address corresponding to the time window, the number of access URLs, the number of access ports, the number of different payload data, the accumulated access duration, the average length of the payload data, the average length of the accessed URLs, the number of successful status code returns, the number of failed status code returns, the number of identical data and identical URLs, the number of identical data and identical domain names, and the number of sensitive words in the payload data.

The access times refer to the times of sending access requests to the target object by the source IP address in the time window. The number of URL visited refers to how many different URLs the source IP address visited in the time window. The number of access ports refers to the number of source IP address access ports in the time window. The different number of payload data refers to the different number of payloads within the time window. The accumulated access duration refers to the accumulated duration of source IP address access in the time window. The average length of the payload data is the quotient of the length of all payload data and the number of payloads in the time window. The average length of the accessed URLs is the quotient of the total byte length of the accessed URLs within the time window and the number of different URLs. The number of successful status code returns refers to the number of successful status code returns to the source IP address in the time window. The number of failed status code returns refers to the number of unsuccessful returns to the source IP address within the time window. Wherein, whether the return is successful or failed can be characterized by different status codes, for example: the state code of 200 represents the return success, the state code of 404 represents the return failure, and the number of the state code return success and the number of the state code return failure are determined by counting the number of the state codes of 200 and 404. It should be noted that other numbers may also be used to indicate the return success and the return failure, that is, they may be configured in advance, and this is not specifically limited in this embodiment of the present application. The number of the same data and the same URL refers to the number of the access requests with the same payload and the same URL accessed in the time window. The number of the same data and the same domain name refers to the number of the access requests with the same payload and the same access domain name in the time window. The number of the sensitive words in the payload data refers to the total number of the sensitive words in each payload data in the time window. Wherein, the sensitive words are preset words.

In another embodiment, key field information of a plurality of associated source IP addresses in a time period may also be stored in a set of corresponding time windows. Whether the source IP addresses have the association relationship or not can be determined through analysis modes such as accessed URLs, target IP addresses and payload data contained in the feature vectors in the historical time period.

It can be understood that through the analysis of the feature vectors in the historical time period, the portrait of the attacker can be generated, and which business system is more easily attacked by the attacker can be analyzed, so that the protection and the like of the business system which is easily attacked are enhanced.

On the basis of the embodiment, inputting the feature vector into a vulnerability scanning attack discovery model comprises the following steps:

In a specific implementation process, still taking the time window provided in the above embodiment as an example, after 10/00/10/2022, the feature vector corresponding to the first time window may be input into the vulnerability scanning attack discovery model, and if an analysis result output by the vulnerability scanning attack discovery model indicates that the access behavior of the source IP address corresponding to the first time window does not belong to the vulnerability scanning attack behavior. After 20/00 at 10/2022, adding a feature vector corresponding to a second time window and a feature vector corresponding to a first time window to obtain a new feature vector, and inputting the new feature vector to a vulnerability scanning attack discovery model, if an analysis result output by the vulnerability scanning attack discovery model indicates that an access behavior of a source IP address corresponding to the first time window does not belong to a vulnerability scanning attack behavior, inputting a feature vector corresponding to a third time window and feature vectors corresponding to the first two time windows to the vulnerability scanning attack discovery model for analysis, and so on, if the vulnerability scanning attack discovery model analyzes the sum of the feature vector of the nth time window and the feature vectors of the first N-1 time windows, and then obtaining an analysis result indicating that the access behavior of the source IP address belongs to a vulnerability scanning attack behavior, then performing subsequent processing on the source IP address. Or until the vulnerability scanning attack discovery model analyzes the sum of the feature vectors of the previous N time windows to obtain an analysis result representing that the access behavior corresponding to the source IP address is still not the vulnerability scanning attack behavior. It is understood that N is preset, for example, N =1440, and may also be the number of time windows corresponding to one day, which is not specifically limited in the embodiment of the present application.

The method has the advantages that by taking the time window as 1 hour as an example, if the vulnerability scanning attack discovery model detects that the lawbreaker has vulnerability scanning attack behaviors in the first two hours of a day, the source IP address used by the lawbreaker can be sealed and forbidden at the 2 nd hour of the day. Therefore, the smaller the time window is, the more the vulnerability scanning attack behavior can be found in time, and the larger the load of the electronic equipment is; conversely, the larger the time window, the less timely the vulnerability scanning attack behavior is found, and the load of the electronic device is relatively small. In the actual application process, the size of the time window can be set according to the actual situation.

In the embodiment of the application, the number of the time windows corresponding to the characteristic vectors input into the vulnerability scanning attack discovery model is changed from small to large, vulnerability scanning attack behaviors can be discovered in time, and low-frequency and time-sharing vulnerability scanning attack behaviors can be identified.

On the basis of the above embodiment, after obtaining the analysis result output by the vulnerability scanning attack discovery model, the method further includes:

In a specific implementation process, in order to ensure the security of a target object, after determining that a behavior corresponding to an access log is a vulnerability scanning attack behavior, a source IP address corresponding to the access log is forbidden. It will be appreciated that the source IP address is prohibited from accessing the target object.

In another embodiment, when the analysis result output by the vulnerability scanning attack discovery model indicates that the behavior corresponding to the access log belongs to vulnerability scanning attack behavior, the electronic device sends an alarm prompt to the terminal corresponding to the operation and maintenance personnel, the operation and maintenance personnel judges the behavior again, if the judgment result indicates that the behavior belongs to vulnerability scanning attack behavior, a confirmation message is sent to the electronic device, and the electronic device seals the source IP address. The advantage of this is that normal source IP addresses are prevented from being blocked after the model is found to be misdetected due to vulnerability scanning attacks.

On the basis of the foregoing embodiment, fig. 2 is a schematic flow chart of a model training method provided in the embodiment of the present application, and as shown in fig. 2, the method includes:

step 201: and acquiring training logs corresponding to a plurality of training source IP addresses. It can be understood that the training log is an access log corresponding to the training source IP address collected by the electronic device in a historical time period.

Step 202: and extracting keywords from the training logs to obtain training keywords. After the electronic equipment acquires the training logs, analyzing the training logs and extracting training keywords. It can be understood that the specific field corresponding to the training keyword is the same as the specific field corresponding to the key field information used in detecting the vulnerability scanning attack in the above embodiment, and is not described herein again. After the training keywords are obtained, labeling is carried out on the training keywords of each training log based on the experience of a safety expert. The label is used for representing whether vulnerability scanning attack behaviors exist in the training source IP address, for example: 1 can be adopted to represent that vulnerability scanning attack behaviors exist and used as a negative sample; adopt "0" to represent there is no vulnerability scanning attack behavior, as a positive sample. Other identifiers may also be used to identify the labels of the training log, which is not specifically limited in this embodiment of the present application.

Step 203: and generating training characteristic vectors corresponding to the training IP addresses according to the training keywords corresponding to the training source IP addresses. The preset time period can be determined according to actual conditions, for example, the preset time period can be set to one day, namely, the daily behavior of the training source IP is judged, whether the behavior is a vulnerability scanning behavior is judged, and the training source IP and the date are aggregated to obtain the training feature vector. Wherein training the feature vector comprises: at least one of the number of visits, the number of URLs visited, the number of ports visited, the number of different payload data, the accumulated visit duration, the average length of the payload data, the average length of the URLs visited, the number of successful status code returns, the number of failed status code returns, the number of URLs with the same data, the number of domain names with the same data, and the number of sensitive words in the payload data, which correspond to the training source IP address in one day. Training feature vectors and corresponding labels are generated into training samples, and a plurality of training samples form a training set. Therefore, the training set includes training samples corresponding to a plurality of training source IP addresses, and each training source IP address corresponds to a plurality of training samples. It can be understood that, in model training, the features included in the training feature vector input into the model to be trained are the same as the features included in the feature vector input into the trained vulnerability scanning attack discovery model during actual detection.

Step 204: and training a model to be trained by using the training feature vectors and the corresponding labels to obtain the vulnerability scanning attack discovery model. The label is used for representing whether vulnerability scanning attack behaviors exist in the training source IP address. The training set is split, for example, 80% of training samples in the training set can be used for training, 20% of training samples in the training set can be used for verification, and data used for training is trained by adopting a ten-fold cross-validation method. And after the training is finished, verifying the model obtained after the training by using a verification set, and if the accuracy of the model obtained after the training is more than 95%, taking the model obtained after the training as a final vulnerability scanning attack discovery model. It should be noted that the splitting ratio of the training set and the accuracy of stopping training may be set according to actual situations, which is not specifically limited in the embodiment of the present application.

After training is finished, the model file corresponding to the vulnerability scanning attack discovery model is stored in a database and is called when the model file is detected by a system.

In another embodiment, the deployment method of the vulnerability scanning attack discovery model comprises the following steps:

(1) Real-time vectorization of nginx logs

The vulnerability scanning attack discovery model is a real-time calculation model and can be deployed in a server cluster, after the nginx log is analyzed and sent to the cluster, grouping is carried out for one time, grouping is carried out according to the access time and the source IP address of the nginx log, namely the same source IP address is obtained, the access time is input into a divided time window and enters a set corresponding to the corresponding time window, the time window can be set to be five minutes or ten minutes, in the time window, the nginx log is converted into a feature vector according to feature engineering, the feature vector is sent to the vulnerability scanning attack discovery model to obtain an analysis result, and if the classification result shows that the behavior is attack, an alarm is sent. If not, the feature vectors are written into a cache, the feature vectors calculated twice are summed when the next window is a packet calculated this time, the summed feature vectors are sent to the vulnerability scanning attack discovery model again, and the result is returned after calculation. And analogizing until the analysis result represents that the vulnerability scanning attack behavior occurs in the source IP address, or the number of the time windows corresponding to the characteristic vectors input into the vulnerability scanning attack discovery model reaches a preset number.

(2) Deployment method of vulnerability scanning attack discovery model in server

A program of a vulnerability scanning attack discovery model and a program of a server cluster for processing logs are both deployed on the same server and communicate with each other in a process communication manner, and fig. 3 is an interaction diagram of the model and the server provided in the embodiment of the present application, as shown in fig. 3. After the log processing program generates the feature vector, the process of the vulnerability scanning attack discovery model is called, and the vulnerability scanning attack discovery model sends the analysis result back to the server in an interprocess communication mode.

(3) Alarm generation when an anomaly is found

When the vulnerability scanning attack discovery model finds that the vulnerability scanning attack is abnormal, the result is sent to an alarm front-end page, the detailed situation of the vulnerability scanning attack is displayed on the page, the detailed situation is provided for operators to judge, if the vulnerability scanning attack is determined to be the scanning attack, the server can be matched with a firewall to carry out blocking, if the vulnerability scanning attack is determined to be the normal behavior, the operators can mark the behavior, and the marked result can be stored to be used for updating the model regularly and improving the accuracy.

Fig. 4 is a schematic structural diagram of a vulnerability scanning attack detection apparatus provided in an embodiment of the present application, where the apparatus may be a module, a program segment, or a code on an electronic device. It should be understood that the apparatus corresponds to the above-mentioned embodiment of the method of fig. 1, and can perform various steps related to the embodiment of the method of fig. 1, and the specific functions of the apparatus can be referred to the description above, and the detailed description is appropriately omitted here to avoid redundancy. The device includes: a log obtaining module 401, a feature extraction module 402 and a detection module 403, wherein:

the log obtaining module 401 is configured to obtain an access log, and extract key field information from the access log;

the feature extraction module 402 is configured to generate a feature vector according to the key field information;

the detection module 403 is configured to input the feature vector into the vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating a corresponding training feature vector in advance according to a training log and performing model training by using the training feature vector and a label for representing whether vulnerability scanning attack behaviors exist in a training source IP address corresponding to the training log; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

On the basis of the above embodiment, the key field information includes access time and source IP address; the feature extraction module 402 is specifically configured to:

On the basis of the foregoing embodiment, the detection module 403 is specifically configured to:

On the basis of the above embodiment, the features in the feature vector include: at least one of the number of access times of the source IP address, the number of URL access, the number of access ports, the number of different payload data, accumulated access time, the average length of the payload data, the average length of the URL accessed, the number of successful status code returns, the number of failed status code returns, the number of the same data and the same URL, the number of the same data and the same domain name, and the number of sensitive words in the payload data.

On the basis of the above embodiment, the key field information further includes: at least one of returned status code, access duration, length of access header, byte number of access data packet, request method, destination port, accessed URL, accessed domain name, destination IP address, and payload data sent during access.

On the basis of the above embodiment, the apparatus further includes a processing module configured to:

On the basis of the above embodiment, the apparatus further includes a model training module configured to:

extracting keywords from the training logs to obtain training keywords;

generating training characteristic vectors corresponding to the training IP addresses according to the training keywords corresponding to the training source IP addresses; wherein training the feature vector comprises: at least one of the number of visits, the number of URLs visited, the number of access ports, the number of different payload data, accumulated visit duration, the average length of the payload data, the average length of the URLs visited, the number of successful status code returns, the number of failed status code returns, the number of URLs with the same data, the number of domain names with the same data, and the number of sensitive words in the payload data, which correspond to the training source IP address within a preset time period;

In another embodiment, an embodiment of the present application provides a vulnerability scanning attack discovery model training apparatus, including: sample acquisition module, keyword extraction module, training eigenvector generation module and training module, wherein:

the system comprises a sample acquisition module, a training sample acquisition module and a training sample acquisition module, wherein the sample acquisition module is used for acquiring a training sample, and the training sample comprises training logs corresponding to a plurality of training source IP addresses and labels corresponding to the training source IP addresses; the label is used for representing whether vulnerability scanning attack behaviors exist in the training source IP address or not;

the keyword extraction module is used for extracting keywords from the training logs to obtain training keywords;

the training feature vector generating module is used for generating training feature vectors corresponding to the training IP addresses according to the training keywords corresponding to the training source IP addresses;

and the training module is used for training the model to be trained by utilizing the training characteristic vectors and the corresponding labels to obtain the vulnerability scanning attack discovery model.

On the basis of the above embodiment, the features in the training feature vector include: at least one of the number of visits, the number of URLs visited, the number of ports visited, the number of different pieces of payload data, the accumulated visit duration, the average length of the payload data, the average length of the URLs visited, the number of successful status code returns, the number of failed status code returns, the number of URLs with the same data and the same data, the number of domains with the same data and the same domain name, and the number of sensitive words in the payload data, which correspond to the training source IP address within a preset time period.

Fig. 5 is a schematic structural diagram of an entity of an electronic device provided in an embodiment of the present application, and as shown in fig. 5, the electronic device includes: a processor (processor) 501, a memory (memory) 502, and a bus 503; wherein the content of the first and second substances,

the processor 501 and the memory 502 are communicated with each other through the bus 503;

the processor 501 is configured to call program instructions in the memory 502 to perform the methods provided by the above-mentioned method embodiments, for example, including: acquiring an access log, and extracting key field information from the access log; generating a feature vector according to the key field information; inputting the characteristic vector into a vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating corresponding training characteristic vectors in advance according to training logs and performing model training by using the training characteristic vectors and labels for representing whether vulnerability scanning attack behaviors exist in training source IP addresses corresponding to the training logs; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

The processor 501 may be an integrated circuit chip having signal processing capabilities. The Processor 501 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. Which may implement or perform the various methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The Memory 502 may include, but is not limited to, random Access Memory (RAM), read Only Memory (ROM), programmable Read Only Memory (PROM), erasable Read Only Memory (EPROM), electrically Erasable Read Only Memory (EEPROM), and the like.

The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above method embodiments, for example, including: acquiring an access log, and extracting key field information from the access log; generating a feature vector according to the key field information; inputting the characteristic vector into a vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating corresponding training characteristic vectors in advance according to training logs and performing model training by using the training characteristic vectors and labels for representing whether vulnerability scanning attack behaviors exist in training source IP addresses corresponding to the training logs; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the above method embodiments, for example, including: acquiring an access log, and extracting key field information from the access log; generating a feature vector according to the key field information; inputting the characteristic vector into a vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating corresponding training feature vectors according to training logs in advance and performing model training by using the training feature vectors and labels for representing whether vulnerability scanning attack behaviors exist in training source IP addresses corresponding to the training logs; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vulnerability scanning attack detection method is characterized by comprising the following steps:

generating a feature vector according to the key field information;

2. The method of claim 1, wherein the key field information comprises an access time and a source IP address; generating a feature vector according to the key field information includes:

and extracting the characteristics of the key field information in the set to obtain the characteristic vector.

3. The method of claim 2, wherein the inputting the feature vector into a vulnerability scanning attack discovery model comprises:

inputting the feature vector corresponding to the time window into the vulnerability scanning attack discovery model, and if the analysis result output by the vulnerability scanning attack discovery model indicates that the behavior corresponding to the time window is not vulnerability scanning attack behavior, obtaining a new feature vector according to the feature vector corresponding to the time window and the feature vector of the next time window corresponding to the source IP address;

and inputting the new feature vector into the vulnerability scanning attack discovery model.

4. The method of claim 1, wherein the features in the feature vector comprise: at least one of the number of access times of the source IP address, the number of URL access, the number of access ports, the number of different payload data, accumulated access time, the average length of the payload data, the average length of the URL accessed, the number of successful state code returns, the number of failed state code returns, the number of the same data and the same URL, the number of the same data and the same domain name, and the number of sensitive words in the payload data.

5. The method of claim 1, wherein the key field information further comprises: at least one of returned status code, access duration, length of access header, byte number of access data packet, request method, destination port, accessed URL, accessed domain name, destination IP address, and payload data sent during access.

6. The method of claim 1, wherein after obtaining the analysis results output by the vulnerability scanning attack discovery model, the method further comprises:

7. The method according to any one of claims 1-6, further comprising:

extracting keywords from the training logs to obtain training keywords;

generating training feature vectors corresponding to the training IP addresses according to the training keywords corresponding to the training source IP addresses;

and training a model to be trained by using the training feature vectors and the corresponding labels to obtain the vulnerability scanning attack discovery model.

8. A vulnerability scanning attack discovery model training method is characterized by comprising the following steps:

extracting keywords from the training logs to obtain training keywords;

9. The method of claim 8, wherein training the features in the feature vector comprises: at least one of the number of visits, the number of visited URLs, the number of visited ports, the number of different payload data, the accumulated visit duration, the average length of the payload data, the average length of the visited URLs, the number of successful status code returns, the number of failed status code returns, the number of the same data and the same URLs, the number of the same data and the same domain names, and the number of sensitive words in the payload data, which correspond to the training source IP address within a preset time period.

10. A vulnerability scanning attack detection apparatus, comprising:

the log acquisition module is used for acquiring an access log and extracting key field information from the access log;

the detection module is used for inputting the characteristic vector into a vulnerability scanning attack discovery model to obtain an analysis result output by the vulnerability scanning attack discovery model; the vulnerability scanning attack discovery model is obtained by generating a corresponding training feature vector in advance according to a training log and performing model training by using the training feature vector and a label for representing whether vulnerability scanning attack behaviors exist in a training source IP address corresponding to the training log; and the analysis result is used for representing whether the behavior corresponding to the access log is a vulnerability scanning attack behavior.

11. An electronic device, comprising: a processor, a memory, and a bus, wherein,

the processor and the memory are communicated with each other through the bus;

the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1-9.