CN107659570B - Webshell detection method and system based on machine learning and dynamic and static analysis - Google Patents

Abstract

The embodiment of the invention provides a Webshell detection method and system based on machine learning and dynamic and static analysis, and relates to the technical field of Webshell detection. The method comprises the steps of obtaining a sample file, extracting static characteristics and dynamic characteristics of the sample file, obtaining a classification model according to the static characteristics, the dynamic characteristics and a machine learning algorithm, and analyzing a file to be detected by the classification model to obtain a detection result. The method adopts an analysis means combining dynamic and static states, the extracted features are more comprehensive, a machine learning algorithm combining various classification algorithms is adopted to learn a large number of Webshell samples and normal webpage samples to form a classification model, the stability of the classification model is higher, and the classification is more accurate; by adopting the classification model, Webshell and variants thereof can be effectively detected, the novel Webshell can be predicted, text confusion means can be well dealt with, and the defect that a feature code matching detection mode is adopted in the prior art is overcome.

Description

Webshell detection method and system based on machine learning and dynamic and static analysis

technical field

The invention relates to the technical field of Webshell detection, in particular to a Webshell detection method and system based on machine learning and dynamic and static analysis.

Background technique

With the vigorous development of Internet applications and the rapid growth of Internet data, server security problems are becoming more and more serious, and web application-based backdoor programs such as Webshell are extremely harmful to user information and even the entire application system. Therefore, timely detection and discovery of server vulnerabilities And backdoors, it is crucial to keep the server safe.

Since most Webshells are written in scripting languages and are easy to be modified and deformed, their features are not limited to signature codes, but also include file operation functions, malicious execution functions, file comment size, single-line string length, and degree of confusion. When the feature code is deliberately obfuscated, the traditional method will miss such webshells, that is, it is easy to bypass the detection of firewalls and antivirus software through obfuscation. Therefore, the current webshell detection methods based on feature matching are difficult to quickly detect and identify the variant.

Therefore, how to overcome the singleness and hysteresis of the traditional Webshell detection method based on signature matching, deal with the text obfuscation method of Webshell, and realize the rapid detection of Webshell and its variants has always been the focus of those skilled in the art.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a Webshell detection method based on machine learning and dynamic and static analysis, in order to overcome the singleness and lag of the traditional Webshell detection method based on feature code matching, improve the accuracy of Webshell detection, and quickly detect Webshell and its variant.

The purpose of the present invention is also to provide a Webshell detection system based on machine learning and dynamic and static analysis, so as to overcome the singleness and lag of the traditional Webshell detection method based on feature code matching, improve the accuracy of Webshell detection, and quickly detect Webshell and its variants.

In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:

In a first aspect, an embodiment of the present invention proposes a Webshell detection method based on machine learning and dynamic and static analysis. The Webshell detection method based on machine learning and dynamic and static analysis includes: acquiring a sample file; extracting static features of the sample file and dynamic features; a classification model is obtained according to the static features, the dynamic features and the machine learning algorithm, and the classification model analyzes the files to be detected and obtains the detection results.

Further, the step of extracting the static features and dynamic features of the sample file includes: performing static analysis on the sample file to obtain the static features, wherein the static features include document features, basic Function features and file behavior features; the dynamic features are obtained by dynamically analyzing the sample file, wherein the dynamic features include file inclusion operation features, sensitive function running features, and sensitive string features.

Further, the step of obtaining the classification model according to the static features, the dynamic features and the machine learning algorithm includes: learning the static features and the dynamic features using the machine learning algorithm to obtain the classification model .

Further, the machine learning algorithm is a collective learning method combining multiple classification algorithms.

Further, the Webshell detection method based on machine learning and dynamic and static analysis also includes: when the to-be-detected file is confirmed to be a Webshell after being detected, performing machine learning again according to the to-be-detected file and the sample file to update. the classification model.

In a second aspect, an embodiment of the present invention also provides a webshell detection system based on machine learning and dynamic and static analysis, the webshell detection system based on machine learning and dynamic and static analysis includes a sample acquisition module, a feature extraction module, and a model building module. The sample acquisition module is used for acquiring sample files; the feature extraction module is used for extracting static features and dynamic features of the sample files; the model building module is used for The algorithm obtains a classification model, which analyzes the file to be detected and obtains a detection result.

Further, the feature extraction module includes a static analysis module and a dynamic analysis module. The static analysis module is configured to perform static analysis on the sample file to obtain the static features, wherein the static features include document features, basic function features, and file behavior features of the sample file; the dynamic analysis module uses The dynamic features are obtained by dynamically analyzing the sample file, wherein the dynamic features include file inclusion operation features, sensitive function running features, and sensitive character string features.

Further, the model building module is configured to use the machine learning algorithm to learn the static features and the dynamic features to obtain the classification model.

Further, the machine learning algorithm adopted by the model building module is a collective learning method combining multiple classification algorithms.

Further, the Webshell detection system based on machine learning and dynamic and static analysis also includes a model update module, and the model update module is used for when the file to be detected is confirmed to be a Webshell after being detected, according to the to-be-detected file. The file is re-machined with the sample file to update the classification model.

Relative to the prior art, the present invention has the following beneficial effects: the Webshell detection method and system based on machine learning and dynamic and static analysis provided by the embodiments of the present invention extract the static features and dynamic features of the sample files by acquiring sample files, according to The static feature, the dynamic feature and the machine learning algorithm obtain a classification model, and the classification model analyzes the file to be detected and obtains a detection result. The embodiment of the present invention adopts a dynamic and static analysis method to extract features more comprehensively, and uses a machine learning algorithm combined with multiple classification algorithms to learn a large number of Webshell samples and normal webpage samples to form a classification model, the classification model is more stable, and the classification more precise. The use of machine learning algorithms can deal with complex classification calculations with multiple features, so that the features involved in detection are not limited to a single feature code. Using this classification model, users can effectively detect Webshells and their variants, predict new Webshells, and can better deal with text obfuscation methods, making up for the shortcomings of traditional signature matching detection methods.

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore do not It should be regarded as a limitation of the scope, and for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 shows a schematic block diagram of a server provided by an embodiment of the present invention.

FIG. 2 shows a functional module diagram of the Webshell detection system based on machine learning and dynamic and static analysis provided by the first embodiment of the present invention.

FIG. 3 shows a functional block diagram of the feature extraction module in FIG. 2 .

FIG. 4 shows a schematic flow chart of Webshell detection by a Webshell detection system based on machine learning and dynamic and static analysis.

FIG. 5 shows a schematic flowchart of the Webshell detection method based on machine learning and dynamic and static analysis provided by the second embodiment of the present invention.

FIG. 6 shows a schematic flowchart of the specific flow of step S202 in FIG. 5 .

Icon: 100-server; 400-Webshell detection system based on machine learning and dynamic and static analysis; 110-memory; 120-storage controller; 130-processor; 410-sample acquisition module; 420-feature extraction module; 430-model Building module; 440-model update module; 421-static analysis module; 422-dynamic analysis module.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings are not intended to limit the scope of the invention as claimed, but are merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

The Webshell detection method and system based on machine learning and dynamic and static analysis provided by the embodiments of the present invention can be applied to the server 100 shown in FIG. 1 . In this embodiment, the server 100 may be, but not limited to, a network server, a database server, a cloud server, and the like. As shown in FIG. 1 , the server 100 may include a memory 110 , a storage controller 120 and a processor 130 .

The memory 110 , the storage controller 120 and the processor 130 are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these elements may be electrically connected to each other through one or more communication buses or signal lines. The Webshell detection system 400 based on machine learning and dynamic and static analysis includes at least one software that can be stored in the memory 110 in the form of software or firmware or fixed in an operating system (OS) of the server 100 functional module. The processor 130 is configured to execute executable modules stored in the memory 110, for example, software function modules and computer programs included in the Webshell detection system 400 based on machine learning and dynamic and static analysis.

Wherein, the memory 110 may be, but not limited to, a random access memory (Random Access Memory, RAM), a read only memory (Read Only Memory, ROM), a programmable read only memory (Programmable Read-Only Memory, PROM), an erasable memory In addition to read-only memory (Erasable Programmable Read-Only Memory, EPROM), Electrical Erasable Programmable Read-Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM) and the like. The memory 110 can be used to store software programs and modules, and the processor 130 is used to execute the programs after receiving the execution instructions.

The processor 130 may be an integrated circuit chip with signal processing capability. The above-mentioned processor 130 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. A general purpose processor may be a microprocessor or the processor 130 may be any conventional processor or the like.

It can be understood that the structure shown in FIG. 1 is only for illustration, and the server 100 may further include more or less components than those shown in FIG. 1 , or have different configurations from those shown in FIG. 1 . Each component shown in FIG. 1 may be implemented in hardware, software, or a combination thereof.

first embodiment

Please refer to FIG. 2 , which is a functional block diagram of a Webshell detection system 400 based on machine learning and dynamic and static analysis provided by the first embodiment of the present invention. The webshell detection system 400 based on machine learning and dynamic and static analysis includes a sample acquisition module 410 , a feature extraction module 420 and a model establishment module 430 .

The sample acquisition module 410 is used to acquire sample files. In this embodiment, the sample file includes a large number of Webshell samples and normal website samples, wherein the types of the Webshell samples include: Trojan horses written in various languages such as ASP Trojan horse, PHP Trojan horse, JSP Trojan horse, etc. It can be divided into one-sentence Trojan horse, image code, functional uploader, etc. Normal website samples are all kinds of CMS in PHP language, or the original code of the website to be detected, etc., which is not limited. Preferably, a large number of obtained sample files are stored in a database, and users can freely add Webshell samples and normal webpage codes collected by themselves. Depending on the detection environment, providing the original file code of the website as a positive sample can usually improve the accuracy of the model and reduce the false positive rate.

The feature extraction module 420 is used for extracting static features and dynamic features of the sample file.

In this embodiment, the feature extraction module 420 is used to perform dynamic and static analysis on a large number of sample files. As shown in FIG. 3 , the feature extraction module 420 specifically includes a static analysis module 421 and a dynamic analysis module 422 .

The static analysis module 421 is configured to perform static analysis on the sample file to obtain the static feature, wherein the static feature includes the document feature, basic function feature, and file behavior feature of the sample file. In this embodiment, the static analysis module 421 mainly analyzes the characters in the sample file, and counts the values of the sample file in multiple feature dimensions. Specifically, the document features may include but are not limited to: the number of words, the number of different words, the number of lines, the average number of words per line, the number of blank characters and spaces, the maximum word length, the number of comments, etc.; the basic function features may include But not limited to: character operation functions, sensitive function calls, number of system function calls, number of script blocks, maximum length of function parameters, encryption and decryption function calls, etc.; the file behavior characteristics may include but are not limited to: file operations, ftp operations, database operations, etc.

The dynamic analysis module 422 is configured to perform dynamic analysis on the sample file to obtain the dynamic characteristics, wherein the dynamic characteristics include file inclusion operation characteristics, sensitive function running characteristics, and sensitive character string characteristics. In this embodiment, the dynamic analysis module 422 is mainly for different programming languages, establishes a compilation environment or hook extension, monitors and combines the tag tracking of external input variables, and the black and white list mechanism to perform real-time dynamic detection of Webshell. Dynamic characteristics of sample files. In this embodiment, the features for dealing with Webshell confusion include: calculating the text entropy value, the number of invalid characters in the text, a sensitive character string generated by dynamic analysis and running, and a sensitive function.

The model building module 430 is configured to obtain a classification model according to the static feature, the dynamic feature and the machine learning algorithm, and the classification model analyzes the file to be detected and obtains a detection result.

In this embodiment, the user uploads the file to be detected into the system, the classification model can complete the Webshell detection of the file to be detected, obtain a classification result, and generate a detection report for the user to view.

In this embodiment, the model building module 430 is configured to use the machine learning algorithm to learn the static features and the dynamic features to obtain the classification model. Specifically, the model building module 430 first performs a normalization operation on the static features and the dynamic features to obtain a feature vector set, uses a machine learning algorithm to learn the feature vector set, and calculates to obtain a classification model. Preferably, in this embodiment, the machine learning algorithm is a collective learning method combining multiple classification algorithms, which may specifically include: random forest algorithm, decision tree algorithm, logic algorithm, and the like. The collective learning method combining multiple classification algorithms can improve the stability and robustness of the model, thereby improving the detection accuracy of the classification model.

It should be noted that, in this embodiment, after the classification model is established, the classification model can also be tested by using some data that has not been learned and so on, and then adjust the classification model according to the tested data, such as adjusting the proportion, number, and type of positive and negative samples in the sample file, so as to improve the accuracy of the classification model and realize the optimization of the classification model.

Further, the Webshell detection system 400 based on machine learning and dynamic and static analysis also includes a model update module 440, and the model update module 440 is used for when the to-be-detected file meets the Webshell feature, according to the to-be-detected file and the The sample files are re-machined to update the classification model.

In this embodiment, the user can use the webshell detection system 400 based on machine learning and dynamic and static analysis to perform webshell detection on unknown files (that is, files to be detected). The to-be-detected file is added to the malicious sample database, and machine learning is performed again together with the previous sample file to optimize and update the classification model. Refer to Figure 4 for the specific process of using the system to perform Webshell detection, including:

Step S101, acquiring a file to be detected.

Specifically, the user connects to the system and uploads the file to be detected, and the system obtains the file to be detected through the sample acquisition module 410 .

Step S102, extracting static features and dynamic features of the to-be-detected file.

In this embodiment, the system automatically performs dynamic and static feature extraction on the to-be-detected file through the feature extraction module 420 .

Step S103, using the classification model to analyze the to-be-detected file and obtain a detection result.

Specifically, after the feature extraction of the file to be detected is completed, the established classification model is used for detection to determine whether the file to be detected is a malicious file Webshell, the detection result is obtained, and then combined with the dynamic and static features extracted by the feature extraction module 420, Form a test report for the user to view. For example, the content displayed in the detection report may include: probability percentage that the file to be detected is a malicious file Webshell, extracted features (such as malicious functions, file operation behaviors, blacklist characters that appear), and the like.

Second Embodiment

Please refer to FIG. 5 , which is a schematic flowchart of a Webshell detection method based on machine learning and dynamic and static analysis provided by the second embodiment of the present invention. It should be noted that the Webshell detection method based on machine learning and dynamic and static analysis described in the embodiment of the present invention is not limited to the specific sequence shown in FIG. 5 and the following. The examples are the same. For a brief description, for the parts not mentioned in this embodiment, reference may be made to the corresponding content in the first embodiment. It should be understood that, in other embodiments, the order of some steps in the Webshell detection method based on machine learning and dynamic and static analysis of the present invention may be exchanged according to actual needs, or some steps may be omitted or deleted. The specific flow shown in FIG. 5 will be described in detail below.

Step S201, obtaining a sample file.

It can be understood that this step S201 may be performed by the above-mentioned sample acquisition module 410 .

Step S202, extracting static features and dynamic features of the sample file.

It can be understood that this step S202 can be performed by the feature extraction module 420 described above.

As shown in FIG. 6, in this embodiment, the step S202 specifically includes the following sub-steps:

Sub-step S2021, performing static analysis on the sample file to obtain the static feature, wherein the static feature includes the document feature, basic function feature, and file behavior feature of the sample file.

It can be understood that this step S2021 may be performed by the static analysis module 421 described above.

Sub-step S2022, dynamically analyze the sample file to obtain the dynamic feature, wherein the dynamic feature includes the file containing operation feature, sensitive function running feature, and sensitive character string feature.

It can be understood that this step S2022 may be performed by the dynamic analysis module 422 described above.

It should be noted that, in this embodiment, the order of the sub-steps S2021 and S2022 is not limited, and they may be performed simultaneously.

Step S203, a classification model is obtained according to the static feature, the dynamic feature and the machine learning algorithm, and the classification model analyzes the file to be detected and obtains a detection result.

It can be understood that this step S203 may be performed by the above-mentioned model establishment module 430 .

Step S204, when the to-be-detected file is confirmed to be a Webshell after being detected, machine learning is performed again according to the to-be-detected file and the sample file to update the classification model.

It can be understood that this step S204 may be performed by the above-mentioned model updating module 440 .

To sum up, in the Webshell detection method and system based on machine learning and dynamic and static analysis provided by the embodiments of the present invention, by acquiring sample files, static analysis and dynamic analysis are performed on the sample files to extract the static state of the sample files respectively. Features and dynamic features. According to the static features and the dynamic features, a machine learning algorithm is used to learn a classification model, and the classification model analyzes the file to be detected and obtains a detection result. Further, when the to-be-detected file is confirmed to be a Webshell after being detected, the to-be-detected file is added to the sample database, and machine learning is performed again together with the previous sample file to update the classification model. The embodiment of the present invention adopts a dynamic and static analysis method to extract features more comprehensively, and uses a machine learning algorithm combined with multiple classification algorithms to learn a large number of Webshell samples and normal webpage samples to form a classification model, the classification model is more stable, and the classification more precise. The use of machine learning algorithms can deal with complex classification calculations with multiple features, so that the features involved in detection are not limited to a single feature code. Using this classification model, users can effectively detect Webshells and their variants, predict new Webshells, and can better deal with text obfuscation methods, making up for the shortcomings of traditional signature matching detection methods.

It should be noted that, in this document, relational terms such as "first" and "second" etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these There is no such actual relationship or sequence between entities or operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention. It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

Claims (8)

1. a Webshell detection method based on machine learning and dynamic and static analysis, is characterized in that, the described Webshell detection method based on machine learning and dynamic and static analysis comprises: get sample files; Characters in the sample file are analyzed to obtain static features of the sample file, wherein the static features include document features, basic function features, and file behavior features of the sample file, and the document features include the number of words , the number of different words, the number of lines, the average number of words per line, the number of empty characters and spaces, the maximum word length, the number of comments, the basic function features include character operation functions, sensitive function calls, system function calls number, number of script blocks , the maximum length of function parameters, encryption and decryption function calls, and the file behavior characteristics include file operations, ftp operations, and database operations; For different programming languages, build a compilation environment or hook extension, monitor and combine the tag tracking of external input variables and the black and white list mechanism to perform real-time dynamic detection of Webshell, and summarize the dynamic characteristics of the sample file, wherein the dynamic characteristics Including file contains operation characteristics, sensitive function running characteristics, sensitive string characteristics; A classification model is obtained according to the static feature, the dynamic feature and the machine learning algorithm, and the classification model analyzes the file to be detected and obtains a detection result. 2. the Webshell detection method based on machine learning and dynamic and static analysis as claimed in claim 1, is characterized in that, the described step that obtains classification model according to described static feature, described dynamic feature and machine learning algorithm comprises: The machine learning algorithm is used to learn the static features and the dynamic features to obtain the classification model. 3 . The Webshell detection method based on machine learning and dynamic and static analysis according to claim 1 , wherein the machine learning algorithm is a collective learning method combining multiple classification algorithms. 4 . 4. the Webshell detection method based on machine learning and dynamic and static analysis as claimed in claim 1, is characterized in that, the described Webshell detection method based on machine learning and dynamic and static analysis also comprises: When the to-be-detected file is confirmed to be a Webshell after being detected, machine learning is performed again according to the to-be-detected file and the sample file to update the classification model. 5. a Webshell detection system based on machine learning and dynamic and static analysis, is characterized in that, the described Webshell detection system based on machine learning and dynamic and static analysis comprises: The sample acquisition module is used to acquire sample files; A feature extraction module for extracting static features and dynamic features of the sample file; the feature extraction module includes: a static analysis module for analyzing characters in the sample file to obtain the static features of the sample file , wherein the static features include document features, basic function features, and file behavior features of the sample file, and the document features include the number of words, the number of different words, the number of lines, the average number of words per line, the number of empty characters and spaces , the maximum word length, the number of comments, the basic function features include character operation functions, sensitive function calls, the number of system function calls, the number of script blocks, the maximum length of function parameters, encryption and decryption function calls, the file behavior features include file operations , ftp operation, database operation; dynamic analysis module, which is used to establish a compilation environment or hook extension for different programming languages, monitor and combine the tag tracking of external input variables, black and white list mechanism to perform real-time dynamic detection of Webshell, and summarize the results. The dynamic characteristics of the sample file, wherein the dynamic characteristics include file inclusion operation characteristics, sensitive function running characteristics, and sensitive character string characteristics; The model building module is used to obtain a classification model according to the static features, the dynamic features and the machine learning algorithm, and the classification model analyzes the files to be detected and obtains the detection results. 6. the Webshell detection system based on machine learning and dynamic and static analysis as claimed in claim 5, is characterized in that, described model establishment module is used to adopt described machine learning algorithm to study to described static feature and described dynamic feature , to obtain the classification model. 7 . The Webshell detection system based on machine learning and dynamic and static analysis as claimed in claim 5 , wherein the machine learning algorithm adopted by the model building module is a collective learning method combining multiple classification algorithms. 8 . 8. the Webshell detection system based on machine learning and dynamic and static analysis as claimed in claim 5, is characterized in that, the described Webshell detection system based on machine learning and dynamic and static analysis also comprises: The model updating module is configured to re-perform machine learning according to the to-be-detected file and the sample file to update the classification model when the to-be-detected file is confirmed to be a Webshell after being detected.

Images