CN113779581A - Robust detection method and system for lightweight high-precision malicious software identification model - Google Patents
Robust detection method and system for lightweight high-precision malicious software identification model Download PDFInfo
- Publication number
- CN113779581A CN113779581A CN202111077784.4A CN202111077784A CN113779581A CN 113779581 A CN113779581 A CN 113779581A CN 202111077784 A CN202111077784 A CN 202111077784A CN 113779581 A CN113779581 A CN 113779581A
- Authority
- CN
- China
- Prior art keywords
- model
- malicious
- scale
- student
- encoder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 43
- 238000012549 training Methods 0.000 claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 22
- 238000013140 knowledge distillation Methods 0.000 claims abstract description 17
- 230000003042 antagnostic effect Effects 0.000 claims abstract description 8
- 238000005516 engineering process Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 9
- 210000002569 neuron Anatomy 0.000 claims description 9
- 238000013526 transfer learning Methods 0.000 claims description 7
- 230000006835 compression Effects 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 3
- 238000002955 isolation Methods 0.000 claims description 3
- 238000007635 classification algorithm Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 244000035744 Hura crepitans Species 0.000 description 1
- 241000700605 Viruses Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000002155 anti-virotic effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
- G06F21/562—Static detection
- G06F21/563—Static detection by source code analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Virology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a robust detection method and a system for a lightweight high-precision malicious software identification model, which comprise the following steps: step 1: acquiring a byte file data set of application software, and visualizing the byte file data set into a gray scale map; step 2: training to generate an antagonistic network, generating a gray level graph of each category of malicious antagonistic samples, and adding the gray level graph into a malicious data set; and step 3: migrating an encoder of a convolution automatic encoder into a small-scale student model; and 4, step 4: adopting knowledge distillation to draw the knowledge of the large-scale teacher model into the student model; and 5: and predicting the class of the student model by using the finally obtained student model, namely detecting the result. According to the method, only byte files are processed, and then the end-to-end deep convolution model is adopted to automatically extract high-order features and judge potential modes, so that the problem that a classification algorithm highly depends on the integrity of feature space extracted by a fussy feature engineering is solved, and the real-time requirement of malicious detection can be met.
Description
Technical Field
The invention belongs to the field of software security technology protection, and relates to a robust detection method and system for a lightweight high-precision malicious software identification model.
Background
The popularization of various intelligent terminal devices indirectly causes the flooding of malicious software. Most of the detection technologies adopted by various antivirus vendors at present are signature-based methods, and rely on a retrieval technology for increasing the load of a virus library to detect more malicious software, but the detection technologies cannot detect unknown novel malicious software all the time. The infinite malware compels the malware detection technology to have strong generalization ability and robustness. The model can obtain better performance by depending on the sandbox to extract the dynamic behavior information of the software, but the offline detection method has no applicability to the real-time detection of the malicious software. The byte file of the software can be directly visualized into a gray-scale image form, and the method for identifying the malicious software by using the end-to-end convolutional neural network not only breaks away from the complicated characteristic engineering, but also has excellent performance.
The deep detection model scale of malware is generally proportional to detection accuracy and inversely proportional to temporal performance. The large-scale depth model can usually obtain higher precision, but the depth model has the characteristics of consuming a large amount of hardware resources in the equipment and having long reasoning time, and is not beneficial to the deployment on the light-weight equipment; small-scale models can be deployed on lightweight devices, but with far-ranging accuracy compared to that achieved by large-scale models. The popular application of the model must satisfy two requirements: the order is sufficiently small and the accuracy is sufficiently high. Therefore, how to effectively reduce the size of the high-precision large-scale model and simultaneously keep or reduce the precision of the large-scale model as small as possible is one of the key problems to be faced in the field.
Second, malware developers often bypass the detection of models by modifying program structure and adding unnecessary benign components to generate confusing samples. Therefore, how to effectively detect the malware malicious obfuscated sample is an important difficult problem to be solved urgently.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a robust detection method of a lightweight high-precision malicious software identification model;
the invention also provides a robust detection system of the lightweight high-precision malicious software identification model.
The invention aims to reduce the weight of a large-scale model with high precision to a great extent, keep higher precision and have strong discrimination capability on a mixed sample.
Interpretation of terms:
1. benign byte file: byte representation files of normal files or software collected from system or third-party software libraries;
2. malicious byte files: bytes of an executable file of malware represent the file;
3. ResNet101 model: migrating the pre-training model.
In order to achieve the purpose, the technical scheme adopted by the invention comprises the following steps:
a robust detection method for a lightweight high-precision malware identification model comprises the following steps:
step 1: acquiring a byte file data set of application software, wherein the byte file data set comprises benign byte files and multi-class malicious byte files, and all the byte files are visualized into a gray level graph;
step 2: training a plurality of convolution automatic encoders according to malicious categories, transferring a decoding part into a generation model through generation of an antagonistic network, training the generation of the antagonistic network, generating a gray level image of each category of malicious antagonistic samples, and adding the gray level image into a malicious data set;
and step 3: training a large-scale teacher model and a convolution automatic encoder, and migrating the encoder of the convolution automatic encoder into a small-scale student model;
and 4, step 4: by using a model compression technology based on knowledge distillation, the probability distribution and the real labels of multi-class prediction of a large-scale teacher model are respectively used as soft supervision and hard supervision information of a student model, and the loss values of the two are synthesized to be used as total loss to feed back the student model;
and 5: and predicting the class of the student model by using the finally obtained student model, namely detecting the result.
According to the present invention, the specific implementation process of visualizing the byte file into the gray-scale map in step 1 includes: taking every two hexadecimal numbers as a pixel point of the gray level image, wherein the decimal value is the pixel value; setting the width w, converting all bytes in the byte file into a matrix form in a row-by-row tiling mode, and further visualizing the matrix form into a gray scale image.
According to the invention, the specific implementation process of the step 2 comprises the following steps:
step 2.1: training a plurality of convolution automatic encoders in an isolation mode according to the category of the malicious software, and performing feedback by using a reconstruction error through reconstructing an original input image;
step 2.2: migrating an encoder of a convolution automatic encoder into a generation model of a generative confrontation network, and adopting a one-dimensional random number z conforming to standard normal distribution N (0,1) as the input of the generation model for generating sample data G (z);
the generated sample data G (z) and the original image x are used as the input of a discrimination model D in the generative confrontation network, the type judgment is carried out, and the objective function of the generative confrontation network is shown as the formula (I):
d (x) represents the probability that the discrimination model D classifies the original image x as true;
and generating countermeasure samples of each malicious category as an expanded tagged malicious sample set by using a generative model of the generative countermeasure network.
Further preferably, the convolutional automatic encoder comprises an encoder and a decoder, wherein the encoder is formed by stacking a plurality of transposed convolutional layers, a pooling layer, a BN layer and a single fully-connected layer, and the decoder is formed by stacking a plurality of convolutional layers, pooling layers and BN layers; wherein the BN layer follows each convolution and pooling operation;
given an input image X ∈ X ═ X { (X)0,x1,x2...,xmThe encoder E compresses x into compressed data f (x), the decoder reconstructs the compressed data f (x) into a reconstructed image x', and the operation process of the convolution automatic encoder is shown in formulas (II) and (III):
f(x)＝s(Wx+b)(Ⅱ)
x′＝g(f(x))＝s(W′f(x)+b′)(Ⅲ)
in the formulas (II) and (III), s () is an activation function, W and W 'are weight matrix sets, b and b' are bias value sets, and g () is a deconvolution mapping function;
the mean square error is used as the difference value of the original image x and the reconstructed image x' to perform feedback training on the convolution automatic encoder, and the reconstruction error e is shown as the formula (IV):
in the formula (IV), m and n are the height and width of the image respectively, and xij、x′ijThe pixel values of the corresponding positions of the real image x and the reconstructed image x' are respectively.
Further preferably, a one-dimensional random number z conforming to a standard normal distribution N (0,1) is used as a prior distribution of the generator in the original image x, so as to implement mapping x ″ on the class data space by generating a model;
constructing a discrimination model of a full-connection layer which has the same structure as an encoder in a convolution automatic encoder and the tail part of which is added with a layer of neuron number which is the total category number n, and taking an original image x and a generated image x' as input to carry out category discrimination;
according to the invention, the specific implementation process of step 3 preferably comprises the following steps:
step 3.1: the original benign and malicious gray level images generated in the step 1 and the gray level images of the various malicious countermeasure samples generated in the step 2 are sent to a large-scale teacher model together for multi-class supervised training;
step 3.2: the original benign and malicious gray level images generated in the step 1 and the gray level images of various types of malicious countermeasure samples generated in the step 2 are sent to a convolution automatic encoder together for self-supervision training;
step 3.3: the method is characterized in that an encoder in an automatic encoder which is trained is used as a student model, a dense layer is added at the tail of the student model, multi-classification is facilitated, and the number of neurons in the dense layer is the total number n of classes and is used for multi-classification.
Preferably, the number of neurons in the last full connection layer of the existing ResNet101 model is modified into the total category number n, so that the large-scale teacher model is obtained.
Preferably, in step 4, the KL divergence is used as a measure of the difference between the probability distribution of each class predicted by the large-scale teacher model and the probability distribution of each class of the sample corresponding to the student model prediction, and the difference value between the two is shown as the following formula (v):
in the formula (V), p is the probability distribution of each class of the corresponding sample predicted by the student model, and q is the probability distribution of each class predicted by the teacher model; p is a radical ofi、qiThe probability value of predicting the input sample into the ith category for the student model and the teacher model respectively, and n is the total category number.
Setting a hyper-parameter tau for softening probability distribution of various types of corresponding samples of a large-scale teacher model and a student model, and setting a hyper-parameter alpha for adjusting influence factors of soft and hard supervision on total loss, wherein the influence factors are respectively shown in formulas (VI) and (VII):
L＝αKL(p,q)+(1-α)CE(y,p)(Ⅶ)
in the formulae (VI) and (VII),
and expressing the predicted value of the ith category, p is the predicted probability distribution of the student model for each category, and y is a real label.
KL is KL divergence between large-scale teacher model prediction distribution and various types of probability distribution of samples corresponding to student model prediction, and CE is cross entropy of prediction of the student models and real labels;
preferably, in step 4, the sum of the loss values of the two is used as the total loss to feed back the student model, which means that: and setting a hyper-parameter alpha, wherein alpha is used as a soft supervision loss corresponding weight, 1-alpha is used as a hard supervision loss corresponding weight, and the weighted sum of the loss values of the soft supervision and the hard supervision is the total loss of the student model.
A robust detection system of a lightweight high-precision malware identification model is used for operating a robust detection method of the lightweight high-precision malware identification model and comprises a sample generation module, a knowledge distillation module and a detection module;
the sample generation module is to: generating a corresponding category malicious sample data set by using a convolution automatic encoder, a generated countermeasure network and a transfer learning idea; the knowledge distillation module is used for: adopting knowledge distillation to draw the knowledge of the large-scale teacher model into the student model; the detection module is used for: and predicting unknown samples by adopting the trained student model.
The invention has the beneficial effects that:
1. the invention generally uses an automatic encoder and transfer learning to easily make the model quickly converge to a better state.
2. According to the method, only byte files are processed, and then the end-to-end deep convolution model is adopted to automatically extract high-order features and judge potential modes, so that the problem that a classification algorithm highly depends on the integrity of feature space extracted by a fussy feature engineering is solved, and the real-time requirement of malicious detection can be met.
3. And the GAN is used for generating data of all samples according to the types, so that the problem of huge samples required by a teacher model is solved on one hand, and the screening capability of the model on unknown confusing samples is improved by the generated confrontation samples on the other hand.
4. By adopting a knowledge distillation-based model compression technology, knowledge of a large-scale teacher model is extracted into a small-scale student model, so that the hardware resource consumption is reduced, the reasoning speed is improved, and the feasibility is injected for the deployment of the teacher model on small-scale terminal equipment.
Drawings
FIG. 1 is a flow diagram of a robust detection method for a lightweight high-accuracy malware identification model;
FIG. 2 is an overall architecture diagram of a robust detection method for a lightweight high-precision malware identification model;
FIG. 3 is an exemplary illustration of a byte file visualized as a grayscale map;
FIG. 4 is a schematic structural diagram of a malicious countermeasure sample generation module of each category;
FIG. 5 is a schematic diagram of an example of training of a generative confrontation network;
fig. 6 is a schematic diagram of a network structure of a convolutional auto-encoder.
Detailed Description
The invention is further defined in the following, but not limited to, the figures and examples in the description.
Example 1
A robust detection method for a lightweight high-precision malware identification model is shown in fig. 1 and 2, and comprises the following steps:
step 1: acquiring a byte file data set of application software, wherein the byte file data set comprises benign byte files and multi-class malicious byte files, and all the byte files are visualized into a gray level graph; as shown in fig. 3, the specific implementation process for visualizing the byte file into the gray scale map includes: taking every two hexadecimal numbers (eight binary digits) as a pixel point of the gray scale image, wherein decimal values (0-255) are pixel values; setting the width w, converting all bytes in the byte file into a matrix form in a row-by-row tiling mode, and further visualizing the matrix form into a gray scale image.
Step 2: as shown in fig. 4, a plurality of convolutional automatic encoders are trained according to malicious categories, a decoding portion is migrated into a generation model by generating a countermeasure network, the countermeasure network is generated by training, a gray-scale map of malicious countermeasure samples of each category is generated, and the gray-scale map is added to a malicious data set; the specific implementation process comprises the following steps:
step 2.1: training a plurality of Convolutional Automatic Encoders (CAEs) according to the category of the malicious software in an isolation manner, and performing feedback by using a reconstruction error through reconstructing an original input image; the convolutional automatic encoder comprises an encoder and a decoder, as shown in fig. 6, the encoder is formed by stacking a plurality of transposed convolutional layers, a pooling layer, a BN (batch norm) layer and a single fully-connected layer, and the decoder is formed by stacking a plurality of convolutional layers, pooling layers and BN layers; wherein the BN layer follows each convolution and pooling operation;
given an input image X ∈ X ═ X { (X)0,x1,x2...,xmThe encoder E compresses x into compressed data f (x), the decoder reconstructs the compressed data f (x) into a reconstructed image x', and the operation process of the convolution automatic encoder is shown in formulas (II) and (III):
f(x)＝s(Wx+b)(Ⅱ)
x′＝g(f(x))＝s(W′f(x)+b′)(Ⅲ)
in the formulas (II) and (III), s () is an activation function, W and W 'are weight matrix sets, b and b' are bias value sets, and g () is a deconvolution mapping function;
the Mean Square Error (MSE) is used as the difference value of the original image x and the reconstructed image x' to perform feedback training on the convolution automatic encoder, and the reconstruction error e is shown as the formula (IV):
in the formula (IV), m and n are the height and width (rows and columns) of the image respectively, and xij、x′ijThe pixel values of the corresponding positions of the real image x and the reconstructed image x' are respectively.
Step 2.2: migrating an encoder of a convolution automatic encoder into a generation model of a generation countermeasure network (GAN), and adopting a one-dimensional random number z conforming to a standard normal distribution N (0,1) as an input of the generation model for generating sample data G (z);
the generated sample data G (z) and the original image x are used as the input of a discrimination model D in the generative confrontation network, the type judgment is carried out, and the objective function of the generative confrontation network is shown as the formula (I):
d (x) represents the probability that the discrimination model D classifies the original image x as true;
and generating countermeasure samples of each malicious category as an expanded tagged malicious sample set by using a generative model of the generative countermeasure network.
As shown in fig. 5, a one-dimensional random number z conforming to a standard normal distribution N (0,1) is used as a prior distribution of a generator in an original image x, so as to implement mapping x ″ on the class data space by a generation model;
constructing a discrimination model of a full-connected layer which has the same structure as the encoder in the convolution automatic encoder shown in FIG. 6 and the tail part of which is added with a layer of complete-connected layer with neuron number as the total category number n, and performing category discrimination by taking an original image x and a generated image x' as input;
both the generative model and the discriminative model benefit from a binary task of whether to distinguish x from x ", wherein training of the generative model benefits from the discriminative model to classify the generative image x" into a real image, and training of the discriminative model benefits from distinguishing x from x ", which are in an interactive game.
After training iteration for a certain number of times, the gray image X 'generated by each generated model is saved as the expanded gray pattern sample data X'.
The large-scale teacher model is a large-scale convolutional neural network formed by stacking a plurality of convolutional layers, pooling layers, linear layers and the like; and modifying the number of neurons in the last full connecting layer of the existing ResNet101 model into the total class number n to obtain the large-scale teacher model. The invention has a lightweight structure as a whole, and large network models with good performances can be used as the preference of the transfer learning.
And step 3: training a large-scale teacher model and a convolution automatic encoder, and migrating the encoder of the convolution automatic encoder into a small-scale student model; the specific implementation process comprises the following steps:
step 3.1: the original benign and malicious gray level images generated in the step 1 and the gray level images of the various malicious countermeasure samples generated in the step 2 are sent to a large-scale teacher model together for multi-class supervised training;
the multi-class supervised training refers to: 1 benign category and n-1 malicious category are set, and each category data comprises original data and generated data. And performing feedback training according to the difference value between the label and the forecast.
Step 3.2: the original benign and malicious gray level images generated in the step 1 and the gray level images of various types of malicious countermeasure samples generated in the step 2 are sent to a convolution automatic encoder together for self-supervision training;
the self-supervision training refers to: and (5) taking the input sample as a training target, and performing feedback training according to the difference between the reconstructed image and the input image.
Step 3.3: the encoder in the trained automatic encoder is used as a student model (small-scale detection model), a dense layer is added at the tail of the student model to facilitate multi-classification, and the number of neurons in the dense layer is the total number n of classes and is used for multi-classification.
And 4, step 4: by using a model compression technology based on knowledge distillation, the probability distribution and the real labels of multi-class prediction of a large-scale teacher model are respectively used as soft supervision and hard supervision information of a student model, and the loss values of the two are synthesized to be used as total loss to feed back the student model; and setting a hyper-parameter alpha, wherein alpha is used as a soft supervision loss corresponding weight, 1-alpha is used as a hard supervision loss corresponding weight, and the weighted sum of the loss values of the soft supervision and the hard supervision is the total loss of the student model.
In step 4, the KL divergence (the contrast entropy) is used for measuring the difference between the probability distribution of each category predicted by the large-scale teacher model and the probability distribution of each category of the corresponding sample predicted by the student model, and the difference value of the two is shown as the following formula (V):
in the formula (V), p is the probability distribution of each class of the corresponding sample predicted by the student model, and q is the probability distribution of each class predicted by the teacher model; p is a radical ofi、qiThe probability value of predicting the input sample into the ith category for the student model and the teacher model respectively, and n is the total category number.
Considering that the student model learns the soft knowledge from the large-scale teacher model, if the teacher model is over-determined for the sample belonging category, that is, the determination value for the sample belonging category is extremely high, the student model is difficult to learn the soft knowledge of the teacher model.
Setting a hyper-parameter tau for softening probability distribution of various types of corresponding samples of a large-scale teacher model and a student model, and setting a hyper-parameter alpha for adjusting influence factors of soft and hard supervision on total loss, wherein the influence factors are respectively shown in formulas (VI) and (VII):
L＝αKL(p,q)+(1-α)CE(y,p)(Ⅶ)
in the formulae (VI) and (VII),
and expressing the predicted value of the ith category, p is the predicted probability distribution of the student model for each category, and y is a real label.
KL is KL divergence between large-scale teacher model prediction distribution and various types of probability distribution of samples corresponding to student model prediction, and CE is cross entropy of prediction of the student models and real labels;
the invention uses the model compression technology based on knowledge distillation, and on the basis, the student models adopt the technologies of pruning optimization, quantitative acceleration and the like, which can be used as supplements of the scheme.
The error and loss functions used in the gradient back propagation stage of each model in the present invention are not limited to the above-mentioned schemes, and the loss functions such as L1, L2, cross entropy, mean square error, and root mean square error can be used instead of or as an alternative to the above-mentioned loss functions.
And 5: and predicting the class of the student model by using the finally obtained student model, namely detecting the result.
In this embodiment, for prediction of an unknown software sample, a byte file of the unknown software sample is visualized as a gray-scale map, and then the gray-scale map is sent to a student model to perform probability prediction of each category, and the category to which the unknown software sample belongs is output.
The invention generally uses an automatic encoder as a pre-training stage of other network models, and initializes other model parameters by using transfer learning. Compared with a model without a pre-training stage, the method is easy to make the model quickly converge to a better state.
The invention is based on static analysis to process only a single data source of the software, namely byte files. The method can be visualized as a gray level image and directly sent into an end-to-end deep convolution model, so that the problem that a classification algorithm highly depends on the integrity of a feature space extracted by a fussy feature engineering is solved, and the real-time requirement of malicious detection can be met. And secondly, generating data of all samples according to categories by using the GAN, so that the problem of a huge amount of samples required by a teacher model is solved on one hand, and the discrimination capability of the model on unknown confusing samples is improved by the generated confrontation samples on the other hand. And thirdly, adopting a knowledge distillation-based model compression technology to draw the knowledge of the large-scale teacher model into the small-scale student model. The detection precision is reduced as little as possible, and simultaneously the model scale is obviously reduced, so that the hardware resource consumption is reduced, the reasoning speed is improved, and the feasibility is injected for the deployment of the model on small-sized terminal equipment.
Example 2
A robust detection system of a lightweight high-precision malware identification model is used for operating the robust detection method of the lightweight high-precision malware identification model in embodiment 1, and comprises a sample generation module, a knowledge distillation module and a detection module;
the sample generation module is to: generating a corresponding category malicious sample data set by using a convolution automatic encoder, a generated countermeasure network and a transfer learning idea; the knowledge distillation module is used for: adopting knowledge distillation to draw the knowledge of the large-scale teacher model into the student model; the detection module is used for: and predicting unknown samples by adopting the trained student model.
The method generally uses the idea of transfer learning, so that the model can be converged to a higher precision more quickly. In addition, the generated countermeasure sample is used as an expanded label data set, on one hand, data enhancement is favorable for improving model precision, and on the other hand, the countermeasure sample is used for training and can enhance the discrimination capability of the model on the confusion sample. And thirdly, a knowledge distillation lightweight large-scale model is adopted, so that feasibility is provided for deployment of lightweight equipment.
Claims (10)
1. A robust detection method for a lightweight high-precision malicious software identification model is characterized by comprising the following steps:
step 1: acquiring a byte file data set of application software, wherein the byte file data set comprises benign byte files and multi-class malicious byte files, and all the byte files are visualized into a gray level graph;
step 2: training a plurality of convolution automatic encoders according to malicious categories, transferring a decoding part into a generation model through generation of an antagonistic network, training the generation of the antagonistic network, generating a gray level image of each category of malicious antagonistic samples, and adding the gray level image into a malicious data set;
and step 3: training a large-scale teacher model and a convolution automatic encoder, and migrating the encoder of the convolution automatic encoder into a small-scale student model;
and 4, step 4: by using a model compression technology based on knowledge distillation, the probability distribution and the real labels of multi-class prediction of a large-scale teacher model are respectively used as soft supervision and hard supervision information of a student model, and the loss values of the two are synthesized to be used as total loss to feed back the student model;
and 5: and predicting the class of the student model by using the finally obtained student model, namely detecting the result.
2. The robust detection method for the lightweight high-precision malware identification model according to claim 1, wherein the specific implementation process of visualizing the byte file into the gray-scale map in step 1 comprises: taking every two hexadecimal numbers as a pixel point of the gray level image, wherein the decimal value is the pixel value; setting the width w, converting all bytes in the byte file into a matrix form in a row-by-row tiling mode, and further visualizing the matrix form into a gray scale image.
3. The robust detection method for the lightweight high-precision malware identification model according to claim 1, wherein the specific implementation process of step 2 comprises:
step 2.1: training a plurality of convolution automatic encoders in an isolation mode according to the category of the malicious software, and performing feedback by using a reconstruction error through reconstructing an original input image;
step 2.2: migrating an encoder of a convolution automatic encoder into a generation model of a generative confrontation network, and adopting a one-dimensional random number z conforming to standard normal distribution N (0,1) as the input of the generation model for generating sample data G (z);
the generated sample data G (z) and the original image x are used as the input of a discrimination model D in the generative confrontation network, the type judgment is carried out, and the objective function of the generative confrontation network is shown as the formula (I):
d (x) represents the probability that the discrimination model D classifies the original image x as true;
and generating countermeasure samples of each malicious category as an expanded tagged malicious sample set by using a generative model of the generative countermeasure network.
4. The robust detection method of a lightweight high-precision malware identification model according to claim 3, wherein the convolutional automatic encoder comprises an encoder and a decoder, the encoder is formed by stacking a plurality of transposed convolutional layers, a pooling layer, a BN layer and a single fully-connected layer, and the decoder is formed by stacking a plurality of convolutional layers, a pooling layer and a BN layer; wherein the BN layer follows each convolution and pooling operation;
given an input image X ∈ X ═ X { (X)0,x1,x2...,xmThe encoder compresses x into compressed data f (x), the decoder reconstructs the compressed data f (x) into a reconstructed image x', and the operation process of the convolution automatic encoder is shown in formulas (II) and (III):
f(x)＝s(Wx+b) (Ⅱ)
x′＝g(f(x))＝s(W′f(x)+b′) (Ⅲ)
in the formulas (II) and (III), s () is an activation function, W and W 'are weight matrix sets, b and b' are bias value sets, and g () is a deconvolution mapping function;
the mean square error is used as the difference value of the original image x and the reconstructed image x' to perform feedback training on the convolution automatic encoder, and the reconstruction error e is shown as the formula (IV):
in the formula (IV), m and n are the height and width of the image respectively, and xij、x′ijThe pixel values of the corresponding positions of the real image x and the reconstructed image x' are respectively.
5. The robust detection method for the lightweight high-precision malware identification model according to claim 4, wherein a one-dimensional random number z conforming to a standard normal distribution N (0,1) is used as a prior distribution of a generator in an original image x, so as to realize mapping x ″ of the class data space by generating the model;
and constructing a discrimination model of a full-connection layer which has the same structure as an encoder in the convolution automatic encoder and the tail part of which is added with a layer of neuron number which is the total category number n, and taking the original image x and the generated image x' as input to carry out category discrimination.
6. The robust detection method for the lightweight high-precision malware identification model according to claim 1, wherein the specific implementation process of step 3 comprises:
step 3.1: the original benign and malicious gray level images generated in the step 1 and the gray level images of the various malicious countermeasure samples generated in the step 2 are sent to a large-scale teacher model together for multi-class supervised training;
step 3.2: the original benign and malicious gray level images generated in the step 1 and the gray level images of various types of malicious countermeasure samples generated in the step 2 are sent to a convolution automatic encoder together for self-supervision training;
step 3.3: the method is characterized in that an encoder in an automatic encoder which is trained is used as a student model, a dense layer is added at the tail of the student model, multi-classification is facilitated, and the number of neurons in the dense layer is the total number n of classes and is used for multi-classification.
7. The robust detection method for the lightweight high-precision malware identification model according to claim 1, wherein the large-scale teacher model is obtained by modifying the number of neurons in the last full-link layer of the existing ResNet101 model to the total number n of classes.
8. The robust detection method for the lightweight high-precision malware identification model according to claim 1, wherein in step 4, KL divergence is used as a measure of the difference between the probability distribution of each class predicted by the large-scale teacher model and the probability distribution of each class of a sample corresponding to the student model prediction, and the difference value between the two is shown as formula (v):
in the formula (V), p is the class summary of the corresponding sample predicted by the student modelRate distribution, q is the probability distribution of each category predicted by the teacher model; p is a radical ofi、qiPredicting the input sample into the probability value of the ith category for the student model and the teacher model respectively, wherein n is the total number of categories;
setting a hyper-parameter tau for softening probability distribution of various types of corresponding samples of a large-scale teacher model and a student model, and setting a hyper-parameter alpha for adjusting influence factors of soft and hard supervision on total loss, wherein the influence factors are respectively shown in formulas (VI) and (VII):
L＝αKL(p,q)+(1-α)CE(y,p) (Ⅶ)
in the formulae (VI) and (VII),
expressing the predicted value of the ith category, wherein p is the predicted probability distribution of the student model for each category, and y is a real label; KL is KL divergence between large-scale teacher model prediction distribution and various types of probability distribution of samples corresponding to student model prediction, and CE is cross entropy of prediction of the student models and real labels;
9. the robust detection method for the lightweight high-precision malware identification model according to claim 1, wherein in step 4, the step of feeding back the student model by integrating the loss values of the two as the total loss is: and setting a hyper-parameter alpha, wherein alpha is used as a soft supervision loss corresponding weight, 1-alpha is used as a hard supervision loss corresponding weight, and the weighted sum of the loss values of the soft supervision and the hard supervision is the total loss of the student model.
10. A robust detection system of a lightweight high-precision malware identification model is used for operating the robust detection method of the lightweight high-precision malware identification model as claimed in any one of claims 1 to 9, and is characterized by comprising a sample generation module, a knowledge distillation module and a detection module;
the sample generation module is to: generating a corresponding category malicious sample data set by using a convolution automatic encoder, a generated countermeasure network and a transfer learning idea; the knowledge distillation module is used for: adopting knowledge distillation to draw the knowledge of the large-scale teacher model into the student model; the detection module is used for: and predicting unknown samples by adopting the trained student model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111077784.4A CN113779581B (en) | 2021-09-15 | 2021-09-15 | Robust detection method and system for lightweight high-precision malicious software identification model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111077784.4A CN113779581B (en) | 2021-09-15 | 2021-09-15 | Robust detection method and system for lightweight high-precision malicious software identification model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113779581A true CN113779581A (en) | 2021-12-10 |
| CN113779581B CN113779581B (en) | 2022-08-26 |
Family
ID=78843910
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111077784.4A Active CN113779581B (en) | 2021-09-15 | 2021-09-15 | Robust detection method and system for lightweight high-precision malicious software identification model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113779581B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115017948A (en) * | 2022-06-02 | 2022-09-06 | 电子科技大学 | Lightweight processing method of intelligent signal detection and identification model |
| CN116916318A (en) * | 2023-07-19 | 2023-10-20 | 西华师范大学 | Lightweight intrusion detection method based on separable convolution for Internet of things equipment |
| CN116910752A (en) * | 2023-07-17 | 2023-10-20 | 重庆邮电大学 | Malicious code detection method based on big data |
| CN117077141A (en) * | 2023-10-13 | 2023-11-17 | 国网山东省电力公司鱼台县供电公司 | Smart power grid malicious software detection method and system |
| CN117610002A (en) * | 2024-01-22 | 2024-02-27 | 南京众智维信息科技有限公司 | Multi-mode feature alignment-based lightweight malicious software threat detection method |
| CN118368150A (en) * | 2024-06-20 | 2024-07-19 | 山东省计算中心（国家超级计算济南中心） | Knowledge distillation-based light malicious encryption flow detection method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110826059A (en) * | 2019-09-19 | 2020-02-21 | 浙江工业大学 | Method and device for defending black box attack facing malicious software image format detection model |
| CN112116030A (en) * | 2020-10-13 | 2020-12-22 | 浙江大学 | Image classification method based on vector standardization and knowledge distillation |
| CN112560034A (en) * | 2020-12-11 | 2021-03-26 | 宿迁学院 | Malicious code sample synthesis method and device based on feedback type deep countermeasure network |
-
2021
- 2021-09-15 CN CN202111077784.4A patent/CN113779581B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110826059A (en) * | 2019-09-19 | 2020-02-21 | 浙江工业大学 | Method and device for defending black box attack facing malicious software image format detection model |
| CN112116030A (en) * | 2020-10-13 | 2020-12-22 | 浙江大学 | Image classification method based on vector standardization and knowledge distillation |
| CN112560034A (en) * | 2020-12-11 | 2021-03-26 | 宿迁学院 | Malicious code sample synthesis method and device based on feedback type deep countermeasure network |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115017948A (en) * | 2022-06-02 | 2022-09-06 | 电子科技大学 | Lightweight processing method of intelligent signal detection and identification model |
| CN116910752A (en) * | 2023-07-17 | 2023-10-20 | 重庆邮电大学 | Malicious code detection method based on big data |
| CN116910752B (en) * | 2023-07-17 | 2024-03-08 | 重庆邮电大学 | Malicious code detection method based on big data |
| CN116916318A (en) * | 2023-07-19 | 2023-10-20 | 西华师范大学 | Lightweight intrusion detection method based on separable convolution for Internet of things equipment |
| CN117077141A (en) * | 2023-10-13 | 2023-11-17 | 国网山东省电力公司鱼台县供电公司 | Smart power grid malicious software detection method and system |
| CN117610002A (en) * | 2024-01-22 | 2024-02-27 | 南京众智维信息科技有限公司 | Multi-mode feature alignment-based lightweight malicious software threat detection method |
| CN117610002B (en) * | 2024-01-22 | 2024-04-30 | 南京众智维信息科技有限公司 | Multi-mode feature alignment-based lightweight malicious software threat detection method |
| CN118368150A (en) * | 2024-06-20 | 2024-07-19 | 山东省计算中心（国家超级计算济南中心） | Knowledge distillation-based light malicious encryption flow detection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113779581B (en) | 2022-08-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113779581B (en) | Robust detection method and system for lightweight high-precision malicious software identification model | |
| Singh et al. | PlantDoc: A dataset for visual plant disease detection | |
| Bi et al. | APDC-Net: Attention pooling-based convolutional network for aerial scene classification | |
| CN110991549A (en) | Countermeasure sample generation method and system for image data | |
| Zhao et al. | A malware detection method of code texture visualization based on an improved faster RCNN combining transfer learning | |
| CN110390347B (en) | Condition-guided countermeasure generation test method and system for deep neural network | |
| Oliva et al. | Multilevel thresholding by fuzzy type II sets using evolutionary algorithms | |
| JP2022509030A (en) | Image processing methods, devices, equipment and storage media | |
| CN114419468A (en) | Paddy field segmentation method combining attention mechanism and spatial feature fusion algorithm | |
| CN114896594B (en) | Malicious code detection device and method based on image feature multi-attention learning | |
| CN114329031A (en) | Fine-grained bird image retrieval method based on graph neural network and deep hash | |
| CN112711985A (en) | Fruit identification method and device based on improved SOLO network and fruit picking robot | |
| CN114972886A (en) | Image steganography analysis method | |
| CN118212572A (en) | Road damage detection method based on improvement YOLOv7 | |
| CN114387454A (en) | Self-supervision pre-training method based on region screening module and multi-level comparison | |
| CN115292538A (en) | Map line element extraction method based on deep learning | |
| Yu et al. | Artificial intelligence in paleontology | |
| CN116524357A (en) | High-voltage line bird nest detection method, model training method, device and equipment | |
| Devi et al. | Classification of satellite images using perceptron neural network | |
| Prasvita et al. | Deep Learning Model for Automatic Detection of Oil Palm Trees in Indonesia with YOLO-V5 | |
| Chen | Using satellite imagery to automate building damage assessment: A case study of the xbd dataset | |
| Yuan et al. | An efficient attention based image adversarial attack algorithm with differential evolution on realistic high-resolution image | |
| CN114861178A (en) | Malicious code detection engine design method based on improved B2M algorithm | |
| He et al. | Relationship prior and adaptive knowledge mimic based compressed deep network for aerial scene classification | |
| CN113505783B (en) | Oracle word recognition method and oracle word recognition device based on less learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221214 Address after: 250014 No. 19, ASTRI Road, Lixia District, Shandong, Ji'nan Patentee after: SHANDONG COMPUTER SCIENCE CENTER(NATIONAL SUPERCOMPUTER CENTER IN JINAN) Patentee after: Qilu University of Technology Address before: 250014 No. 19, ASTRI Road, Ji'nan, Shandong Patentee before: SHANDONG COMPUTER SCIENCE CENTER(NATIONAL SUPERCOMPUTER CENTER IN JINAN) |
|
| TR01 | Transfer of patent right | ||
| CP03 | Change of name, title or address |
Address after: 250014 No. 19, ASTRI Road, Lixia District, Shandong, Ji'nan Patentee after: SHANDONG COMPUTER SCIENCE CENTER(NATIONAL SUPERCOMPUTER CENTER IN JINAN) Country or region after: China Patentee after: Qilu University of Technology (Shandong Academy of Sciences) Address before: 250014 No. 19, ASTRI Road, Lixia District, Shandong, Ji'nan Patentee before: SHANDONG COMPUTER SCIENCE CENTER(NATIONAL SUPERCOMPUTER CENTER IN JINAN) Country or region before: China Patentee before: Qilu University of Technology |
|
| CP03 | Change of name, title or address |