KR102258910B1 - Method and System for Effective Detection of Ransomware using Machine Learning based on Entropy of File in Backup System - Google Patents

Abstract

An effective ransomware detection method and system using machine learning based on the entropy of a file in a backup system is presented. In one aspect, the effective ransomware detection method using machine learning based on the entropy of a file in the backup system proposed by the present invention measures the entropy for each user and each file format through the backup system, and each measured entropy The step of deriving an entropy reference value for detecting a file infected with ransomware using machine learning based on Measuring the entropy of the file synchronized from the detection module to the backup system, and detecting the file infected with the ransomware by comparing the measured entropy of the file synchronized to the backup system with the entropy reference value transmitted from the backup system.

Description

{Method and System for Effective Detection of Ransomware using Machine Learning based on Entropy of File in Backup System}

The present invention relates to an effective ransomware detection method and apparatus using machine learning based on the entropy of a file in a backup system.

With the advent of big data and cloud services, user data is one of the important factors, and ensuring data reliability and integrity is one of the core technologies of these services. The attacker took the data as hostage and demanded money, paying attention to the fact that these services value data, which is called ransomware. Ransomware is a compound word of ransom and malware, and after infiltrating the system, it encrypts the user's files to prevent the user from reading or locks the system. After taking control of the system in this way, the attacker demands money in exchange for decrypting encrypted files or unlocking the system.

This ransomware did not appear recently, but first appeared as "PC CYBORG/AIDS information Trojan" in 1989. The ransomware that appeared at this time was not produced much by the malware creators and was ignored, and was abandoned for almost 15 years. However, with the recent Internet use, the advent of e-commerce, and the advent of big data and cloud services, ransomware was reborn in 2005. These ransomwares include PC BYBORG, Reveton, CryZip, May Archieve, FAVEAC, FastBsod, CryptoLocker, GPCoder, SimpleLocker, TeslaCrypt, CryptorBit, KeRanger, and CryptoWall.

Despite the fact that many ransomware appear and cause serious damage, one of the reasons why it is difficult to protect a system from ransomware is that it is difficult to detect and prevent unknown ransomware. First of all, after the ransomware infiltrates and infects the system, the user does not have the decryption key, so the system and files cannot be recovered. In addition, it is difficult for users or anti-virus software to detect ransomware by concealing their own infiltration and system control. One of the reasons is that it is possible for attackers to hide their identity by using the anonymous network Tor netowrk. In addition, even if the attacker pays the requested amount, it causes secondary damage by failing to decrypt data or unlock the system.

If we take a closer look at Ransomware, it is classified into Non-Cryptographic Ransomware (NCR) and Cryptographic Ransomware (CGR). Unencrypted ransomware disrupts user interaction with the system by locking the screen or modifying the MBR or partition table. Encrypting ransomware uses a cryptographic algorithm to encrypt files, preventing even legitimate users from reading or executing files. Private-key Cryptosystem Ransomware (PrCR) and Public Key Encryption Ransomware (Public -key Cryptosystem Ransomware, PuCR). Secret key encryption ransomware encrypts data using secret key-based cryptographic algorithms such as DES and AES. Public key encryption ransomware encrypts data using a public key-based cryptographic algorithm such as RSA. Serious damage occurred due to various ransomware like this, and according to ZDNet, the attacker generated about $27 million in revenue. Therefore, it is urgent to detect ransomware early and to restore files or systems infected with ransomware.

In order to solve this problem, various methods for detecting and preventing the system from being infected with ransomware have been studied, and these methods are classified into four; File-based detection, system-based behavior detection, resource-based behavior detection, connection-based behavior detection. The file-based detection method is a method of detecting a malicious signature in a file of a specific format. However, this method has a limitation in that it cannot detect unknown ransomware. The system-based behavior detection method detects ransomware by verifying the integrity of files and directories, and blocks ransomware by monitoring malicious behavior of the system. This method has a disadvantage in that the false positive rate and false positive rate are high, and it takes a long time for integrity verification and behavior monitoring. The resource-based behavior detection is a method of detecting ransomware based on the usage rate of resources consumed by the ransomware to encrypt files. However, this method requires a lot of time to collect resources such as CPU and I/O, and has disadvantages in that the false positive rate and the false positive rate are high. The connection-based behavior detection method is a method for detecting and preventing ransomware by monitoring external connections due to the fact that the ransomware receives a key required for encryption from a server. However, this method does not detect ransomware that does not connect to the network. Therefore, most of the currently proposed methods have a problem in that they cannot effectively detect ransomware due to high false positive and false positive rates, and cannot be detected by backup systems including the cloud.

Ransomware infection in general systems such as PCs causes serious problems. For example, all of the user's files are encrypted and cannot be recovered, or the system is locked, preventing the system from running. Therefore, it is possible to protect the system by detecting and preventing some ransomware using the existing ransomware detection methods as described above. However, most of the existing methods have limitations in considering only the detection of ransomware in a general system. In other words, files delivered to a backup system such as the cloud are not considered, and files infected with ransomware delivered to the backup system cannot be detected, causing serious problems. The main purpose of using a backup system is to back up your files. Therefore, if a user's files are encrypted, the original files can be restored by synchronizing the backed up files from cloud services such as dropbox and Google OneDrive, USB storage, and backup systems such as external disks. Nevertheless, if the files infected with ransomware are synchronized with such a backup system, it is impossible to restore them even through the backed up files. For example, if the data in the system is infected with ransomware and the encrypted file is delivered to the backup system, the original file is replaced with the encrypted file. After the file is replaced, the user cannot restore the original file even if he tries to recover it from the backup system. As described above, a method for decrypting the backed-up file by determining whether the file synchronized with the backup system is infected with ransomware is urgently needed.

The technical task to be achieved by the present invention is to derive a method for accurately and effectively classifying infected files through machine learning, and by selecting an optimal model based on various machine learning models, simply define a reference value of entropy and use it as a backup system. An object of the present invention is to provide a method and an apparatus for solving a problem of a method of detecting a file infected with a synchronized ransomware.

In one aspect, the effective ransomware detection method using machine learning based on the entropy of a file in the backup system proposed by the present invention measures the entropy for each user and each file format through the backup system, and each measured entropy The step of deriving an entropy reference value for detecting a file infected with ransomware using machine learning based on Measuring the entropy of the file synchronized from the detection module to the backup system, and detecting the file infected with the ransomware by comparing the measured entropy of the file synchronized to the backup system with the entropy reference value transmitted from the backup system.

The step of measuring entropy for each user and file format through the backup system, and deriving the entropy reference value for detecting files infected with ransomware using machine learning based on each measured entropy, is to classify files infected with ransomware. And the machine learning models to derive the entropy reference value are KNN (K-Nearest Neighbors), Linear model, Decision tree, Decision tree ensemble, Kernel trick, Deep. Including deep learning.

By performing learning and classification for each user based on a machine learning model, the optimal entropy reference value for each file format is extracted, and it has an artificial intelligence feature by continuously extracting and updating the reference value.

The step of measuring the entropy of files synchronized from the ransomware detection module of each user to the backup system includes a machine learning model inside the ramsomware detection module and uses the synchronized files as a test set to detect files infected with ransomware. it is possible to do

By utilizing the reference value derived through machine learning based on entropy for each user and file format, it detects files infected with ransomware delivered to the backup system in real time and safely protects the files stored in the backup system. It restores the original files from the backup system even in case of infection.

In another aspect, an effective ransomware detection system using machine learning based on the entropy of a file in the backup system proposed in the present invention measures the entropy of each file format of users, and a machine based on each measured entropy Using learning, the entropy standard for detecting files infected with ransomware is derived, and the entropy standard for each user and file format is transmitted to the user's ransomware detection module, and the entropy of the file synchronized with the backup system is determined. and a ransomware detection module of each user that detects a file infected with ransomware by measuring and comparing the measured entropy of the file synchronized to the backup system with the entropy reference value transmitted from the backup system.

The backup system measures the entropy of each user's file format, and the learning unit that performs machine learning based on each measured entropy and the entropy standard for detecting files infected with ransomware are derived for each user and each file format. and an entropy reference value setting unit that transmits the entropy reference value to the user's ransomware detection module.

The ransomware detection module includes a measurement unit that measures the entropy of a file synchronized with the backup system, and each user who detects a file infected with ransomware by comparing the measured entropy of the file synchronized with the backup system with the entropy reference value received from the backup system. of ransomware detection unit.

According to embodiments of the present invention, it is possible to detect based on the entropy of a file transmitted to a backup system, and continuously reflect an optimal reference value through machine learning. Therefore, the detection rate is very high compared to the existing detection methods, and there are advantages of low false positive and false positive rates. In addition, it is very effective and highly accurate because it selects an optimal model for detecting files infected with ransomware based on various machine learning models and derives an entropy reference value. In addition, it has an artificial intelligence element by automatically applying the entropy reference value and model according to the user in the backup system, and this feature can detect ransomware very advanced.

1 is a graph showing a compression file entropy comparison result according to an embodiment of the present invention.
2 is a diagram illustrating entropy measurement results for each file format according to an embodiment of the present invention.
3 is a schematic diagram illustrating an operation of a ransomware detection system according to an embodiment of the present invention.
4 is a flowchart illustrating an effective ransomware detection method using machine learning based on entropy of a file in a backup system according to an embodiment of the present invention.
5 is a diagram showing the configuration of an effective ransomware detection system using machine learning based on the entropy of a file in the backup system according to an embodiment of the present invention.
6 is a graph showing a precision-recall curve according to an embodiment of the present invention.
7 is a graph showing an ROC curve according to an embodiment of the present invention.

The present invention proposes a method of detecting a file infected with ransomware based on the entropy of the file. Based on how the ransomware encrypts the file, it utilizes the characteristics that appear in the encrypted file. One of the characteristics appearing in ciphertext is uniformity, and in the present invention, entropy is used as one of the methods for measuring uniformity. Entropy can be measured through various methods, among which NIST 800-90b is representative.

In order to verify the ransomware detection method proposed in the present invention, the entropy of the original file and the encrypted file for the system file, document file, image file, source code file, executable file, and compressed file, which are important files for the system and user. compare If the entropy of these files is measured, system files, source code files, and executable files are relatively easy to detect ransomware-infected files. However, some of the entropy of a document file, an image file, and particularly a compressed file may be lower than the entropy of an intact file, so it is difficult to detect effectively, which is shown in FIG. 1 .

1 is a graph showing a compression file entropy comparison result according to an embodiment of the present invention.

Therefore, in the present invention, in order to solve the problem of a method of detecting a file infected with ransomware synchronized with a backup system by simply defining a reference value of entropy, a method for accurately and effectively classifying infected files is derived through machine learning. That is, it is an object of the present invention to select an optimal model based on various machine learning models. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The contribution of the present invention is as follows.

Existing ransomware detection methods cannot detect ransomware-infected files in the backup system. However, the present invention effectively detects a file infected with ransomware delivered to a backup system in real time by utilizing a reference value derived through machine learning based on entropy for each file format. Through this, the files stored in the backup system are safely protected, and even if the user's system is infected with ransomware, the original files are restored from the backup system.

Existing ransomware detection methods have problems with a low detection rate and high false positive and non-detection rates. In addition, resource and network information must be collected, and all activities in the system are monitored, which consumes a lot of resources. However, the proposed method can detect based on the entropy of the file delivered to the backup system and continuously reflect the optimal reference value through machine learning. Therefore, the detection rate is very high compared to the existing detection methods, and there are advantages of low false positive and false positive rates.

The proposed method is very effective and has high accuracy because it selects an optimal model for detecting files infected with ransomware based on various machine learning models and derives an entropy reference value. In addition, it has an artificial intelligence element by automatically applying the entropy reference value and model according to the user in the backup system, and this feature is considered to be able to detect ransomware very advanced.

First, the prior knowledge required for the proposed ransomware detection method will be described. It describes entropy, which is key information for detecting ransomware, and entropy measurement method. In addition, in the present invention, machine learning is used to classify files infected with ransomware and clean files based on the measured entropy, and necessary machine learning models are described. It is an object of the present invention to derive a suitable machine learning model based on these models. Machine learning models include KNN (K-Nearest Neighbors), linear models, decision trees, decision tree ensembles, kernel tricks, and deep learning. do. In addition, model evaluation results are derived to evaluate these models, and evaluation models include cross-validation, cross-validation including divider, LOOCV, and random-split cross-validation.

One of the methods to evaluate the performance of generated ciphertext in cryptography is entropy measurement. Entropy is a method of measuring uniformity, which measures the disorder or randomness of the generated ciphertext. For example, if the values appearing as a result of encryption range from 0x00 to 0xFF, the probability of each generated value should be the same. If there is a statistical bias such as a high or low frequency of occurrence of a specific value or a specific range of values, a decryption vulnerability occurs based on the probability of generating more or less values. In order to solve this problem, the encryption technology is designed so that the probability of occurrence of each value generated by the ciphertext is almost the same, which is called uniformity.

Uniformity can be measured based on entropy, and methods for measuring entropy include a poisson distribution, a hamming distance, and spontaneous emission. Among them, NIST provided a method and a tool for measuring entropy, and it was published as NIST 800-90b. It is classified into a statistic-based measurement method and a predictor-based measurement method according to the measurement method.

2 is a diagram illustrating entropy measurement results for each file format according to an embodiment of the present invention.

In the present invention, a statistical-based measurement method is used to speed up the measurement speed. These measures include a most common value estimate, a collision test estimate, a markov test estimate, and a compression test estimate. These methods measure entropy based on frequent occurrences of probabilities, random repeating patterns, and dependencies between successive values. The entropy measurement results for each file format are shown in FIG. 2 .

Referring to FIG. 2 , in most files, the maximum common value evaluation has the highest entropy, and the Markov test evaluation has the lowest entropy. Looking at the entropy by file format, the source code file has the lowest entropy, and the document file and the compressed file have the highest entropy.

A machine learning model according to an embodiment of the present invention includes KNN, a linear model, a decision tree, a decision tree ensemble, a kernel technique, and deep learning.

KNN finds one nearest training data point as its nearest neighbor and uses it for prediction. Classify k random neighbors, and label classes with more neighbors based on the number of neighbors in each class. Therefore, it is a model that divides the decision boundary into areas designated by each class and classifies data based on the boundary. The important parameters in this model are the distance between data points and the number of neighbors, and the performance of the model depends on the parameters.

The linear model performs prediction using a linear function for input characteristics, and the generalized prediction function is as follows.

(1)

(One)

는 모델이 생성한 예측값이다. 이 모델은 다양한 회귀 모델을 포함하며, 회귀 모델로는 선형 회귀(linear regression), 릿지 회귀(Ridge regression), 라쏘 회귀(Lasso regression), 로지스틱 회귀(Logistic regression), 선형 서포트 벡터 머신(LinearSVC, support vector classifier)을 포함한다. From x[0] to x[p] are the characteristics of one data point, and w and b are parameters for the model to learn. That is, w is the slope, and b is the intercept that intersects the y-axis.

is the predicted value generated by the model. This model includes various regression models, including linear regression, ridge regression, lasso regression, logistic regression, and linear support vector machine (LinearSVC, support). vector classifier).

Linear regression finds parameters w and b that minimize the mean squared error between the prediction and the target y in the training set. The mean square error is obtained by adding the difference between the predicted value and the target value, then dividing by the number of samples, and is shown in Equation (2).

(2)

(2)

where n is the number of samples. The slope parameter w is called a weight or coefficient, and the parameter b is called an offset or intercept. This model has the disadvantage that the complexity cannot be controlled. Therefore, in order to solve this problem, a ridge regression model that can control the complexity is used.

The ridge regression model is to make the absolute value of the weight w as small as possible, which minimizes the influence of all features on the output. This is called regularization. In other words, it means that the model is forcibly restricted so that it does not overfit, and the regulatory method used in this model is called L2 regulation. This model applies the square of the L2 norm of the coefficient as a penalty, and is shown in Equation (3).

(3)

(3)

Here, if a is increased, the effect of the penalty is increased by decreasing the weight, and when a is decreased, the effect of the penalty is decreased by increasing the weight.

There are ridge regression model and lasso regression model as a method of applying regulation to linear regression model. Ridge regression models use L2 regulation, and Lasso regression models use L1 regulation. This model uses the L1 norm of the coefficient vector, which is the sum of the absolute values of the coefficients, as a penalty, and is shown in Equation (4).

(4)

(4)

Similarly to the ridge regression model in this model, if a is increased, the effect of the penalty is increased by decreasing the weight, and when a is decreased, the effect of the penalty is decreased by increasing the weight. In general, the ridge regression model is preferred among the ridge regression model and the lasso regression model. However, if there are many features and only a few features are important, the Lasso regression model may be better.

Linear classification models include logistic regression models and linear support vector machine models. It is a classifier that classifies two classes using a line, a plane, and a hyperplane, and is shown in Equation (5).

(5)

(5)

The model compares the weighted sum of the features to a threshold of zero. If the result is less than 0, the class is predicted as -1, if it is greater than 0, it is predicted as +1, and the class is classified based on the prediction result.

Decision tree models are widely used models for classification and regression problems, and are learned by successive yes/no questions to arrive at a decision. This model partitions the data into a list of yes/no questions, and partitioning the data iterates until each partitioned region (a leaf of the decision tree) has one target value (one class or one regression analysis result). do. A leaf node consisting of only one target is called a pure node. The disadvantage of this model is that if we partition the data until all leaf nodes are pure nodes, the model becomes very complex and overfits the training data. There are two ways to control this; Pre-pruning and post-pruning. Pre-pruning is a strategy to stop tree creation early, and post-pruning is a strategy to delete or merge nodes with few data points after tree creation.

Ensemble is a technique to build a more powerful model by linking multiple machine learning models, and it has been proven to be effective in various datasets of classification and regression problems. Decision tree-based ensemble models include a random forest model and a gradient boosting decision tree model. The random forest model is a method of calculating the average of several decision trees that are slightly different in order to solve the disadvantage of overfitting the decision tree to the training data. There are two other methods of generating decision trees: randomly selecting data points and randomly selecting features in a split test. The gradient boosting decision tree model creates trees sequentially in a way that compensates for the errors of the previous tree, and this model has no randomness. Instead, it uses strong pre-pruning, which is usually a good model from a performance standpoint as it uses a shallow tree of five or so.

A well-known kernel technique is kernelized support vector machines. This model is an extension of the input data to create more complex models that are not defined by simple hyperplanes. For this reason, straight lines and hyperplanes are not flexible, and linear models are very limited for low-dimensional datasets. To solve this, a new feature is added by multiplying or powering the feature. However, there is a disadvantage that the calculation cost increases as the number of characteristics increases.

In order to reduce the computational cost, there is a kernel trick that can train a classifier in a high dimension without generating many new features. This technique does not actually expand the data, but rather computes the distance (more precisely, the scalar product) of the data points for the extended feature. Based on this, in SVM, each training data point distinguishes a decision boundary between two classes, and data points located at the boundary are called a support vector. Therefore, to predict a new data point, the distance to each support vector is measured, which is calculated by a Gaussian kernel. The formula for calculating the distance is shown in Equation (6).

(6)

(6)

, gamma)는 가우시안 커널의 폭을 제어하는 매개변수이다. 이 모델은 감마 매개변수로 하나의 훈련 샘플이 미치는 영향의 범위를 결정한다. 작은 값은 넓은 영역이며, 큰 값은 제한적인 범위를 가진다. 그리고 C 매개변수는 유클리디안 거리를 제한하는 것으로, 선형 모델에서 나타낸 매개변수와 비슷한 규제 매개변수이며, 각 포인트의 중요도를 제한한다.where x1 and x2 are data points, and ||x1-x2|| is the Euclidean distance. gamma(

, gamma) are parameters that control the width of the Gaussian kernel. The model determines the range of influence of one training sample as a gamma parameter. A small value is a large area, and a large value has a limited range. And the C parameter limits the Euclidean distance, which is a regulatory parameter similar to the parameter shown in the linear model, and limits the importance of each point.

In the present invention, multilayer perceptrons (MLP), which are neural network (ie, deep learning) models that can be relatively simply used for classification and regression, are used. This model repeats the process of making the weighted sum, which is the process from Equation (1), which is a linear regression model, to the output, not once, but several times, and calculates the hidden unit constituting the intermediate step and re-weights the sum. It is a model that calculates This hidden unit constitutes a hidden layer, and deep learning is performed due to this hidden layer. An important parameter in this model is the number of units in the hidden layer, and as many hidden units are added, the computation cost increases.

The model evaluation method utilized in the present invention includes cross-validation, cross-validation including divider, LOOCV, and randomized cross-validation.

Cross-validation is a method of iteratively dividing data multiple times and training multiple models. The most widely used cross-validation method is k-fold cross-validation, which usually uses 5 or 10 folds. For example, in 5-fold cross-validation, a subset of approximately similar size is divided into 5, and then a series of models is created. The first model is trained using the first fold as the test set and the other folds 2 through 5 as the training set. This evaluation method is a verification method that repeats up to the fifth model. In this way, simply classifying sets may result in erroneous evaluation results.

To solve this problem, stratified k-fold cross-validation is used for each layer. This verification method classifies the data so that the class ratio in the fold is the same as the class ratio of the entire dataset, which is called a cross-validation method including a divider.

Leave-one-out cross-validation (LOOCV) is a k-fold cross-validation method that includes only one sample per fold. This method selects one data point from each iteration and uses it as a test set.

A very flexible cross-validation strategy is shuffle-split cross-validation. In this method, a specific size is divided to make a training set and a test set, and each division is repeated a specific number of times. This method is useful when the number of iterations needs to be adjusted independently of the size of the training or test set.

3 is a schematic diagram illustrating an operation of a ransomware detection system according to an embodiment of the present invention.

In order to detect ransomware, the backup system performs extraction 320 for each file format on the file 310 on the backup system. Thereafter, entropy for each file format of users is measured ( 300 ), and a reference value for detecting a file infected with ransomware is derived ( 350 ) using machine learning ( 340 ) based on the measured entropy. The standard values derived for each user and each file format are delivered to the user's client software. Each user's client software has a built-in ransomware detection module, and the detection module measures the entropy of files synchronized to the backup system. A file infected with ransomware can be detected by comparing the entropy of the file with the entropy threshold received from the backup system. In addition, if system resources are available, a machine learning model can be included in the detection module, and files infected with ransomware are detected using synchronized files as a test set. The proposed ransomware detection system will be described in more detail with reference to FIGS. 4 and 5 .

4 is a flowchart illustrating an effective ransomware detection method using machine learning based on entropy of a file in a backup system according to an embodiment of the present invention.

In the proposed backup system, an effective ransomware detection method using machine learning based on the entropy of a file measures the entropy for each user and each file format through the backup system, and uses machine learning based on each measured entropy to detect ransomware. Step 410 of deriving an entropy reference value for detecting a file infected with ware, a step of transferring the derived entropy reference value for each user and file format to the ransomware detection module of users (420), each user's ransomware detection module Measuring the entropy of the file synchronized to the backup system in step 430 and detecting the file infected with ransomware by comparing the measured entropy of the file synchronized to the backup system with the entropy reference value transmitted from the backup system (440) includes

In step 410, entropy for each user and file format is measured through the backup system, and an entropy reference value for detecting a file infected with ransomware is derived using machine learning based on each measured entropy.

The backup system stores each user's backup data. To detect ransomware-infected files, the entropy threshold of each user's backup data is measured. Since the entropy reference value has a different reference value for each file format, entropy for each file format is measured. The entropy measurement uses a relatively fast most common value estimate, a collision test estimate, a markov test estimate, and a compression test estimate. .

The backup system uses machine learning to classify files infected with ransomware and derive entropy standards. The machine learning models used are KNN (K-Nearest Neighbors), linear model, decision tree, decision tree ensemble, kernel trick, deep learning. to be. Based on these models, by performing learning and classification for each user, the optimal entropy reference value for each file format is extracted, and it has an artificial intelligence feature by continuously extracting and updating the reference value.

In step 420, the entropy reference value derived for each user and each file format is transmitted to the users' ransomware detection module.

The entropy reference value extracted in step 410 is transmitted to the user's client software. The ransomware detection module built into the client software measures the entropy of files synchronized to the backup system.

In step 430, the entropy of the file synchronized from the ransomware detection module of each user to the backup system is measured.

In step 440, the ransomware-infected file is detected by comparing the measured entropy of the file synchronized to the backup system with the entropy reference value transmitted from the backup system.

The ransomware detection module detects files infected with ransomware by comparing the measured entropy of synchronized files with the standard value for each file format received from the backup system. It is also good practice to determine the entropy threshold as the average of a large number of files. However, the entropy of files may be different for each user. Therefore, the backup system derives a reference value optimized for user files by measuring and learning the entropy of the files stored in the backup system for each user, which helps to more accurately detect files infected with ransomware.

The data used for machine learning in the proposed system are file format, entropy using maximum common value evaluation, entropy using collision test evaluation, entropy using Markov test evaluation, and entropy using compressed test evaluation, It consists of the presence or absence of an encryption file (0/1) The file format was expressed as a system file (1), a document file (2), an image file (3), a source code file (4), an executable file (5), and a compressed file (6). Entropy was measured based on 8 bits, and the result has a value from 0 to 8. The presence or absence of the encrypted file was expressed as 0 for the original file and 1 for the encrypted file infected with ransomware. For entropy measurement, data of a total of 1,200 files, 100 files for each file format and 100 files for each file format that encrypted these files, were collected. An example of the collected data is shown in Table 1.

<Table 1>

5 is a diagram showing the configuration of an effective ransomware detection system using machine learning based on the entropy of a file in the backup system according to an embodiment of the present invention.

The ransomware detection system 500 according to the present embodiment may include a ransomware detection module 510 , a bus 520 , a network interface 530 , a memory 540 , a database 550 , and a backup system 560 . can The memory 540 may include an operating system 541 and a ransomware detection routine 542 utilizing machine learning. The ransomware detection module 510 may include a measurement unit 511 and a ransomware detection unit 512 . In other embodiments, the ransomware detection system 500 may include more components than those of FIG. 5 . However, there is no need to clearly show most of the prior art components. For example, the ransomware detection system 500 may include other components such as a display or a transceiver.

The memory 540 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. In addition, program codes for the operating system 541 and the ransomware detection routine 542 using machine learning may be stored in the memory 540 . These software components may be loaded from a computer-readable recording medium separate from the memory 540 using a drive mechanism (not shown). The separate computer-readable recording medium may include a computer-readable recording medium (not shown) such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memory 540 through the network interface 530 instead of a computer-readable recording medium.

Bus 520 may enable communication and data transfer between components of ransomware detection system 500 . Bus 520 may be configured using a high-speed serial bus, parallel bus, storage area network (SAN), and/or other suitable communication technology.

The network interface 530 may be a computer hardware component for connecting the ransomware detection system 500 to a computer network. The network interface 530 may connect the ransomware detection system 500 to a computer network through a wireless or wired connection.

The database 550 may serve to store and maintain all information necessary for effective ransomware detection using machine learning based on the entropy of a file in the backup system. Although FIG. 5 illustrates that the database 550 is built and included in the ransomware detection system 500, it is not limited thereto and may be omitted depending on the system implementation method or environment, or the entire or part of the database is It is also possible to exist as an external database built on another separate system.

The ransomware detection module 510 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations of the ransomware detection system 500 . The instructions may be provided to the ransomware detection module 510 by the memory 540 or network interface 530 and via the bus 520 . The ransomware detection module 510 may be configured to execute program codes for the measurement unit 511 and the ransomware detection unit 512 . Such program codes may be stored in a recording device such as the memory 540 .

The backup system 560 may include a learning unit 561 and an entropy reference value setting unit 562 .

The measuring unit 511 , the ransomware detecting unit 512 , the learning unit 561 , and the entropy reference value setting unit 562 may be configured to perform steps 410 to 440 of FIG. 4 .

The ransomware detection system 500 may include a measurement unit 511 and a ransomware detection unit 512 .

The measurement unit 511 measures the entropy of a file synchronized with the backup system. The measurement unit 511 detects a file infected with ransomware by using a synchronized file as a test set by including a machine learning model in the ramsomware detection module when system resources are available.

The ransomware detection unit 512 may be embedded in each user's software and detects a file infected with ransomware by comparing the measured entropy of the file synchronized to the backup system with the entropy reference value transmitted from the backup system.

The learning unit 561 measures entropy for each file format of users, and performs machine learning based on each measured entropy.

The entropy reference value setting unit 562 derives an entropy reference value for detecting a file infected with ransomware, and delivers the derived entropy reference value for each user and file format to the ransomware detection module of users. The entropy reference value setting unit 562 has an artificial intelligence feature by extracting an optimal entropy reference value for each file format and continuously extracting and updating the reference value.

The proposed ransomware detection system 500 detects files infected with ransomware delivered to the backup system in real time by utilizing a reference value derived through machine learning based on entropy for each user and file format, and stores the files stored in the backup system. It is safe and secure, and even if the user's system is infected with ransomware, the original files can be restored from the backup system.

For supervised learning, training sets and test sets were classified into arbitrary numbers, and the scores of the training sets and test sets are shown in Table 2.

<Table 2>

Looking at the results, KNN, LinearSVC, SVM, and MLP completely classified the data in both the training set and the test set. Decision trees, random forests, and gradient boosting classify data at a very high level, such as linear regression, ridge regression, and logistic regression. regression) classifies data at a high level. However, in lasso regression, the training and test sets score very low.

Most of the parameters used here were set as default values, and since the performance of the machine learning model depends on the parameters, the parameters with the best scores were derived. For more accurate measurement, the training set, validation set, and test set were classified, and each score is shown in Table 3.

<Table 3>

Looking at the results, the scores of Lasso regression, logistic regression, decision tree, random forest, and gradient boosting improved. Looking at each, Lasso regression increased the training set score from 0.16 to 0.90, an improvement of 562%. The test set score increased from 0.16 to 0.88, an improvement of 550%. Logistic regression improved the training set score by 109% from 0.91 to 1.0 and the test set score improved by 112% from 0.887 to 1.0. The decision tree improved the training set score by 100.2% from 0.998 to 1.0, and the test set score improved by 100.3% from 0.997 to 1.0. The random forest improved the training set score by 100.4% from 0.996 to 1.0, and the test set score improved by 101% from 0.987 to 1.0. Gradient boosting improved the training set score by 100.2% from 0.998 to 1.0, and the test set score improved by 100.3% from 0.997 to 1.0. By applying the optimal parameters, it was verified that all models except for linear regression, ridge regression, and Lasso regression perfectly classify ransomware-infected files and detect them.

In order to derive the model evaluation results, an optimal verification model was derived based on the cross-validation method described above, and the results are shown in Table 4.

<Table 4>

Looking at the results, only decision trees have perfect cross-validation scores. KNN, logistic regression, linear SVC, random forest, gradient boosting, SVM, and MLP have very high level of cross-validation scores, linear regression, ridge regression Ridge regression, and Lasso regression have a relatively high level of cross-validation model score. All cross-validation scores are 0.9 or higher, suggesting that the model can be utilized at an appropriate level.

Looking at the validation model, cross-validation and stratified k-fold cross-validation are used for validation in many machine learning models, and LOOCV and shuffle-split cross validation -validation) is used for validation in some models.

In order to derive the performance of the machine learning model used in the proposed system, the accuracy, precision, recall, and F1-score of each model were evaluated. Accuracy is a value obtained by dividing the number of accurately predicted numbers (TP (True Positive) and TN (True Negative)) by the total number of samples, and this is shown in Equation (7). Precision measures how many samples among those predicted to be positive (TP+FP) are truly positive (TP), and this is shown in Equation (8). Recall measures how many samples out of the total positive samples (TP+FN) are classified into the positive class (TP), which is shown in Equation (9). The F1-score is the harmonic mean of precision and recall, summarizing the two into one. This is shown in Equation (10). For all performance evaluation indicators, 0 means the lowest and 1 means the highest result. Table 5 shows the results of this performance evaluation.

(7)

(7)

(8)

(8)

(9)

(9)

(10)

(10)

<Table 5>

Looking at the results, linear regression, ridge regression, and lasso regression have no evaluation results because they have no meaning in performance evaluation. Except for gradient boosting, the other models, KNN, logistic regression, linear SVC, decision tree, random forest, SVM, and MLP, are precision), recall, and F1-score are all 1.0, which means that the performance of machine learning used in the proposed method is almost perfect.

6 is a graph showing a precision-recall curve according to an embodiment of the present invention.

As an evaluation for examining all thresholds or looking at precision or recall at once, there is a precision-recall curve. It can represent a curve graph as a sorted list of values of precision and recall for all possible thresholds, the closer the curve of the graph is to the upper right, the better the classifier. That is, the upper right point is a place where both precision and recall are high at a threshold, and in the present invention, logistic regression, linear SVC, decision tree, and random forest that can be expressed as graphs are used. , the precision-recall curves of gradient boosting and SVM are shown in FIG. 6 .

Looking at the results, the results of all the models to be evaluated have a curve pointing upwards to the right. Therefore, the performance of machine learning used in the proposed method is almost perfect.

A widely used tool for characterizing a classifier at multiple thresholds is the Receiver Operating Characteristics (ROC) curve. Similar to the precision-recall curve, this curve takes into account all thresholds of the classifier and represents the false positive rate (FPR) to the true positive rate (TPR). The true positive rate is the recall rate, and the false positive rate is the proportion that is misclassified as false positives. The formula for FPR is shown in Equation (11).

식 (11)

Equation (11)

7 is a graph showing an ROC curve according to an embodiment of the present invention.

The closer the ROC curve is to the upper left, the better the performance. In the present invention, the ROC curves of logistic regression, linear SVM, decision tree, random forest, gradient boosting, and SVM that can be expressed as graphs are shown in Figure 5.

Looking at the results, the results of all models to be evaluated have a curve toward the upper left. Therefore, the performance of machine learning used in the proposed method is almost perfect.

The final evaluation index is AUC (Area Under the Curve). This is a summary of the ROC curve as an area value under the ROC curve, and the AUC result always has a value between the worst 0 and the best 1. Therefore, this result means to rank the positive samples. That is, it is equal to the probability that the score of the randomly selected positive class point is higher than the score of the randomly selected negative class point. If the AUC is 1, it means that the scores of all positive points are higher than the scores of all negative points. Table 6 shows the AUC evaluation results of the machine learning model utilized in the present invention.

<Table 6>

Looking at the results, the AUC score of all the machine learning models used is 1.0. This means that the performance of machine learning used in the proposed method is almost perfect.

Finally, Table 7 shows the detection rate comparison results between the proposed method and the existing method.

<Table 7>

Existing methods include file-based detection that detects based on file information or signature, resource-based detection such as I/O requests and CPU utilization, system-based detection that detects based on system information such as user activity and messages, and API calls. Based on this, it was classified into machine learning-based detection using machine learning, deep learning-based detection, and other detection methods. Looking at the results, various detection rates were derived from a minimum of 91% to a maximum of 99%. However, the proposed method rather than these methods is used in all machine learning classifiers such as logistic regression, linear SVC, decision tree, random forest, gradient boosting, SVM, and MLP. It has a higher detection rate than the existing method. Therefore, the proposed method detects ransomware more effectively than the existing method, and in particular, protects the original file of the user by detecting the file infected with the ransomware synchronized with the backup system.

The present invention proposes a method of detecting a file infected with ransomware through machine learning based on the entropy of the file in the backup system. Some of the image files and compressed files have high entropy characteristics, and there is a limit to detecting a file infected with ransomware only with the entropy reference value. In order to overcome this limitation, the method proposed by the present invention detects an infected file using machine learning based on entropy for each file format. As a result of the experiment, if entropy is used, most machine learning classifies infected files at a very high level, and has a very low false positive and false positive rate. Also, the model evaluation results are very high, which means that the used model can be trusted. Finally, performance evaluation indicators such as accuracy, precision, recall, F1-score, precision-recall curve, ROC curve, and AUC were all evaluated at high levels. Therefore, the proposed method detects files infected with ransomware very effectively. Through this, files infected with ransomware are not synchronized to the backup system, and the original files can be restored by restoring the files stored in the backup system.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. Further, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or, to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiments have been described by the limited embodiments and drawings as described above, various modifications and variations can be made from the above description to those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and those equivalent to the claims also fall within the scope of the claims to be described later.

Claims (11)

Measuring entropy for each user and file format through a backup system, and deriving an entropy reference value for detecting a file infected with ransomware using machine learning based on each measured entropy;
transmitting the entropy reference value derived for each user and each file format to the users' ransomware detection module;
Measuring the entropy of files synchronized from the ransomware detection module of each user to the backup system; and
Detecting files infected with ransomware by comparing the measured entropy of files synchronized to the backup system with the entropy threshold received from the backup system
Including,
The steps of measuring entropy for each user and file format through the backup system and deriving an entropy reference value for detecting files infected with ransomware using machine learning based on each measured entropy include:
Machine learning models for classifying ransomware-infected files and deriving entropy thresholds are KNN (K-Nearest Neighbors), linear model, decision tree, decision tree ensemble, and kernel. Including Kernel trick, Deep learning,
Based on the machine learning model, by performing learning and classification for each user, the entropy reference value for each file format is extracted, and it has artificial intelligence characteristics by continuously extracting and updating the reference value,
In order to increase the entropy measurement speed for each file format, the most common value estimate, the collision test estimate, the markov test estimate, and the compression test estimate are used. It uses a statistical-based measurement method that includes, and measures entropy based on probability frequency, random repeating pattern, and dependence between successive values.
How to detect ransomware. delete delete The method of claim 1,
The step of measuring the entropy of the file synchronized from the ransomware detection module of each user to the backup system is:
It includes a machine learning model inside the ransomware detection module and uses the synchronized files as a test set to detect files infected with ransomware.
How to detect ransomware. The method of claim 1,
By utilizing the reference value derived through machine learning based on entropy for each user and file format, it detects files infected with ransomware delivered to the backup system in real time and safely protects the files stored in the backup system. to restore original files from backup system even if infected with
How to detect ransomware. Measure the entropy of each user's file format, and use machine learning based on each measured entropy to derive the entropy reference value for detecting files infected with ransomware, and set the entropy threshold for each user and file format to the user's ransomware. a backup system that delivers to the wear detection module; and
Ransomware detection module for each user that measures the entropy of files synchronized to the backup system and detects files infected with ransomware by comparing the measured entropy of the files synchronized to the backup system with the entropy threshold received from the backup system.
Including,
backup system,
a learning unit that measures entropy for each file format of users and performs machine learning based on each measured entropy; and
To detect files infected with ransomware, the entropy threshold for each user and file format is derived and delivered to the user's ransomware detection module, and the entropy threshold for each file format is extracted and continuously extracted and updated to provide artificial intelligence. Entropy reference value setting unit with characteristics
Including,
Ransomware detection module,
a measurement unit measuring the entropy of a file synchronized to a backup system; and
Ransomware detection unit for each user that detects files infected with ransomware by comparing the measured entropy of the files synchronized to the backup system with the entropy threshold received from the backup system.
Including,
Machine learning models for classifying ransomware-infected files and deriving entropy thresholds are KNN (K-Nearest Neighbors), linear model, decision tree, decision tree ensemble, and kernel. Including Kernel trick, Deep learning,
In order to increase the entropy measurement speed for each file format, the most common value estimate, the collision test estimate, the markov test estimate, and the compression test estimate are used. It uses a statistical-based measurement method that includes, and measures entropy based on probability frequency, random repeating pattern, and dependence between successive values.
Ransomware detection system. delete delete delete The method of claim 6,
the measuring unit,
If system resources are available, including a machine learning model inside the ransomware detection module, it uses the synchronized file as a test set to detect files infected with ransomware.
Ransomware detection system. The method of claim 6,
By utilizing the reference value derived through machine learning based on entropy for each user and file format, it detects files infected with ransomware delivered to the backup system in real time and safely protects the files stored in the backup system. to restore original files from backup system even if infected with
Ransomware detection system.

Images