CN104376260A - Malicious code visualized analyzing method based on Shannon information entropy - Google Patents

Abstract

The present invention provides a malicious code visual analysis method based on Shannon information entropy, comprising: the first step: convert the binary bytes of the malicious file into the yellow value of the pixel point in the "pixel map", and use the green channel Ox50 to The point where the marked pixel value is 0x20-0x7E; second step: calculate the local entropy of the pixel value in each 256-byte block in the "pixmap" based on the pixel value of the "pixmap", and the local entropy is as follows Shannon information entropy formula calculation: Wherein, pi _represents the probability that byte (pixel) value i occurs, and the value range of i is 0x00-0xFF, and Entropy is local entropy; Calculate the f (Entropy) value of local entropy value Entropy, its calculation formula is: f( Entropy)=2 ^Entropy -1; an "entropy map" is generated from the calculation result of f(Entropy); the third step: the calculation result of f(Entropy) is normalized to generate an "entropy normalization map". The invention can effectively distinguish samples of various families, and can easily find potential differences when analyzing malicious codes of the same family, and provides a basis for grasping the variation evolution law of the family.

Description

A Visual Analysis Method of Malicious Code Based on Shannon Information Entropy

technical field

The invention relates to a malicious code visualization analysis method based on Shannon information entropy.

Background technique

Malware (Malicious Software) is a software used to destroy computer operating systems, steal sensitive information, or illegally access private systems, usually in the form of code, scripts, dynamic text, or other software. Due to the complex and time-consuming traditional malware analysis process, it is difficult for even experienced security analysts to discover potential attack modes. In order to reduce the cognitive burden and improve interactivity, introducing information visualization technology into the field of malicious code analysis, that is, Malware security visualization, is the frontier hotspot in network security research in recent years.

In 2008, Gregory Conti et al. of the United States Military Academy West Point proposed the idea of gray-scale images for the first time in their designed visual analysis system (as shown in Figure 1), to be independent of Text analysis perspective to quickly identify files and dissect unknown file formats. As shown in Figure 1, the d area and g area of the user interface of the system correspond to the ASCII format string and the hexadecimal format command line of the analyzed file respectively; the pixel (pixel) value of c area (Byteview) and the g area word The binary number of the section (Byte) corresponds to the internal characteristics of the file in the form of a grayscale image; the b area (Byte Presence) identifies its own area according to the presence or absence of the extended ASCII code value (0-255) in each line of the c area The column where the corresponding row is located in, through such a mapping operation, helps users grasp the rules of the file and find abnormalities in it; the f area (Dot Plot) uses the byte sequence matrix of the file itself to compare the similarity between files, and provides a basis for judgment when users classify ; Other areas a, e, and h integrate a variety of auxiliary functions that interact with users.

However, Gregory Conti, Erik Dean, Matthew Sinda and Benjamin Sangster.Visual Reverse Engineering of Binary and Data Files[C].VizSec 2008Symposium on Visualization for CyberSecurity(VizSEC2008) method makes the calculation amount and The size of the file is proportional to the size of the file, and the degree of automation of the analysis is restricted by the performance of the computer hardware. At the same time, in terms of presenting the internal characteristics of the file, the pixel value of the c area corresponding to the byte is separated from the b area reflecting the existence of the ASCII code value. Facilitate a comprehensive understanding of the characteristics of the analyzed files.

Contents of the invention

The purpose of the present invention is to provide a method for more comprehensive research and analysis of malicious codes.

In order to achieve the above object, the present invention provides a method for visual analysis of malicious code based on Shannon information entropy, which is characterized in that, comprising:

Step 1: Convert the binary bytes of the malicious file into the yellow shade value of the pixel in the "pixel map", and use the green channel 0x50 (the display effect may have slight differences due to different hardware devices) to mark the pixel value 0x20- The point of 0x7E (that is, the printable character in ASCII code);

Step 2: Calculate the local entropy of the pixel values in each 256-byte block in the "pixel map" based on the pixel values of the "pixel map", and the local entropy is calculated according to the following Shannon information entropy formula:

$\mathrm{<span\; class="notranslate">\; Entropy</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 255</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; lo</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}$

Among them, p _i represents the probability of occurrence of byte (pixel) value i, the value range of i is 0x00-0xFF, and Entropy is local entropy;

Calculate the f(Entropy) value of the local entropy Entropy, and its calculation formula is:

f(Entropy)= ^2Entropy -1;

Generate an "entropy map" from the calculation result of f(Entropy);

Step 3: Normalize the calculation result of f(Entropy) to generate an "entropy normalized graph".

Preferably, the described malicious code visual analysis method based on Shannon information entropy also includes:

Step 4: Normalize the "pixel map" in the first step to generate a "pixel normalized map".

The present invention draws on the idea of the grayscale image, combines Shannon's definition of information entropy, and uses the K-Nearest Neighbor (KNN) classification algorithm to provide a new visualization method for researching malicious code pedigree classification. The specific technical solution is: first convert the binary file to be detected into a "pixel map" of yellow and green channels; on this basis, generate a blue-green "entropy map" by calculating the local entropy value of the "pixel map" ; The "entropy map" is then normalized to form a green dot matrix distribution with different light and shade, that is, the "entropy normalized map". This method uses local entropy calculation and sliding window normalization processing mechanism, which can not only greatly reduce the amount of computation in similarity analysis of large files, but also improve the visualization effect of malicious code family classification.

The present invention adopts Python language programming to realize, can run under Windows and Linux environment.

Compared with the existing security visualization technology, the beneficial effects of the present invention are:

1. From the perspective of visual effects, it can effectively distinguish various malicious code families;

2. When analyzing malicious codes of the same family, it is relatively easy to find potential differences, which provides a basis for grasping the evolution law of this family of variants;

3. Combining the three visualization methods of "entropy map", "entropy normalized map" and "pixel normalized map", it can comprehensively study and analyze the overall and local characteristics of malicious code.

4. The present invention can provide more comprehensive information perception per unit of time by establishing image communication between people and data, which not only improves the work efficiency of network security analysts, but also reduces the difficulty of analysis and the technical level and experience of analysts. requirements.

5. The present invention is simple to implement and can be used for automatic operation. Due to the use of dimension reduction mapping and fast similarity comparison algorithm, the time cost of image generation is small and the similarity comparison efficiency is high.

Description of drawings

Figure 1 Schematic diagram of the visualization system designed by Gregory Conti;

Fig. 2 is an example diagram of display results converted from a binary file to a "pixel map";

Fig. 3 is an example diagram of "pixel map" converted into "entropy map" by local entropy calculation;

Figure 4a is an example diagram of "entropy map" transformed into "entropy normalized map" after normalization;

Fig. 4b is a mapping relationship diagram between local entropy value and pixel value;

Fig. 5 is an example diagram of converting a "pixel map" into a "pixel normalized map" after normalization processing;

Figure 6 is the "entropy map" of the Email-Worm.joleee.av sample;

Figure 7 is the "entropy map" of the Email-Worm.joleee.aw sample;

Figure 8 is the "entropy map" of the Email-Worm.joleee.ba sample.

Detailed ways

In order to make the present invention more comprehensible, a preferred embodiment is described in detail below with accompanying drawings. (473 harmful samples from 59 families that are used for testing of the present invention are all downloaded from VX Heavens official website, and all samples adopt Kaspersky nomenclature)

Example

A method for visual analysis of malicious code based on Shannon information entropy, specifically:

Step 1: Convert the binary bytes of the malicious file (Trojan.Regrun.rk) into the yellow shade value of the pixel in the "pixel map", and use the green channel 0x50 (the display effect may have slight differences due to different hardware devices) Mark points with pixel values of 0x20-0x7E (that is, printable characters in ASCII code); as shown in Figure 2, the black part is the background color, that is, the binary byte is 0 value.

Step 2: Calculate the local entropy of the pixel values in each 256-byte block in the "pixel map" based on the pixel values of the "pixel map", and the local entropy is calculated according to the following Shannon information entropy formula:

$\mathrm{<span\; class="notranslate">\; Entropy</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 255</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; lo</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}$

Among them, p _i represents the probability of occurrence of byte (pixel) value i, the value range of i is 0x00-0xFF, and Entropy is local entropy;

Usually, the calculation result of the local entropy value Entropy is between [0, 8]. If it is directly output as the actual value, the visual effect will be poor because the brightness is too low. Therefore, in order to form an image mapped with the pixel brightness value [0, 255], the local entropy value Entropy is calculated according to the function f(Entropy)=2 ^Entropy -1 and then output. The purpose of this processing is to display the brightness of the high entropy value more obvious. The local entropy value Entropy generates an "entropy map" after calculating the result with the function f(Entropy). As shown in Figure 3, the present invention uses a blue-green scheme to display the "entropy map" of the analyzed file. Similarly, the black part is the background color, that is, the entropy value is 0.

Step 3: Since there should be a unified judgment standard when performing similarity analysis, and the size of the analyzed malicious code files is different, it is necessary to normalize the above-mentioned results calculated by the function f(Entropy), Generate an "entropy normalized plot". The normalization algorithm proposed by the present invention adopts a sliding window mechanism with a window size of 2 bytes and a moving step size of 1 byte. The two bytes before and after the same window are used as the "entropy normalization map" respectively. The position coordinates (x, y) of the point. The number of occurrences of (x, y) combinations is proportional to the brightness value of the point in the "entropy normalization map", and the "entropy normalization map" is displayed as a square map with a size of 256*256, rendered in pure green, and The color scheme of the "entropy map" is different, as shown in Figure 4a. Wherein, the mapping relationship between the local entropy value and the pixel value is shown in Fig. 4b.

Step 4: Based on the same reason, the present invention also provides a normalized display operation of the "pixel map", the normalization algorithm of which is the same as the normalization process of the "entropy map", as shown in Figure 5. The "Pixel Normalized Map" is rendered in pure yellow, which is different from the color scheme of the "Pixel Map".

In the case of unknown classification, using the KNN classification algorithm, the present invention can correctly distinguish 59 classes in the downloaded samples, and combined with the prior knowledge of the samples, only 28 of the 437 samples are classified incorrectly, namely The average classification accuracy is 93.59%.

In the present invention, the generation time of the "entropy normalization map" is proportional to the size of the sample file, and the average generation time of the "entropy normalization map" is 0.91ms; the similarity comparison of the "entropy normalization map" The time is proportional to the number of local entropy blocks, and the average similarity comparison time of the "entropy normalized map" is 0.56ms. The resulting time data is an average calculated after 100 samples. These results show that the time efficiency of classification and comparison realized by the present invention is high, and the Python programming structure design adopted is reasonable.

Example 2

The malicious code visual analysis method based on Shannon information entropy described in Embodiment 1 is used to analyze the "entropy map" generated by malicious samples Email-Worm.joleee.av, Email-Worm.joleee.aw and Email-Worm.joleee.ba as follows As shown in Figures 6-8, when the present invention analyzes malicious codes of the same family, it is relatively easy to find potential differences, which provides a basis for grasping the evolution law of the variants of the same family.

Claims (2)

1. A method for visual analysis of malicious code based on Shannon information entropy, characterized in that, comprising: Step 1: Convert the binary bytes of the malicious file into the yellow shades of the pixels in the "pixel map", and use the green channel 0x50 to mark the points with pixel values 0x20-0x7E; Step 2: Calculate the local entropy of the pixel values in each 256-byte block in the "pixel map" based on the pixel values of the "pixel map", and the local entropy is calculated according to the following Shannon information entropy formula: $\mathrm{<span\; class="notranslate">\; Entropy</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 255</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; log</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}$ Among them, p _i represents the probability of occurrence of byte value i, the value range of i is 0x00-0xFF, and Entropy is local entropy; Calculate the f(Entropy) value of the local entropy value Entropy, and its calculation formula is: f(Entropy)= ^2Entropy -1; Generate an "entropy map" from the calculation result of f(Entropy); Step 3: Normalize the calculation result of f(Entropy) to generate an "entropy normalized graph". 2. the malicious code visual analysis method based on Shannon information entropy as claimed in claim 1, is characterized in that, the described malicious code visual analysis method based on Shannon information entropy also comprises: Step 4: Normalize the "pixel map" in the first step to generate a "pixel normalized map".