RU2786352C1 - System and method for detecting a malicious script based on a set of hash codes - Google Patents

Abstract

FIELD: information security.

SUBSTANCE: invention relates to the field of information security, in particular to a system and method for detecting malicious scripts. The effect is achieved by detecting malicious scripts or their parts written in a certain programming language by detecting a file containing a script, recognizing the script language using language recognition rules, generating a set of hash codes of the detected script and comparing the set of hash codes of the script with known sets. hash codes of malicious scripts.

EFFECT: improve the efficiency of detecting malicious scripts.

12 cl, 4 dwg

Description

Technical field

The invention relates to the field of information security, and more specifically to systems and methods for detecting malicious scripts.

State of the art

The number of users, the number of companies and the diversity of their activities continues to grow continuously. At the same time, the number of created websites and web pages is also increasing.

Creating a web page as the simplest tool for accessing information on the Internet involves the use of computer system resources. The use of resources is increasingly implemented using scripting programming languages (PHP, JavaScript, Python, Ruby, and others).

A script is a program code (script) written in various interpreted programming languages. All scripts are executed using an external program - an interpreter. Unlike executable files, scripts mostly exist as text files and can be read by humans. Malicious scripts can be roughly divided into two types. The first type includes scripts that are embedded in the code of web pages, interpreted by the browser, and perform actions laid down by attackers. The second type includes scripts that are designed to run on the user's computer. They are executed by operating system components and have access to APIs (file system, processes, etc.).

There are many ways to detect malicious scripts. One of the effective ways is to analyze the text of the web page files in detail.

Publication US7707634B2 presents a system that uses signatures to detect malicious scripts. Signatures are created based on known samples of malicious scripts. Before using the signature, the text containing the script is normalized. In the course of normalization, the beginning of the script is searched, the names of the functions and variables of the script are brought into a form suitable for using signatures.

This solution partially detects malicious scripts, but does not effectively solve the problem of detecting truncated and partially damaged malicious scripts sent in a way other than embedding into a web page.

Disclosure of invention

The invention is intended to improve the efficiency of detecting malicious scripts. The technical result of the present invention is to detect malicious scripts or their parts written in a certain programming language by detecting a file containing a script, recognizing the script language using language recognition rules, generating a set of hash codes of the detected script and comparing the set of hash codes of the script with known ones. hash codes of malicious scripts.

In one embodiment, a method is provided for detecting a malicious script based on a set of hash codes, comprising the steps of using a computer to perform operations that include the steps of: detecting a file containing the script by parsing each file in a set of files that are checked for malicious code; generating a script summary based on the detected file; counting static and dynamic parameters contained in the generated script summary; recognizing the programming language of the script based on the calculated parameters of the generated script summary using language recognition rules; processing the detected file based on the recognized programming language data and the detected file data; generating a set of hash codes based on the processed file using hash code generation rules; a malicious script is detected by comparing the generated set of hash codes with known malicious hash codes from the database of script hash codes.

In another embodiment of the method, as a result of analyzing each file from a set of files that check for malicious code, using heuristic algorithms designed to search for structured file types in which the presence of scripts is unlikely, the found files are excluded from the specified set of files.

In yet another embodiment of the method, a script summary is generated by extracting a set of significant bytes and a set of excluded bytes from the detected file.

In yet another embodiment of the method, a set of significant bytes is extracted from the detected file by filtering the detected file using filtering algorithms.

In another embodiment of the method, the set of excluded bytes is obtained by removing the set of significant bytes from the detected file.

In another embodiment of the method, a language recognition rule is understood as a set of decision tree results, in the presence of which it is believed that a script written in a certain programming language has been detected.

In yet another embodiment of the method, a set of significant bytes is extracted from the detected file by filtering the detected file using filtering algorithms.

In another embodiment of the method, the detected file is processed by: removing all characters except visible characters from the ASCII table; line breaks, spaces, characters from the Unicode table, lines containing comments and individual marked lines that are specific to the recognized programming language; converting text to lowercase; specifying the start and end of string constants specific to the recognized programming language.

In another variant of the implementation of the method, a set of hash codes is generated by: dividing the processed file into structures containing a set of characters from 4 to 10 characters long; counting the number of occurrences of all identified constructs in the processed file; determining the type of hash code to be used using hash code generation rules; generating a set of hash codes from hash codes of a certain type.

In another embodiment of the method, a hash code generation rule is understood as a set of conditions under which a certain type of hash code is used.

In another embodiment of the method, the static parameters of the generated script summary are understood as a list of features calculated on the basis of a set of significant bytes, the number of occurrences of comments, lines containing symbolic expressions, constructs containing expressions, known script programming languages.

In another embodiment of the method, the dynamic parameters of the generated script summary are understood as a list of features calculated based on the number of occurrences of each type of constructs containing a set of characters specific to each programming language in a set of significant bytes.

In one implementation, a system is provided that contains at least one computer, including means interacting with each other: a detection tool, a counting tool, a generation tool, a detection tool, a rules database, a script hash code database and storing machine-readable instructions, when executing which the system performs detection of a malicious script based on a set of hash codes according to the method according to any one of paragraphs. 1-11.

Brief description of the drawings

Fig. 1 illustrates an example of a file containing script source code.

Fig. 2 illustrates the structure of a malicious script detection system based on a set of hash codes.

Fig. 3 illustrates the algorithm of the system for detecting a malicious script based on a set of hash codes.

Fig. 4 represents an example of a general purpose computer system, a personal computer or a server.

Although the invention may have various modifications and alternative forms, the characteristic features shown by way of example in the drawings will be described in detail. It should be understood, however, that the purpose of the description is not to limit the invention to a particular embodiment thereof. On the contrary, the purpose of the description is to cover all changes, modifications included in the scope of this invention, as defined in the attached claims.

Description of embodiments of the invention

The objects and features of the present invention, methods for achieving these objects and features will become apparent by reference to exemplary embodiments. However, the present invention is not limited to the exemplary embodiments disclosed below, but may be embodied in various forms. The gist of the description is nothing but the specific details necessary to assist a person skilled in the art in a thorough understanding of the invention, and the present invention is defined within the scope of the appended claims.

We introduce a number of definitions and concepts that will be used in describing embodiments of the invention.

A hash function is an algorithm that converts an arbitrary-length string (message) into a fixed-length bit string called a hash code, checksum, or fingerprint. Many security technologies use one-way encryption functions, also called hash functions. The main purpose of such functions is to obtain from the message an arbitrary size of its hash code - a value of a fixed size. The hash code can be used as a checksum of the original message, thus providing (when using the appropriate protocol) information integrity control (J. Brassard. Modern cryptology / J. Brassard - M.: POLYMED, 1999. - 178 p.).

Scenario, script (English script, scenario) - a program (a special kind of program code), usually written in some interpreted (not compiled) language and containing separate instruction commands. For example, a script is included in the text of a web page in the form of source code and is interpreted by the system installed on the computer of the remote user who requested this page.

Java script (eng. Java script) - a script in the form of text in the Java programming language or javascript scripting language, included directly in a web page and loaded with it. It is performed by the Java interpreter, which is built into almost every modern browser (Dorot V. L., Novikov F. A., Explanatory Dictionary of Modern Computer Vocabulary. - 3rd from D., revised and added. - St. Petersburg: BHV-Petersburg , 2004. - 608 p.: ill.).

Attackers use the ability of scripts to access computer system resources for malicious purposes. The currently existing data transfer methods allow you to mask the script in a file of any type and force the user to activate it in one way or another. To detect a script of a known script programming language or its part, a system for detecting a malicious script based on a set of hash codes is used. Fig 1. illustrates an example of a file containing the source code of the script. The file contains script 1 written in the Bash 110 programming language. Bash command-line scripts are sets of commands for the Linux command line that can be entered from the keyboard, collected in files and united by a common purpose. At the same time, the results of the work of commands can either be of independent value, or serve as input data for other commands and files.

Fig. 2 illustrates the structure of a malicious script detection system based on a set of hash codes, which includes a detection engine 220 , a counting engine 230 , a generation engine 240 , a detection engine 250 , a rule database 260 , a script hash code database 270 .

Shown in FIG. 2 , the system is a computer system, such as the general purpose computer shown in FIG. 4 and having a hardware processor and memory. Said system contains functional and/or hardware modules and means 220 - 270 , which in turn contain instructions for execution on a hardware processor. The embodiments of said modules and means 220-270 of the system are described below .

The detection tool 220 is designed to detect a file containing a script. The detection tool 220 detects the file containing the script by parsing each file out of a plurality of files that need to be checked for malicious code 210 . The plurality of files 210 may contain various files that the user has interacted with in one form or another, including website and web page files, email message files, and so on. The analysis is performed using heuristic algorithms that are designed to search for structured file types in which the presence of scripts is unlikely (for example, executable files in PE/ELF/MACHO format, archives, certain types of document files, etc.). The mentioned files found are excluded from the set of files 210 , the presence of malicious code in which must be checked. The files remaining after the exclusion are considered files that may contain malicious code.

In addition, the finder 220 is configured to generate a script summary based on the detected file, and transmit data about the detected file and data about the generated script summary to the calculator 230 .

The finder 220 generates a script summary by extracting a set of significant bytes and a set of excluded bytes from the discovered file. Next, based on the analysis of the selected sets, the discoverer 220 populates the script summary.

A set of significant bytes is extracted from the detected file by filtering the detected file using filtering algorithms.

Filtering algorithms include the following actions:

• selection of the part of the detected file, which presumably contains the script;

• removal of all characters, except for visible characters, from the ASCII table;

• removal of line breaks, spaces, characters from the Unicode table;

• reduction of the text to lower case;

• deletion of lines containing comments of popular script programming languages;

• deleting lines containing the expression "arbitrary text $ arbitrary text and punctuation mark";

• deletion of lines containing continuous expressions (without spaces, hyphens, punctuation marks) longer than the character limit. The length limit is a configurable setting.

The set of excluded bytes is obtained by removing the set of significant bytes from the found file.

A script summary is a data structure calculated based on the parameters of a detected file that characterizes the script contained in the file. The script summary contains the following data:

• set of significant bytes;

• set of excluded bytes;

• the number of bytes in the form of comments of each template from the set of excluded bytes;

• the size of the array, which occupies a set of significant bytes;

• the number of lines containing at least one significant byte;

• the number of lines containing the expression "[arbitrary text (a set of arbitrary one or more characters)]" or part of it;

• the number of lines containing the expression "arbitrary text = arbitrary text" or part of it;

• the number of lines containing the expression "$ = arbitrary text" or part of it;

• the number of constructions containing the expression "if arbitrary text - fi arbitrary text" or part of it;

• the number of constructions containing the expression "arbitrary text do arbitrary text" or part of it;

• number of constructions containing the expression "arbitrary text case - esac arbitrary text" or its part;

• the number of constructions containing the expression "var arbitrary text" or part of it;

• the number of constructions containing the expression "def arbitrary text" or part of it;

• the number of constructions containing the expression "use arbitrary text" or part of it;

• the number of constructions containing the expression "arbitrary number: arbitrary number" or part of it;

• the number of lines starting with a character from the Latin alphabet;

• the number of lines containing only one character ":" among all characters in the line;

• the number of lines in which the last character is the character "\";

• the number of lines in which the last character is the character ":";

• the number of lines in which the last character is the character ";";

• total number of lines;

• number of significant lines;

• the number of bytes read from the file;

• the presence of a set of shebang characters "#!/bin/bash".

Data about the detected file includes the file itself and the data obtained as a result of its analysis.

The calculator 230 is designed to calculate the static and dynamic parameters of the generated script summary. The static parameters of the generated script summary are the following list of features:

• entropy of an array that occupies a set of significant bytes;

• the number of occurrences of each of the possible significant bytes in the array that occupies the set of significant bytes, divided by the total number of significant bytes and multiplied by 1024;

• the number of excluded comments of each type divided by the total number of excluded comments multiplied by 1024;

• the presence of a set of shebang characters: 512 if yes, 0 otherwise;

• the number of lines containing the expression "[free text]" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of lines containing the expression "free text = free text" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of lines containing the expression "$ = free text" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of structures containing the expression "if arbitrary text - fi arbitrary text" or its part, divided by the number of significant lines and multiplied by 1024;

• the number of constructs containing the expression "arbitrary text do arbitrary text" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of structures containing the expression "arbitrary text case - esac arbitrary text" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of constructs containing the expression "var arbitrary text" or part of it, divided by the number of significant lines and multiplied by 1024

• the number of constructs containing the expression "def arbitrary text" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of constructs containing the expression "use free text" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of constructs containing the expression "arbitrary number: arbitrary number" or part of it, divided by the number of significant lines and multiplied by 1024;

• the number of lines starting with a character from the Latin alphabet, divided by the number of significant lines and multiplied by 1024;

• the number of lines containing only one ":" character among all characters in the line, divided by the number of significant lines and multiplied by 1024;

• the number of lines in which the last character is the character "\";

• the number of lines where the last character is "\" divided by the number of significant lines and multiplied by 1024;

• the number of lines in which the last character is the character ":", divided by the number of significant lines and multiplied by 1024;

• the number of lines in which the last character is the character ";", divided by the number of significant lines and multiplied by 1024;

• the number of lines beginning with Latin characters divided by the number of significant lines and multiplied by 1024;

The dynamic parameters of the generated script summary are understood as the number of occurrences of each type of constructs containing a set of characters characteristic of each programming language in the set of significant bytes, divided by the total number of occurrences of all constructs and multiplied by 1024. For the Bash language, for example, the number of occurrences in a set of meaningful bytes of constructs containing a set of characters from the following dynamic list of character sets: my$; $$; "${; $!; $?; $x; $#; $*; $@; $-; $_; $(; $0; $1; $2; $3; ./; ==; !=; <= ; >=; =~; --; +r; +w; +x; -a; -b; -c; -e; -f; -g; -h; -i; -j; -k; - l; -m; -n; -o; -p; -q; -r; -s; -t; -u; -v; -w; -x; -y; -z; >>; <<; /bin; /boot; /dev; /etc; /opt; /X11; /sgml; /xml; /home; /lib; /media; /mnt; /opt; /proc; /root; /run; /sbin ; /srv; /sys; /tmp; /usr; /bin; /include; /lib; /local; /sbin; /share; /src; /var; /cache; /lib; /lock; /log; / mail; /run; /spool; /mail; /tmp; breaksw; endsw; *); if; fi; then; else; elif; case; esac; [[; ]]; true; false; do; done; while ; for; until; break; continue; select; function; if["$; if[$; if("$; if($; if[-d; if[!-d; if[-f; if[!-f; iftest; ;then; case"$; echo$; echo"$; sed-; grep-; ne-; while[$; ]do; whileread; test; apt; zypp; dnf; dig; chown; sync; chsh; chattr; puts; rm-; rm*; rpm; fold; rmdir; less; more ; readelf; mv; pg_dump; popd; last; @echooff; @echoon; echooff; echoon; scp; pwd; logout; find; lsof; uuencode; yum; cat; clear; xmllint; stat; ssh-copy-i; Prints ; printf; host; chmod; split; xxd; top; cd; od; sed; bind; whoami; ionice; touch; screen; diff; set; dirname; ifconfig; uniq; uname; cpio; eval; brew; awk; which ; shopt; pkg; pdfunite; bunzip2; watch; ifconfig; bzip2; join; exec; ...; seq; xhost; bash; unalias; env; comm; node; ping; file; zcat; cowsay; tac; unzip; which ; netstat; pstree; readlink; getent; lspci; /bin/sh; apropos; ssh; cd$; bar; ant; ipcs; ldd; vagrant; tail; .sh; tar; hostname; setarch; mate; kill; cut; alias; nohup; trap; chgrp; grep; wget; lshw; rev; locate; shred; sshpass; make; ssh-keygen; nslookup; tee; mktemp; shift; objdump; gunzip; m ount; cron; stdbuf; pushd; groups; qstat; xargs; tmux; sudo; mkdir; strace; timex; mvn; tree; eof; eod; exit$; /dev/null; expect; ../; pkill; 2>&1;install;gcc; python perl; java; .so; "$@"; http; ->; class; module; import; def; std::; var; function; usestrict; export; default; jQuery; const; from; namespace; public; private; protected; interface; usewarnind; uselib; foreach; my$; eval; =>; package; sub; usevars; useparent; $self; <-; body{; typedef; @shift; window; microsoft; font; .PHONY; CMAKE_; PROJECT_; CTEST_; CPACK_; gem::; gem"; gemspec; docker; ENTRYPOINT; post; host:; user-agent; Accept:; DNT:; Content-Ty; Content-Le; Disass; PUSHBUTTON; COMBOBOX; LEXT; CONTROL; EDITTEXT; EDITTEXT; IDS_; DS_; WS_; IDD_; <tr><td. Sets of characters specific to a particular language are updated using machine learning methods and mathematical statistics. First of all, keywords, expressions, names or parts of names of functions, classes, variables from popular libraries are subject to consideration certain programming language.

The occurrences are counted by calculating the values of the rolling hash code (English rolling hash function) over a set of significant bytes. Optimal in terms of complexity and speed of calculation is a sequence of bytes of 10 bytes. Starting from the first byte from the byte sequence, said hash code is calculated. Then the hash code is shifted by one byte, and already from the second byte, the said hash code is again calculated from the byte sequence. Next, repeat the above action for the entire set of significant bytes. After that, the number of occurrences of hash codes from the mentioned structures is counted up to the accuracy of their falling into the Bloom's probabilistic set.

In addition, the calculator 230 is adapted to recognize the script programming language based on the calculated parameters of the generated script summary using the language recognition rules from the rule database 260 and transmit data about the detected file, data about the generated script summary and data about the recognized programming language of the script to the generation engine. 240 .

Recognition of the programming language of the script is carried out using machine learning methods. The calculated parameters are used as features that are fed to the input of pre-trained machine learning models in the form of decision trees.

This machine learning model uses five decision tree patterns to validate a programming language. For each programming language, a set of features is formed and used to train decision trees. As a result of training, five decision trees are formed for one language. The result of each tree is confirmation or lack of confirmation of whether the script in question is written in the current programming language. Based on the results of the work of all five decision trees to confirm the current programming language, a final conclusion can be drawn. In addition to sets of trees for confirming the script programming language, trees are created for confirming alternative languages, for example, the language of the configuration file (ini file), the PostScript programming language (pdf file). These decision trees are used to reduce false positives.

Language recognition rule - a set of results of the work of all decision trees for confirming a programming language for recognizing all languages, in the presence of which it is believed that the identified script was written in a certain programming language.

An example of a language recognition rule is the following set of decision tree outcomes: at least 3 out of 5 decision trees to validate one programming language gave a positive result; decision trees to validate other languages yielded no more than 2 positive results for each language. When the above set of conditions is met, it is considered that the identified script is written in a programming language, upon recognition of which 3 out of 5 decision trees gave a positive result.

Another example of a language recognition rule is the following set of decision tree results: at least 3 out of 5 decision trees to validate one decision language yielded a positive result; decision trees to validate other languages gave no more than 2 positive results for one language; at least 4 out of 5 decision trees for recognizing one alternative language gave a positive result. When the above set of conditions is met, it is considered that the identified script is written in an alternative language, upon recognition of which 4 out of 5 decision trees gave a positive result.

Another example of a language recognition rule is the following set of decision tree results: at least 3 out of 5 decision trees for confirming one language gave a positive result; at least 3 out of 5 decision trees for recognizing another language gave a positive result. When the above set of conditions is met, it is considered that it is necessary to use an additional decision tree to refine the result of the decision trees that gave a positive result.

The generator 240 is adapted to process the detected file based on the recognized programming language, thereby converting the detected file to a specific form.

Creator 240 processes the detected file by:

• removing all characters except visible characters from the ASCII table; line break characters, spaces, characters from the Unicode table; lines containing comments and individual marked lines specific to the recognized programming language;

• converting text to lower case;

• specifying the beginning and end of string constants specific to the recognized programming language.

In addition, generator 240 is configured to generate a hash code set based on the processed file using hash code generation rules from rule database 260 and pass the generated hash code set to finder 250 .

Generator 240 performs the generation of a set of hash codes by performing the following list of actions:

1) division of the processed file into structures containing a set of characters from 4 to 10 characters long;

2) counting the number of occurrences of all identified constructs containing a set of characters with a length of 4 to 10 characters in the processed file; for example, if the processed file is represented as the characters "colacola", then as a result of division into constructs, the following array is formed {cola: 2, colacola: 1, olacola: 1, lacola: 1, acola: 1, colac: 1, colaco: 1, colacol: 1};

3) using the rules for generating hash codes to determine the type of hash code used;

4) generating a set of hash codes from hash codes of a certain type.

The counting of the number of occurrences of each type of constructs from all programming languages containing a set of characters from 4 to 10 characters in length in the processed file is performed by calculating a sliding hash code built from parts of the file in the form of a byte sequence of 10 bytes, where the number of characters in character set has a length of 4 to 10 characters. The number of occurrences of the hash codes from the mentioned constructs is calculated up to their hit in the Bloom's probability set (the size of which coincides with the size of the Bloom's probability set used at the stage of language recognition).

A hash code generation rule is a set of conditions under which one or another type of hash code is used. The counted number of occurrences is the main condition for creating rules for generating hash codes. An example of a hash code generation rule is when the number of occurrences is below the limit. When this set of conditions is met, a shortened type of hash codes is used when generating a set of hash codes. Another example of a hash code generation rule is when the number of occurrences is equal to or greater than the limit value. When this set of conditions is met, the full-length hash code type is used when generating a set of hash codes. The limit value of the number of occurrences can be determined statistically, empirically or set by an expert, established as a result of the operation of machine learning algorithms.

The shortened type of hash codes is a 64-bit value obtained from the combination of two halves of the 128-bit hash function murmur3 (eng. MurmurHash3 hash function) from the processed file.

Full-length type of hash codes - type of hash codes that count from the pair "hash code from a set of characters - the number of occurrences of the mentioned hash code at the stage of processing the detected file when counting sliding hash codes", using the murmur hash calculation algorithm 3, xor and cosine distance. Moreover, if there are several pairs with different hash codes, but the same number of occurrences, then only one of these pairs can be used to calculate the full-length hash code.

The detection tool 250 is designed to detect a malicious script by comparing the generated hash code set with known malicious hash code sets from the script hash code database 270 . Comparison is mainly carried out using algorithms locally sensitive hashing (Local sensitive hash function) and detection of the minimum cosine distance. If the generated set of hash codes is similar to known malicious hash codes, the script is recognized as containing malicious code.

The rule database 260 is designed to store language recognition rules and hash code generation rules. The database of hash codes of scripts 270 is designed to store known hash codes, sets of hash codes of known scripts, including malicious ones. Various types of databases can be used as a database of rules 260 and a database of script hash codes 270 , namely: hierarchical (IMS, TDMS, System 2000), network (Cerebrum, Сronospro, DBVist), relational (DB2, Informix, Microsoft SQL Server), object-oriented (Jasmine, Versant, POET), object-relational (Oracle Database, PostgreSQL, FirstSQL/J), functional, etc. Rules can be created using machine learning algorithms and automated processing of large data sets.

Below are the implementation details in a simplified form.

For script 1 from file 110 , the set of significant bytes would look like this:

prog=$(basename$0ctl)admecho1>&2"$(basename$0)depricated,pleaseuse$proginthefuture"if[$#-lt2];thenecho"usage:$(basename$0)-[deu]user[...] "1>&2exit1fiflag=$1shiftfori;doargs="$flag$i"doneecho1>&2"executing:$prog$args"exec$prog$args

For script 1, the summary of the script would look like this:

{

commentsCount = {0, 0, 0, 0, 0, 0, 0, 0, 0};

cleanBytes = " prog=$(basename$0ctl)admecho1>&2"$(basename$0)depricated,pleaseuse$proginthefuture"if[$#-lt2];thenecho"usage:$(basename$0)-[deu]user[. ..]"1>&2exit1fiflag=$1shiftfori;doargs="$flag$i"doneecho1>&2"executing:$prog$args"exec$prog$args";

cleanBytesCount = 230;

significantLinesCount = 13;

linesCount = 13;

iniSectionsCount = 0;

assignmentsCount = 3;

psAssignmentsCount = 0;

ifFiCount = 1;

bashCycleCount = 1;

caseEsacCount = 0;

varCount = 0;

defCount = 0;

useCount = 0;

timestampCount = 0;

bytesRawRead = 293;

linesStartedWithLetterCount = 13;

singleColonInLine = 2;

lastSlashCount = 0;

lastColonCount = 0;

lastSemicolonCount = 0;

hasShebang = false;

}

Full feature set of script 1 passed to the machine learning model for recognition of the Bash programming language:

[4919, 0, 35, 4, 66, 0, 13, 0, 13, 13, 0, 0, 4, 8, 13, 0, 13, 22, 17, 0, 0, 0, 0, 0, 0 , 0, 8, 8, 0, 13, 13, 0, 0, 13, 0, 13, 0, 0, 0, 66, 13, 31, 26, 115, 31, 48, 26, 40, 0, 0 , 22, 17, 31, 44, 26, 0, 48, 48, 40, 31, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 , 0, 0, 0, 0, 236, 0, 78, 78, 0, 0, 0, 0, 0, 1024, 157, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0 , 0, 0, 0, 0, 8, 16, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0 , 0, 0, 8, 8, 0, 0, 0, 0, 16, 0, 0, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 8, 0, 8, 8 , 8, 16, 24, 8, 0, 24, 8, 0, 0, 0, 0, 0, 0, 8, 0, 8, 0, 0, 0, 0, 8, 0, 8, 0, 0 , 8, 0, 0, 0, 16, 0, 8, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 8, 0, 8, 0, 8, 0 , 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 8, 0, 0, 0, 0, 8, 0, 0, 0, 8 , 0, 8, 8, 0, 0, 0, 0, 0, 0, 0, 8, 0, 16, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0 , 0, 8, 0, 8, 0, 0, 0, 16, 0, 0, 8, 0, 8, 0, 8, 0, 0, 0, 0, 8, 0, 0, 0, 8, 0 , 8, 0, 0, 0, 0, 0, 0, 0, 8, 8, 16, 8, 0, 0, 0, 24, 8, 0, 16, 0, 0, 0, 8, 0, 0 , 0, 0, 8, 0, 0, 0, 0, 8, 16, 8, 0, 0, 8, 0, 0, 0, 0, 0, 8, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 16, 8, 0, 0, 0, 0, 0, 0, 0, 8, 0, 8, 8, 0, 16, 8, 0, 0, 8, 0, 0, 8, 8, 0, 8, 8, 8, 0, 0, 0, 0, 0, 8, 0, 0, 0, 8, 24, 0, 0, 0, 0, 0, 8, 8, 8, 16, 0, 8, 8, 0, 0, 0, 0, 16, 0, 0, 8, 0, 0, 8, 0, 0, 0, 8, 0, 0, 8, 0, 0, 8, 0, 0, 8, 0, 0, 24, 8, 8, 0, 0, 0, 8, 8, 0, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0,]

All five decision trees for Bash validation were positive. An example of the implementation of one of the trees: def dt_code(features_original):

features = [0] + [feature * 2 for feature in features_original]

if features[76] <= 3: # CommentStyle.POWERSHELL_MULTILINE_COMMENT

if features[81] <= 178: # has_shebang

if features[79] <= 25: # assignment_share

if features[31] <= 0: # >

if features[81] <= 50: # has_shebang

if features[357] <= 39: # -reference

if features[14] <= 6: # -

if features[68] <= 94: # }

if features[400] <= 72: # break

if features[14] <= 1: # -

if features[382] <= 61: # $psculture

if features[49] <= 73: # j

if features[424] <= 272: # $($

if features[61] <= 250: # v

if features[387] <= 36: # [system.

if features[256] <= 128: #param

if features[420] <= 82: # .ps

if features[269] <= 49: # ]$

if features[12] <= 149: # +

if features[297] <= 298: # format-

if features[324] <= 163: # able-

if features[65] <= 106: #z

if features[367] <= 137: # ext.encodi

if features[5] <= 10: # $

if features[216] <= 59: # -type

if features[244] <= 151: # byte[]

if features[58] <= 201: # s

if features[367] <= 110: # ext.encodi

if features[233] <= 120: # select-

if features[212] <= 67: # @(

if features[362] <= 155: # -lt

if features[37] <= 71: # ^

if features[420] <= 38: # .ps

if features[315] <= 21: # -not

if features[16] <= 185: # /

if features[141] <= 74: # elseif

if features[245] <= 103: # ynamicpara

if features[178] <= 100: # -xor

if features[43] <= 205: # d

if features[162] <= 107: # -bnot

if features[319] <= 33: # -path

if features[92] <= 926: # last_colon_count

if features[221] <= 72: # iex

if features[95] <= 4: # -bxor$

if features[1] <= 5176: # entropy

if features[209] <= 42: # >>

if features[339] <= 60: # -as

if features[219] <= 31: # @{

if features[293] <= 158: # in

if features[18] <= 211: # 1

if features[376] <= 7: # register-

if features[251] <= 5: # -error

return [22482, 0]

else: # if features[251] > 5 # -error

if features[286] <= 71: # [intptr]

if features[145] <= 29: # -and$

if features[377] <= 30: # -in$

if features[320] <= 19: # -match

return [2700, 0] ……

All other trees tested negative. The rule was triggered, so this detected file is recognized as a Bash script. Next, the processed file is formed, and then a set of hash codes:

prog=$(basename$0ctl)admecho1>&2"$(basename$0)depricated,pleaseuse$proginthefuture"if[$echo"usage:$(basename$0)-[deu]user[...]"1>&2exit1fiflag= $1shiftfori;doargs="$flag$i"doneecho1>&2"executing:$prog$args"exec$prog$args

It is impossible to build a full-length hash code, since the rule did not work, and only two pairs (hash entry) were found: (830, 4), (69, 3). Accordingly, the calculation of the reduced type of hash codes will be performed. And since the processed file has a small size, the hash code will be one and equal to 0xd43903d3f56d64c1.

Fig. 3 illustrates the algorithm of the system for detecting a malicious script based on a set of hash codes. At step 311 , the finder 220 detects the file containing the script. In step 312 , the finder 220 generates a script summary based on the detected file, and passes the detected file data and the generated script summary data to the enumerator 230 . In step 313 , the calculator 230 calculates the static and dynamic parameters of the generated script summary. In step 314 , the calculator 230 performs script programming language recognition based on the calculated parameters of the generated script summary using the language recognition rules from the rule database 260 , and transmits the detected file data, the generated script summary data, and the recognized script programming language data to the generating engine 240 . At step 315 , the generator 240 processes the detected file based on the recognized programming language. At step 316 , generator 240 generates a hash code set based on the processed file using hash code generation rules from rule database 260 , and passes the generated hash code set to detector 250 . At step 317 , the detection engine 250 checks whether the generated hash code set is similar to the known malicious hash code sets from the script hash code database 270 . If there is a similarity to known malicious hash codes, at step 319 the detection engine 250 detects the malicious script. Otherwise, at block 318 , the system exits.

Fig. 4 shows an example of a general purpose computer system, a personal computer or server 20 ', comprising a central processing unit 21 ', system memory 22 ', and a system bus 23 ', which contains various system components including memory associated with the central processing unit 21 '. The system bus 23 is implemented as any bus structure known in the art, in turn comprising a bus memory or bus memory controller, a peripheral bus, and a local bus capable of interfacing with any other bus architecture. The system memory contains read-only memory (ROM) 24 , random access memory (RAM) 25 . The main input/output system (BIOS) 26 contains the basic procedures that ensure the transfer of information between the elements of a personal computer 20 , for example, at the time of booting the operating system using ROM 24 .

The personal computer 20 , in turn, contains a hard disk 27 for reading and writing data, a magnetic disk drive 28 for reading and writing to removable magnetic disks 29 and an optical drive 30 for reading and writing to removable optical disks 31 , such as CD-ROM, DVD -ROM and other optical storage media. The hard disk 27 , the magnetic disk drive 28 , the optical drive 30 are connected to the system bus 23 via the hard disk interface 32 , the magnetic disk interface 33 , and the optical drive interface 34 , respectively. Drives and related computer storage media are non-volatile means of storing computer instructions, data structures, program modules, and other personal computer data 20 .

The present description discloses an implementation of a system that uses a hard disk 27 , a removable magnetic disk 29 and a removable optical disk 31 , but it should be understood that other types of computer storage media 56 that are capable of storing data in a computer-readable form (solid-state drives, flash memory cards, digital disks, random access memory (RAM), etc.), which are connected to the system bus 23 through the controller 55 .

The computer 20 has a file system 36 that stores the recorded operating system 35 as well as additional software applications 37 , other program modules 38 and program data 39 . The user has the ability to enter commands and information into the personal computer 20 through input devices (keyboard 40 , mouse 42 ). Other input devices (not shown) may be used: microphone, joystick, game console, scanner, etc. Such input devices are typically connected to the computer system 20 through a serial port 46 which is in turn connected to the system bus, but may be connected in other ways, such as through a parallel port, game port, or universal serial bus (USB). A monitor 47 or other type of display device is also connected to the system bus 23 via an interface such as a video adapter 48 '. In addition to the monitor 47 , the personal computer may be equipped with other peripheral output devices (not shown), such as speakers, a printer, etc.

The personal computer 20 is capable of operating in a networked environment using a network connection to another or more remote computers 49 . The remote computer (or computers) 49 are the same personal computers or servers that have most or all of the elements mentioned earlier in the description of the nature of the personal computer 20 shown in FIG. 4 . Other devices may also be present in the computer network, such as routers, network stations, peer-to-peer devices, or other network nodes.

The network connections may form a local area network (LAN) 50 and a wide area network (WAN). Such networks are used in corporate computer networks, internal networks of companies and, as a rule, have access to the Internet. In LAN or WAN networks, the personal computer 20 is connected to the local network 50 via a network adapter or network interface 51 . When using networks, personal computer 20 may use a modem 54 or other means to communicate with a wide area network, such as the Internet. Modem 54 , which is an internal or external device, is connected to system bus 23 via serial port 46 . It should be clarified that network connections are only indicative and are not required to represent the exact network configuration, i.e. in fact, there are other ways to establish a connection by technical means of communication from one computer to another.

In conclusion, it should be noted that the information given in the description are examples that do not limit the scope of the present invention defined by the formula.

Claims (19)

1. A method for detecting a malicious script based on a set of hash codes, comprising steps in which a computer is used to perform operations, which include steps in which: a) detecting a file containing a script by analyzing each file from a plurality of files, which are checked for the presence of malicious code; b) generating a script summary based on the detected file; c) calculate the static and dynamic parameters contained in the generated script summary; d) recognizing the programming language of the script based on the calculated parameters of the generated script summary using language recognition rules; e) processing the detected file based on the recognized programming language data and the detected file data; f) generating a set of hash codes based on the processed file using hash code generation rules; g) detecting a malicious script by comparing the generated set of hash codes with known malicious hash codes from the database of script hash codes. 2. The method according to claim 1, in which, as a result of analyzing each file from a set of files that are checked for malicious code, using heuristic algorithms designed to search for structured file types in which the presence of scripts is unlikely, the found files are excluded from the specified set files. 3. The method according to claim 1, which generates a summary of the script by extracting a set of significant bytes and a set of excluded bytes from the detected file. 4. The method of claim 3, wherein the set of significant bytes is extracted from the detected file by filtering the detected file using filtering algorithms. 5. The method of claim 4, wherein the set of excluded bytes is obtained by removing the set of significant bytes from the detected file. 6. The method according to claim 1, in which the language recognition rule is understood as a set of decision tree results, in the presence of which it is considered that a script written in a certain programming language has been detected. 7. The method according to p. 1, which process the detected file by: removing all characters except visible characters from the ASCII table; line breaks, spaces, characters from the Unicode table, lines containing comments and individual marked lines that are specific to the recognized programming language; converting text to lower case; specifying the start and end of string constants specific to the recognized programming language. 8. The method according to p. 1, which perform the formation of a set of hash codes by: dividing the processed file into structures containing a set of characters from 4 to 10 characters long; counting the number of occurrences of all identified constructs in the processed file; determining the type of hash code to be used using hash code generation rules; generating a set of hash codes from hash codes of a certain type. 9. Method according to. 1, in which a hash code generation rule is understood as a set of conditions under which a certain type of hash code is used. 10. The method according to claim 1, in which the static parameters of the generated script summary are understood as a list of features calculated based on a set of significant bytes, the number of occurrences of comments, lines containing symbolic expressions, constructs containing expressions, known script programming languages. 11. The method according to claim 1, in which the dynamic parameters of the generated script summary are understood as a list of features calculated based on the number of occurrences of each type of constructs containing a set of characters specific to each programming language in a set of significant bytes. 12. A system containing at least one computer, including means interacting with each other: a detection tool, a counting tool, a generation tool, a detection tool, a rules database, a script hash code database and storing machine-readable instructions, upon execution of which the system performs detection malicious script based on a set of hash codes according to the method according to any one of paragraphs. 1-11.

Images