RU2790330C2 - Method for determination of phishing electronic message - Google Patents

Abstract

FIELD: information security.

SUBSTANCE: method for determination of a phishing electronic message uses at least two machine learning models, while, using the first model, an electronic message is determined as suspicious based on the first type attributes of an intercepted electronic message, and, using the second model, a suspicious electronic message is determined as phishing based on the second type attributes.

EFFECT: increase in the accuracy of determination of a phishing electronic message, reduction in a number of false determinations of a phishing electronic message, reduction in time and resource costs for use of a resource-intensive second machine learning model by preliminary determination of an electronic message as suspicious, using the first model.

7 cl, 4 dwg

Description

Technical field

The invention relates to the field of information security by blocking a phishing email.

State of the art

Phishing is an illegal act carried out in order to force the victim to share confidential information, such as a password or credit card number. Most often, scammers try to trick the user into visiting a fake site and entering their data on it - a login name, password or PIN.

To attract a victim to a trap site, attackers use mass or targeted mailing of electronic messages that disguise themselves as messages sent by a work colleague, a bank employee, or a representative of a government institution, but at the same time contain a malicious link. The text requires the victim to follow the link and immediately perform certain actions in order to avoid danger or any serious consequences. Another option involves using a file attachment that also contains malicious links or uses exploits for vulnerable applications to further compromise the user's computer.

By clicking on the link, the victim is taken to a phishing site, where they are offered to “log in” using their account (some scammers go even further, requiring them to send them copies of documents or a photo with an identity card). If the victim is gullible enough and agrees, then the transmitted data goes directly to the attackers, who use it to steal confidential information or money.

There are two main types of fraud detection schemes. The first type refers to schemes that detect phishing based on the analysis of the content of the target web pages, that is, the analysis of web pages that are linked in an email (or in attached documents). The second type are schemes that work directly with the content of electronic messages.

US Patent Publication 20170085584 A1 describes a technology for detecting a phishing email. Based on the comparison of the address of the sender of the message with the address from the database of legitimate known senders, suspicious letters are determined. If the sender is not in the database and the degree of similarity to the legitimate sender is higher than a predetermined value, the user is notified of a suspicious e-mail.

Although the method described above does a good job of recognizing targeted mailings that imitate emails from trusted senders, it does not allow you to recognize phishing messages from unknown senders. In addition, the definition of phishing based on the degree of similarity of domains can discredit a legitimate sender. Instead, a layered approach should be taken to reduce the number of attacks and false positives for phishing messages.

The present invention allows solving the problem of ensuring information security by blocking a phishing email message.

Disclosure of invention

The invention is intended to provide information security by blocking a phishing email message.

The technical result of the present invention is to identify a phishing email message. Another technical result of the present invention is to reduce the number of false positives of a phishing email. Another technical result is to ensure information security by blocking a phishing email message.

These results are achieved by using a method for determining a phishing email based on the use of at least two machine learning models, with the first model determining the email as suspicious, and using the second model determining the suspicious email as phishing.

In a particular case of the implementation of the method, the first model is trained in advance on the attributes of electronic messages, which include at least: the value of the header of the electronic message Message_ID; X-mailer email header value; sequence of e-mail header values.

In yet another particular case of the implementation of the method, an electronic message identified as suspicious is additionally placed in a temporary quarantine.

In another particular case of the implementation of the method, the second model is trained in advance on the attributes of electronic messages, which include at least: the reputation of a set of links, which characterizes the probability that the electronic message contains a phishing link; the category of the email message; the presence flag of the sender's domain in a pre-formed database of unwanted senders; the presence flag of the sender's domain in a pre-formed database of known senders; the degree of similarity of the sender's domain with domains in a pre-formed database of known senders; flag for the presence of HTML-code in the body of the e-mail; flag for the presence of scripted inserts in the body of the e-mail.

In another particular case of implementing the method, the reputation of a set of links is calculated using a recurrent neural network.

In another particular case of the implementation of the method, N-grams of the text of the electronic message are used as the category of the electronic message, determined by selecting the most important features that most strongly affect the result of the binary classification of the phishing electronic message.

In another particular case of the implementation of the method, the classification of a phishing email message is performed based on the logistic regression algorithm with regularization.

In another particular case of the implementation of the method as a machine learning algorithm of the second model is at least: Bayesian classifiers; logistic regression; modified random forest learning algorithm; support vector method; nearest neighbor methods; decision tree.

In another particular case, the implementation of the method additionally provide information security by at least: blocking a phishing email message; informing the recipient about the phishing nature of the electronic message; placing the identifier of the phishing email in the database of malicious messages.

Brief description of the drawings

Fig. 1 illustrates an example system for collecting and storing email message attributes.

Fig. 2 illustrates an example system implementing a method for detecting a phishing email.

Fig. 3 illustrates a method for detecting a phishing email.

Fig. 4 represents an example of a general purpose computer system, a personal computer or a server.

Although the invention may have various modifications and alternative forms, the characteristic features shown by way of example in the drawings will be described in detail. It should be understood, however, that the purpose of the description is not to limit the invention to a particular embodiment thereof. On the contrary, the purpose of the description is to cover all changes, modifications, included in the scope of this invention, as defined by the attached claims.

Description of embodiments of the invention

The objects and features of the present invention, methods for achieving these objects and features will become apparent by reference to exemplary embodiments. However, the present invention is not limited to the exemplary embodiments disclosed below, but may be embodied in various forms. The gist of the description is nothing but the specific details necessary to assist a person skilled in the art in a thorough understanding of the invention, and the present invention is defined within the scope of the appended claims.

Fig. 1 illustrates an example system for collecting and storing email message attributes.

A block diagram of an exemplary system for collecting and storing attributes of an electronic message contains a communication network 100, a user device 110, an electronic message 111, attributes #1140, an agent 120, a data storage device 130, Model #1150.

The communication network 100 is a system of physical communication channels that implements a protocol for transmitting an electronic message 111 between terminal devices, as well as transmitting attributes No. 1140 to a data storage device 130.

The electronic message 111 has a certain structure. It contains the body (eng. body) and headers (eng. header) - service information about the route of the messages. For example, the headers contain data about when, from where and by what route the letter came, as well as information added to the letter by various utility programs (mail clients).

As attributes No. 1140 are the values of headers relating to information about the route of the electronic message 111, as well as service information generated by mail clients.

For example, attributes #1140 are at least:

• Message_ID - unique identifier of the electronic message 111, assigned by the first mail server that it meets on the way;

• X-mailer (mailer_name) - the value of the header field in which the mail client or service with which the email 111 was created identifies itself;

• the sequence of values for the headers of the e-mail 111.

User device 110 includes an email client and an agent 120. Using the email client, user device 110 generates and transmits email 111 over communication network 100 and also receives email 111 from other devices.

Agent 120 intercepts email 111 at least:

• monitoring traffic received and transmitted via mail protocols (POP3, SMTP, IMAP, NNTP);

• keeping track of files in mail server storages;

• keeping track of files in mail client repositories.

The agent 120 determines the attributes #1140 contained in the intercepted email message 111 and transmits them to the storage device 130 via the communication network 100.

The data storage device 130 is designed to collect, store and process attributes #1140. For example, attributes #1140 are used to train Model #1150.

The storage device 130 is a cloud storage that processes the attributes of No. 1140 in the so-called "cloud", where "cloud" is a storage model that stores data on the Internet using a cloud computing resource provider that provides data storage as a service and provides managing them.

For example, the data storage device 130 can be a tool containing the Kaspersky Security Network (KSN) system from Kaspersky Lab JSC.

Fig. 2 illustrates an example system implementing a method for detecting a phishing email.

The block diagram of the phishing email detection system contains an email 111, an agent 120, a storage device 130, attributes #1140, attributes #2200, model #1150, a filtering tool 220, model #2230, a security tool 240.

Agent 120 is designed to intercept email 111, determine attributes #1140, attributes #2200, and transfer attributes #1140 to storage device 130.

Attributes #1140 are at least:

• the value of the email header 111 Message_ID;

• e-mail header value 111 X-mailer (mailername);

• the sequence of values for the headers of the e-mail 111.

Model #1150 is designed to classify email 111 based on attributes #1140. Model #1150 classifies email 111 as at least:

• suspicious (for example, containing spam, malicious attachment, phishing link);

• regular.

In one embodiment of the system, Model #1150 is pre-trained with attributes #1140 transferred to data storage device 130 such that Model #1150 determines, based on the specified attributes, features by which it classifies email 111 with some probability.

For example, Model No. 1150 can be based on deep learning methods. Specifically, attributes #1140 are represented as a matrix, where each symbol of attribute #1140 is encoded as a vector of fixed length numbers, and subjected to a neural network transformation that calculates the degree of similarity of said attributes to poison message attributes. The attributes No. 1140 transformed by the layer of the neural network act as features.

The filtering tool 220 is designed to put the email 111, which was classified as suspicious by Model #1150, into a temporary quarantine.

In one implementation of the system, the filter engine 220 temporarily quarantines an email message 111 that is more similar to a poison message than a predetermined value (eg, 0.7).

Model #2230 is designed to classify a suspicious email based on attributes #2200. Model #2230 classifies a suspicious email as at least:

• phishing;

• unknown.

Attributes #2 200 are at least:

• reputation of a set of links, which characterizes the probability that an email contains a phishing link;

• category of the electronic message;

• Flag of presence of the sender's domain in the pre-formed database of unwanted senders;

• Flag of presence of the sender's domain in the pre-formed database of known senders;

• the degree of similarity of the sender's domain with domains in a pre-formed database of known senders;

• flag of presence of HTML-code in the body of the electronic message;

• a flag for the presence of scripted inserts in the body of an electronic message.

In one implementation of the system, agent 120 calculates the reputation of a set of links using a recurrent neural network (RNN).

For example, agent 120 encodes a link URL string as a matrix of numbers (in particular, encodes each character of the URL as a fixed length vector), and then passes the encoded string to a recurrent neural network. The network extracts structural and semantic features from a URL and then uses an activation function to calculate how similar the extracted features are to those of phishing URLs. As a result, the link reputation is the probability that the link URL belongs to phishing URLs.

In another implementation of the system, the reputation of a set of links is a measure of the central tendency of the reputations of a set of links.

In one of the implementation options of the system, N-grams of the message text are used as the category of the electronic message, determined by selecting the most important features that most affect the result of the binary classification of the phishing email message.

For example, phishing emails often contain trigrams: “account will be blocked”, “you won money”, “change your password urgently”, which appeal to the emotions of the recipient.

In another embodiment of the system, the classification of a phishing message is performed based on a logistic regression algorithm with regularization.

For example, the message text from the training sample is divided into N-grams of a predetermined length. These N-grams are used as features to train a phishing email classification model based on a logistic regression algorithm with L1 regularization. The use of L1-regularization allows you to determine the weight coefficient of each N-gram, which characterizes the degree of influence of the specified N-gram on the classification result. N-grams whose weight is greater than a predetermined value (eg, greater than 0) are used as the message category.

In one embodiment, the system pre-collects attributes of electronic messages belonging to a pre-known class of messages (eg, phishing). Based on the collected data, Model No. 2230 is trained in such a way that electronic messages similar in their attributes can be classified by the mentioned model with an accuracy higher than the specified one.

The classification algorithm is at least one of the following algorithms (or a combination thereof):

• Bayesian classifiers (English naive bayesian classifier);

• logistic regression;

• MRF classifier (eng. MRF classifier);

• support vector machine (SVM, support vector machine);

• nearest neighbor methods (eng. k-nearest neighbor);

• decision tree.

In one embodiment, the system further comprises a security tool 240 for information security.

Ensuring information security includes at least:

• Blocking a phishing email;

• informing the recipient about the phishing nature of the electronic message;

• Placing the ID of the phishing email in the database of malicious messages.

For example, the security tool 240 is a security application module of Kaspersky Lab JSC (eg, Kaspersky Internet Security).

Fig. 3 illustrates a method for detecting a phishing email.

The block diagram of the method for transferring data to the storage system includes step 310, which determines the email as suspicious, step 320, which places the electronic message determined as suspicious, in a temporary quarantine, step 330, which determines the phishing email, step 340, which provide information security.

At 310, Model #1150 determines the email is suspicious.

At 320, the filtering engine 220 places the e-mail identified as suspicious into a temporary quarantine.

At step 330, using Model #2230, the suspicious email message is determined to be phishing.

At step 340, information security is provided by the security tool 240.

Fig. 4 shows an example of a general purpose computer system, a personal computer or a server 20, comprising a central processing unit 21, a system memory 22, and a system bus 23 that contains various system components, including memory associated with the central processing unit 21. The system bus 23 is implemented as any bus structure known from the prior art, which in turn contains a bus memory or bus memory controller, a peripheral bus, and a local bus that is capable of interfacing with any other bus architecture. The system memory contains read-only memory (ROM) 24, random access memory (RAM) 25. The main input/output system (BIOS) 26 contains the basic procedures that ensure the transfer of information between the elements of a personal computer 20, for example, at the time of loading the operating systems using ROM 24.

The personal computer 20 in turn comprises a hard disk 27 for reading and writing data, a magnetic disk drive 28 for reading and writing to removable magnetic disks 29 and an optical drive 30 for reading and writing to removable optical disks 31, such as CD-ROM, DVD -ROM and other optical storage media. The hard disk 27, the magnetic disk drive 28, the optical drive 30 are connected to the system bus 23 via the hard disk interface 32, the magnetic disk interface 33, and the optical drive interface 34, respectively. Drives and related computer storage media are non-volatile means of storing computer instructions, data structures, program modules, and other data of the personal computer 20.

The present description discloses an implementation of a system that uses a hard disk 27, a removable magnetic disk 29, and a removable optical disk 31, but it should be understood that other types of computer storage media 56 that are capable of storing data in a computer-readable form (solid-state drives, flash memory cards, digital disks, random access memory (RAM), etc.), which are connected to the system bus 23 through the controller 55.

The computer 20 has a file system 36 where the recorded operating system 35 is stored, as well as additional software applications 37, other program modules 38 and program data 39. The user has the ability to enter commands and information into the personal computer 20 through input devices (keyboard 40, manipulator " mouse" 42). Other input devices (not shown) may be used: microphone, joystick, game console, scanner, etc. Such input devices are typically connected to computer system 20 via serial port 46, which in turn is connected to the system bus, but may be connected in other ways, such as through a parallel port, game port, or universal serial bus (USB). A monitor 47 or other type of display device is also connected to the system bus 23 via an interface such as a video adapter 48. In addition to the monitor 47, the personal computer may be equipped with other peripheral output devices (not shown), such as speakers, a printer, etc. .

The personal computer 20 is capable of operating in a networked environment, using a network connection to another or more remote computers 49. The remote computer (or computers) 49 are the same personal computers or servers that have most or all of the elements mentioned earlier in the description of the being personal computer 20 shown in FIG. 4. Other devices may also be present in the computer network, such as routers, network stations, peering devices, or other network nodes.

The network connections may form a local area network (LAN) 50 and a wide area network (WAN). Such networks are used in corporate computer networks, internal networks of companies and, as a rule, have access to the Internet. In LAN or WAN networks, the personal computer 20 is connected to the local area network 50 via a network adapter or network interface 51. When using networks, the personal computer 20 may use a modem 54 or other means to communicate with a wide area network such as the Internet. Modem 54, which is an internal or external device, is connected to system bus 23 via serial port 46. It should be clarified that network connections are only exemplary and are not required to represent the exact network configuration, i. in fact, there are other ways to establish a connection by technical means of communication from one computer to another.

In conclusion, it should be noted that the information given in the description are examples that do not limit the scope of the present invention defined by the formula.

Claims (26)

1. A method for determining a phishing email message based on the use of at least two machine learning models, wherein the first model determines the email message as suspicious based on the attributes of the first type of intercepted email message, which include at least: the value of the message header Message_ID; X-mailer e-mail header value; a sequence of email header values, and using the second model, determine the suspicious email message as phishing based on attributes of the second type, which include at least: link score reputation, which measures the likelihood that an email contains a phishing link; the category of the email message; the presence flag of the sender's domain in a pre-formed database of unwanted senders; the presence flag of the sender's domain in a pre-formed database of known senders; the degree of similarity of the sender's domain with domains in a pre-formed database of known senders; flag for the presence of HTML code in the body of the electronic message; flag for the presence of scripted inserts in the body of the e-mail. 2. The method according to claim. 1, in which the electronic message, determined as suspicious, is additionally placed in a temporary quarantine. 3. The method according to claim 1, wherein the reputation of the set of links is calculated using a recurrent neural network. 4. The method according to claim 1, according to which N-grams of the text of the electronic message are used as the category of the electronic message, determined by selecting the most important features that most strongly affect the result of the binary classification of the phishing electronic message. 5. The method of claim 4, wherein the classification of the phishing email message is performed based on a logistic regression algorithm with regularization. 6. The method according to claim 1, in which at least the following acts as a machine learning algorithm of the second model: Bayesian classifiers; logistic regression; modified random forest learning algorithm; support vector method; nearest neighbor methods; decision tree. 7. The method according to claim 1, according to which information security is additionally provided by at least: blocking phishing email; informing the recipient about the phishing nature of the electronic message; placing the identifier of the phishing email in the database of malicious messages.

Images