CN110784462A - Three-layer phishing website detection system based on hybrid method - Google Patents

Abstract

The invention discloses a three-layer phishing website detection system based on a hybrid method, which comprises three layers: the system comprises a first black and white list layer, a form filtering layer, a second favicon detection layer and a third machine learning detection layer; known phishing websites can be found in time by the first black and white list and the form filtering layer, and the detection cost is reduced. The second favicon detection layer can identify the real identity of the website through favicon so as to detect the phishing website, the speed is high, and too many resources are not required to be consumed. The third machine learning detection layer can accurately identify the websites which are judged to be suspicious by the second layer, and a more accurate judgment result is obtained. The detection of three levels not only ensures the accuracy of the identification result, but also can reduce the detection time as much as possible.

Description

Three-layer phishing website detection system based on hybrid method

Technical Field

The invention relates to a website detection system, in particular to a three-layer phishing website detection system based on a hybrid method.

Background

Phishing is a fraudulent act for spoofing users over the internet, and attackers who launch phishing attacks are often referred to as phishers. The definition of phishing by the international Anti-phishing working Group (APWG) is that phishing is a network attack mode, and social engineering and technical means are utilized to steal personal identity data and financial account certificates of consumers. Phishing attacks that employ social engineering means typically send fraudulent emails, short messages, etc. to users, enticing users to reveal credential information (e.g., username, password) or download malicious software. The attack of the technical means is to directly plant malicious software (such as man-in-the-browser attack) on the PC, and to directly steal the credential information by adopting some technical means, such as intercepting the user name and password of the user by using the system, misleading the user to access a forged website, and the like.

Since Phishing seriously affects the interests of netizens and the reputation of the internet, the international Anti-Phishing Working Group (APWG, Anti-Phishing Working Group) shall approve the requirements of various non-profit organizations and industries in 2003, establishes a database based on the URL of a Phishing website and distributes the data regularly so as to make various industries refer to. According to APWG trend reports; phishing attacks have developed rapidly in recent years. In the report of the phishing activity trend in the quarter of 2018Q1, the total number of phishing detected in the first quarter of 2018 is 263,538. This is a 46% increase over 180,577 observed in 2017Q4, which also far exceeds 190,942 in the third quarter of 2017.

The increasing rampant phishing causes the threats of economic loss, identity fraud and the like to internet users. Therefore, effectively detecting phishing and making the processing is of great significance to network security.

Phishing detection technology identifies phishing attacks by utilizing certain characteristics of the phishing attacks, and therefore attack and prevention of the phishing attacks are achieved. With the continuous expansion of phishing, the research on the related phishing detection technology is also deepened, from the early blacklist technology to the prediction realized by using heuristic rules and machine learning, in recent years, with the development of deep learning theory, the neural network detection technology based on image recognition and rules is also continuously applied to the detection of phishing.

The conventional patent CN106357682A, "a phishing website detection method", proposes a method for extracting characters from favicon to compare with black and white lists. The process flow is shown in figure 1, and the patent has the following disadvantages: the method needs to maintain a database and update frequently to ensure timeliness. The detection fails when there is no text in the logo.

The prior patent CN104166725A "a phishing website detection method" proposes a phishing website detection scheme. In the scheme, a feature vector based on visual content corresponding to a webpage to be detected is established; comparing the feature vector with the feature vectors in a preset feature vector set; and judging whether the webpage to be detected is a phishing website or not according to the comparison result. The detection process is shown in fig. 2: the shortcoming of this patent: 1. the webpage to be detected is blocked, and characteristics such as blocking positions and the like are selected, and the characteristics will fail after the phishers finely adjust the webpage structure. 2. Selecting features such as a DOM tree will be disabled when the phisher uses the picture instead of the text.

The technical problem to be solved by the invention is how to detect the phishing website in a large number of webpages so as to ensure the safety of website information.

Disclosure of Invention

The invention aims to provide a three-layer phishing website detection system based on a mixing method, so as to solve the problems in the background technology.

In order to achieve the purpose, the invention provides the following technical scheme: the three-layer phishing website detection system based on the hybrid method comprises three layers: the system comprises a first black and white list layer, a form filtering layer, a second favicon detection layer and a third machine learning detection layer;

1. the first black and white list filtering layer: constructing a black-and-white list directly through the existing Google API phishing website black list and Alexa website TOP 250;

a login form filter which classifies websites which do not submit forms for login into ordinary websites, and since the purpose is to detect phishing websites, pages which do not submit forms obviously do not have phishing attributes;

through the two filters, if the website is not filtered, the following process is carried out, and the filtered website directly returns a result, so that the response of most common websites can be improved;

when the website to be detected is matched in the black list or the white list, returning a detection result, when the website to be detected is not matched in the black list or the white list, outputting the website to be detected as a legal website if the website to be detected is filtered by the form filter, otherwise, entering the next-layer detection;

2. the second layer is a favicon detection layer, the second layer acquires the identity of the webpage by using favicon, and compared with other visual features of the webpage, the favicon can identify the identity of the webpage better; and the method adopts Google Search to Search favicon, thereby avoiding the consumption of a large amount of computing and storage resources caused by self-maintenance of a database, and the flow is as follows:

2-1, favicon extraction process: obtaining favicon corresponding to the webpage through the corresponding website;

2-2, identity authentication process: the process is completed by utilizing Google image search and a Google picture library, the filtered favicon is subjected to Google search, related URLs are analyzed in returned matching contents, two webpage matching results and one picture matching result are returned in the part, and the webpage matching results only need to be retrieved;

then, in a detection stage, extracting data of four features from a returned result, counting the occurrence times of a secondary domain name of a detected website in the four features, performing linear weighted normalization on the secondary domain name by using a trained GMM (Gaussian mixture model) to obtain a normalized matching score S, classifying [0, S1 ] into a phishing class according to a dual-threshold strategy, classifying (S2, 1) into a legal webpage class, and meanwhile, judging the webpage in an interval of [ S1, S2] into a suspicious class;

and returning results of the second layer, directly returning detection results to websites judged to be legal or phishing, and putting websites classified into suspicious categories into the next layer for detection.

3. A third machine learning detection layer, wherein the third layer classifies websites which do not obtain results in the second layer by using a machine learning method, firstly extracts the characteristics of the webpages to be detected, and then classifies the webpages in the trained Self-Structuring NN;

3-1, selecting the characteristics of the third layer;

selecting characteristics of a UCI data set, wherein the characteristics have strong representativeness and basically comprise most characteristic consideration dimensions in the existing research;

and returning the result of the third layer, namely returning the classification result of the Self-Structuring NN, namely phishing or legal.

Compared with the prior art, the invention has the beneficial effects that: (1) known phishing websites can be found in time by the first black and white list and the form filtering layer, and the detection cost is reduced.

(2) The second favicon detection layer can identify the real identity of the website through favicon so as to detect the phishing website, the speed is high, and too many resources are not required to be consumed.

(3) The third machine learning detection layer can accurately identify the websites which are judged to be suspicious by the second layer, and a more accurate judgment result is obtained.

(4) The detection of three levels not only ensures the accuracy of the identification result, but also can reduce the detection time as much as possible.

Drawings

FIG. 1 is a flowchart of a prior art I operation;

FIG. 2 is a flowchart illustrating a second prior art in the background art;

fig. 3 is a schematic diagram of the system workflow structure of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 3, the three-layer phishing website detection system based on the hybrid method is composed of three layers: the system comprises a first black and white list layer, a form filtering layer, a second favicon detection layer and a third machine learning detection layer;

1. the first black and white list filtering layer: constructing a black-and-white list directly through the existing Google API phishing website black list and Alexa website TOP 250;

a login form filter which classifies websites which do not submit forms for login into ordinary websites, and since the purpose is to detect phishing websites, pages which do not submit forms obviously do not have phishing attributes;

through the two filters, if the website is not filtered, the following process is carried out, and the filtered website directly returns a result, so that the response of most common websites can be improved;

and when the website to be detected is matched in the black list or the white list, returning a detection result, and when the website to be detected is not matched in the black list or the white list, outputting the website to be detected as a legal website if the website to be detected is filtered by the form filter, otherwise, entering the next detection layer.

2. The second layer is a favicon detection layer, the second layer acquires the identity of the webpage by using favicon, and compared with other visual features of the webpage, the favicon can identify the identity of the webpage better; and the method adopts Google Search to Search favicon, thereby avoiding the consumption of a large amount of computing and storage resources caused by self-maintenance of a database, and the flow is as follows:

2-1, favicon extraction process:

the simplified processing procedure is adopted here, and favicon corresponding to the web page can be obtained through the following addresses:

TABLE 1 manner of obtaining favicon

Method	HTML Codes
		A	www.domain.com/favicon.ico
B	<link rel＝“shortcut icon”href＝“/favicon.ico”/>
		C	<link rel＝“icon”href＝“/favicon.ico”/>
D	<link rel＝“apple-touch-icon”href＝“images/favicon.ico”/>

2-2, identity authentication process:

the process is completed by utilizing Google image search and a Google picture library, the filtered favicon is subjected to Google search, related URLs are analyzed in returned matching contents, two webpage matching results and one picture matching result are returned in the part, and the webpage matching results only need to be retrieved;

then, in a detection stage, extracting data of four features from a returned result, counting the occurrence times of a secondary domain name of a detected website in the four features, performing linear weighted normalization on the secondary domain name by using a trained GMM (Gaussian mixture model) to obtain a normalized matching score S, classifying [0, S1 ] into a phishing class according to a dual-threshold strategy, classifying (S2, 1) into a legal webpage class, and meanwhile, judging the webpage in an interval of [ S1, S2] into a suspicious class;

and returning results of the second layer, directly returning detection results to websites judged to be legal or phishing, and putting websites classified into suspicious categories into the next layer for detection.

3. A third machine learning detection layer, wherein the third layer classifies websites which do not obtain results in the second layer by using a machine learning method, firstly extracts 22 characteristics of the to-be-detected webpages, and then puts the to-be-detected webpages into a trained Self-Structuring NN for classification;

3-1, selecting the characteristics of the third layer;

22 characteristics of the UCI data set are selected, and the 22 characteristics have strong representativeness and basically comprise most characteristic consideration dimensions in the existing research.

TABLE 2 feature selection

And returning the result of the third layer, namely returning the classification result of the Self-Structuring NN, namely phishing or legal.

4. Results of the experiment

4.1, testing time consumption, wherein in a training stage, a second favicon detection layer uses statistical data of 132 Alexa legal websites and 91 phistank phishing websites and puts the statistical data into a GMM for training; the third machine learning detection layer uses 30 financial phishing websites, 7044 phishtank phishing websites and 4442 Alexa legal websites to put into Self-structuringNN for training.

In the testing stage, the websites which cannot be determined by the upper layer enter the lower layer for detection by using the URLs of 500 Alexa legal websites and 500 phistank phishing websites. According to the experiment, 1000 websites were detected in the first layer, 796 websites were detected in the second layer, and 239 websites were detected in the third layer.

The time consumption of the three layers is mainly influenced by the network speed, the first layer needs to request HTML of a page when detecting whether the page contains a form, the second layer needs to acquire favicon of a webpage and uses a search API of Google, the third layer needs to acquire HTML of the webpage, the state of each port needs to be tested when checking whether an illegal port is used, and a whois API needs to be accessed to acquire webpage registration information and the like. The local processing is not very time consuming.

TABLE 3 time consuming statistics of layers

4.2, Performance testing

Model performance was evaluated using the following criteria, False Negative Rate (FNR): the proportion of the erroneously predicted positive samples to the total positive samples, i.e., the false alarm rate.

False Positive Rate (FPR): the proportion of the negative samples which are mispredicted to the total negative samples is the rate of missing report.

Recall (R): the proportion of positive samples that are correctly predicted to the total positive samples, i.e., the recall ratio.

Accuracy (acc): the ratio of the samples predicted to be correct to the total samples, i.e. the accuracy.

The calculation formula of each index is as follows:

the test is carried out by adopting 500 Alexa legal websites and 500 phistank phishing websites, and the test results are as follows:

TABLE 4 results of the experiment

FNR	FPR	R	ACC
				1.60％	3.40％	98.40％	97.50％

The detection of three levels not only ensures the accuracy of the identification result, but also can reduce the detection time as much as possible.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (1)

1. Three-layer phishing website detection system based on mixing method is characterized in that: the detection system consists of three layers: the system comprises a first black and white list layer, a form filtering layer, a second favicon detection layer and a third machine learning detection layer;

(1) and a first black and white list filter layer: constructing a black-and-white list directly through the existing Google API phishing website black list and Alexa website TOP 250;

a login form filter which classifies websites which do not submit forms for login into ordinary websites, and since the purpose is to detect phishing websites, pages which do not submit forms obviously do not have phishing attributes;

through the two filters, if the website is not filtered, the following process is carried out, and the filtered website directly returns a result, so that the response of most common websites can be improved;

when the website to be detected is matched in the black list or the white list, returning a detection result, when the website to be detected is not matched in the black list or the white list, outputting the website to be detected as a legal website if the website to be detected is filtered by the form filter, otherwise, entering the next-layer detection;

(2) the second layer uses favicon to obtain the identity of the webpage, and compared with other visual characteristics of the webpage, the favicon can identify the identity of the webpage better; and the method adopts Google Search to Search favicon, thereby avoiding the consumption of a large amount of computing and storage resources caused by self-maintenance of a database, and the flow is as follows:

(2-1), favicon extraction process: obtaining favicon corresponding to the webpage through the corresponding website;

(2-2) identity authentication process: the process is completed by utilizing Google image search and a Google picture library, the filtered favicon is subjected to Google search, related URLs are analyzed in returned matching contents, two webpage matching results and one picture matching result are returned in the part, and the webpage matching results only need to be retrieved;

then, in a detection stage, extracting data of four features from a returned result, counting the occurrence times of a secondary domain name of a detected website in the four features, performing linear weighted normalization on the secondary domain name by using a trained GMM (Gaussian mixture model) to obtain a normalized matching score S, classifying [0, S1 ] into a phishing class according to a dual-threshold strategy, classifying (S2, 1) into a legal webpage class, and meanwhile, judging the webpage in an interval of [ S1, S2] into a suspicious class;

the returned result of the second layer directly returns the detection result to the website judged to be legal or phishing, and the website classified into suspicious categories is put into the next layer for detection;

(3) the third layer classifies websites which do not obtain results in the second layer by using a machine learning method, firstly extracts the characteristics of the webpages to be detected, and then puts the webpages into a trained Self-Structuring NN for classification;

(3-1) selecting characteristics of a third layer;

selecting characteristics of a UCI data set, wherein the characteristics have strong representativeness and basically comprise most characteristic consideration dimensions in the existing research;

and returning the result of the third layer, namely returning the classification result of the Self-Structuring NN, namely phishing or legal.