CN108712403B - Illegal Domain Name Mining Method Based on Domain Name Structure Similarity - Google Patents

Abstract

The invention provides an illegal domain name mining method based on the similarity of domain name structure, which solves the technical problem that the existing method cannot actively mine a large number of illegal domain names; the method comprises the following steps: Step 1, read illegal domain names from the domain name blacklist; step 2. Determine whether there is a class that has successfully aggregated, if not, go to step 10; otherwise, continue to the next step; step 3, determine whether the current domain name can be classified into the i-th aggregation class, if not, go to Step 10; otherwise, proceed to the next step; the judgment is based on whether the current domain name is similar to the central domain name, and the central domain name refers to a representative domain name in the aggregation class; Step 4, merge the current domain name into the i-th aggregation class, and extract The generation pattern generated during the matching process between the current domain name and the central domain name of this type is obtained, and the next step is continued; the generation pattern is the wildcard string extracted from each domain name in the aggregation class and the central domain name. The present invention is widely used in the field of information technology.

Description

Illegal Domain Name Mining Method Based on Domain Name Structure Similarity

technical field

The invention relates to an illegal domain name mining method, in particular to an illegal domain name mining method based on the similarity of domain name structures.

Background technique

With the rapid development of the Internet, the domain name, one of the products of the Internet, has gradually been recognized and popularized by people. While the domain name brings us the convenience of remembering websites and modifying IP, it also hides some unavoidable Security risks.

This year, more and more illegal organizations have carried some illegal activities through domain names, such as botnets, phishing websites, pornography, gambling and drug websites, etc. The vast number of netizens have brought incalculable damage to their property and spirit. Therefore, it is urgent to A method to mine illegal domain names efficiently and quickly is proposed.

At present, most browsers use pre-prepared blacklists to prevent netizens from accessing illegal websites by regularly updating and maintaining the blacklists.

SUMMARY OF THE INVENTION

Aiming at the technical problem that the existing method cannot actively mine a large number of illegal domain names, the invention provides an illegal domain name mining method based on the similarity of the domain name structure, which can actively mine a large number of illegal domain names.

For this reason, the technical scheme of the present invention is, comprising the following steps:

Step 1, read illegal domain names from the domain name blacklist;

Step 2, judge whether there is a successfully aggregated class, if not, go to step 10; otherwise, continue to the next step;

Step 3, determine whether the current domain name can be classified into the ith clustering class, if not, go to step 10; otherwise, continue to the next step; the ith clustering class is the ith clustering class that aggregates similar domain names according to similar rules. kind;

The judgment is based on whether the current domain name is similar to the central domain name, and the central domain name refers to the representative domain name in the aggregated category;

The specific method to determine whether the current domain name is similar to the central domain name is as follows:

(1) If the two domain names are different only in the top-level domain and other parts are the same, the two domain names are similar;

(2) If the top-level domains of two domain names are the same, when the length of the second-level domain is the same, the same position of the second-level domain does not differ by more than 2 characters; or the same position in the same position is different, the two domain names are similar; When the lengths of the second-level domains of the two domain names differ by 1 and the long domain name can be changed to a short domain name by removing one character, the two domain names are similar;

(3) If neither of (1) and (2) is determined to be similar, the two domain names are not similar;

Step 4, merge the current domain name into the i-th aggregation class, and extract the generation pattern generated during the matching process between the current domain name and the central domain name of this class, and continue to the next step;

The generation mode is a wildcard string extracted from each domain name and the central domain name in the aggregation class;

Step 5, generate possible similar domain names similar to the central domain name by enumerating in the generation mode, and filter out the illegal domain names that have been stored in the similar domain names, and continue to the next step;

Step 6: Determine whether the similar domain names screened in Step 5 exist one by one by obtaining the domain name WHOIS information, if not, discard; otherwise, keep it and continue to the next step;

Step 7: Detect whether the reserved domain name is illegal, if it is detected illegal, add it to the illegal domain name set; otherwise, add it to the unknown domain name set; continue to the next step;

Step 8, determine whether the similar domain names screened in Step 5 have been detected, if the detection is completed, continue to the next step; otherwise, go to Step 6;

Step 9, determine whether the illegal domain names of step 1 have been clustered, if the clustering has been completed, the algorithm ends; otherwise, go to step 1;

Step 10, create a new class, set the current domain name as the central domain name of the class, and go to step 9.

Preferably, in step 4, the generation pattern uses wildcards to replace the difference between two illegal domain names, and uses indicators to indicate the enumeration operation specifying wildcards.

Preferably, in step 7, the detection is performed through an authoritative third-party detection interface.

Beneficial effects of the invention: the method is based on the analysis of a large number of existing illegal domain names, so as to excavate a large number of illegal domain names that are not included. First, cluster the set of illegal domain names in the prepared blacklist, and group the illegal domain names that are similar in structure into one class, thereby forming multiple clusters; then, extract one or more generation patterns from each class , to obtain the set of generation patterns; thirdly, enumerate through the generation patterns to generate suspected illegal similar domain names; finally, use the third-party authoritative detection interface to detect the suspected illegal generated domain name set, and filter out the illegal similar domain names. This method starts from the similarity of illegal domain name structure, actively mines a large number of illegal domain names that do not exist in the database, and the illegal domain names mined based on the similarity of domain name structure have a strong correlation, which is conducive to the association of illegal domain names Analysis, group analysis.

Description of drawings

Fig. 1 is the overall function flow chart of the embodiment of the present invention;

FIG. 2 is a flowchart of a method according to an embodiment of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the embodiments.

There are structural similarities between illegal domain names. Often, a batch of illegal domain names can be generated by slightly modifying the structure of a single illegal domain name. registered with the same illegal organization. For example, through the illegal domain name 00080d.com, more similar illegal domain names 00080e.com, 00080f.com, 00080w.com, etc. can be mined.

As shown in Figures 1 and 2, this embodiment provides a method for mining illegal domain names based on the similarity of domain name construction. The main steps include similarity clustering, generating pattern extraction, generating similar domain names, and detecting the existence and illegality of similar domain names. Chunk steps. In this embodiment, illegal domain names such as gambling, pornography, and fraud are used as a blacklist for clustering, and a self-defined similarity rule is adopted to group similar domain names into one category, and then extract the generation pattern of each category to generate similar domain name, and finally detected illegal and actual similar domain names. Specific steps are as follows:

Step 1, read illegal domain names from the domain name blacklist;

Step 2, judge whether there is a successfully aggregated class, if not, go to step 10; otherwise, continue to the next step;

Step 3, determine whether the current domain name can be classified into the i-th aggregation class, if not, go to step 10; otherwise, continue to the next step;

Among them, the judgment method is to judge whether the domain name is similar to the central domain name, and the central domain name refers to the representative domain name in the aggregation class;

The i-th aggregation class is the i-th class that aggregates similar domain names according to the self-defined similarity rules. The similarity rules are as follows:

(1) If the two domain names are different only in the top-level domain, and other parts are the same, such as 08vip.vip and 08vip.tv, then the two domain names are similar;

(2) If the top-level domains of two domain names are the same, when the length of the second-level domain is the same, the same position of the second-level domain should not differ by more than 2 characters, such as 00037b.com and 00037c.com, 099sun.com and 099sky.com, 1188030 .com and 1388033.com; or multiple consecutive identical characters in the same position with different characters, such as 4148ww.com and 4148nn.com, 4040uuu.com and 4040jjj.com, then the two domain names are similar; And when a long domain name can be changed into a short domain name by removing one character, such as 0000524.com and 00001524.com, the two domain names are similar;

(3) If neither of (1) and (2) are determined to be similar, the two domain names are not similar.

Step 4, merge the current domain name into the i-th aggregation class, and extract the generation pattern generated during the matching process between the current domain name and the central domain name of this class, and continue to the next step;

Among them, the generation mode is the wildcard string extracted from each domain name and the central domain name in the aggregation class; the extraction method of the generation mode is to use wildcards to replace the difference between two illegal domain names, and use indicators to indicate the specified wildcards The enumeration operation is as follows:

(1) If only the top-level domain is different between the two domain names, such as 08vip.vip and 08vip.tv, the generation mode 08vip.% can be extracted;

(2) If two domain names are similar and only differ by one character, such as 0000524.com and 00001524.com, the pattern 00001524-.com or 0000524+.com can be extracted;

(3) If two domain names are similar and no more than 2 characters in the same position are different, when the different characters in the same position are all numbers, such as 1188030.com and 1388033.com, the pattern 1#8803#.com can be extracted; when When different characters in the same position are all letters, such as 00037b.com and 00037c.com, 099sun.com and 099sky.com, patterns 00037#.com and 099s**.com can be extracted; when different characters in the same position are When there are letters on one side of the numbers, such as 004zyz.com and 0044y8.com, the pattern 004$y$.com can be extracted;

(4) If two domain names are similar and only the same characters are different in the same position, such as 4148ww.com and 4148nn.com, 4040uuu.com and 4040jjj.com, 1186655.com and 1186699.com, the matching pattern can be extracted respectively 4148**&.com, 4040***&.com, 11866##&.com.

Step 5: Use the wildcards and indicators in the generation mode to enumerate to generate possible similar domain names that are similar to the central domain name, and filter out the illegal domain names that have been stored in the similar domain names, and continue to the next step;

The specific descriptions of wildcards and indicators are as follows:

(1) % is a top-level domain wildcard, when enumerating, replace % with the top-level domain extracted from the blacklist;

(2) -, + are indicators, indicating that a character in the second-level domain needs to be deleted or a character added during enumeration;

(3) * is an alphabetic wildcard, # is a numeric wildcard, $ is an alphanumeric wildcard, * is replaced by letters a～z when enumerating, # is replaced by numbers 0～9 when enumerating, and 0～9 is replaced by $ when enumerating , a～z;

(4) & is a continuous indicator, indicating that all wildcards replace the same character during enumeration;

Step 6: Determine whether the similar domain names screened in Step 5 exist one by one by obtaining the domain name WHOIS information, if not, discard; otherwise, keep it, and continue to the next step;

Step 7: Detect whether the reserved domain name is illegal through an authoritative third-party detection interface. If it is detected to be illegal, add it to the illegal domain name set; otherwise, add it to the unknown domain name set, and continue to the next step;

Regularly check the domain names in the unknown domain name set to determine whether it is an illegal domain name. If it is detected as illegal, add it to the illegal domain name set; otherwise, keep it in the unknown domain name set;

Step 8, determine whether the similar domain names screened in Step 5 have been detected, if the detection is completed, continue to the next step; otherwise, go to Step 6;

Step 9, determine whether the illegal domain names of step 1 have been clustered, if the clustering has been completed, the algorithm ends; otherwise, go to step 1;

Step 10, create a new class, set the current domain name as the central domain name of the class, and go to step 9.

This method is based on the analysis of a large number of existing illegal domain names, so as to excavate a large number of illegal domain names that are not included. First, cluster the set of illegal domain names in the prepared blacklist, and group the illegal domain names that are similar in structure into one class, thereby forming multiple clusters; then, extract one or more generation patterns from each class , to obtain the set of generation patterns; thirdly, enumerate through the generation patterns to generate suspected illegal similar domain names; finally, use the third-party authoritative detection interface to detect the suspected illegal generated domain name set, and filter out the illegal similar domain names. This method starts from the similarity of illegal domain name structure, actively mines a large number of illegal domain names that do not exist in the database, and the illegal domain names mined based on the similarity of domain name structure have a strong correlation, which is conducive to the association of illegal domain names Analysis, group analysis, etc.

However, the above are only specific embodiments of the present invention, which should not limit the scope of the present invention. Therefore, the replacement of the equivalent components, or the equivalent changes and modifications made according to the scope of the patent protection of the present invention, should still belong to the scope of the present invention. The scope of the present invention is covered by the claims.

Claims (3)

1. an illegal domain name mining method based on domain name structure similarity, is characterized in that, comprises the following steps: Step 1, read illegal domain names from the domain name blacklist; Step 2, judge whether there is a successfully aggregated class, if not, go to step 10; otherwise, continue to the next step; Step 3, determine whether the current domain name can be classified into the i-th aggregation class, if not, go to step 10; otherwise, continue to the next step; the i-th aggregation class is to aggregate similar domain names according to similar rules class i; The judgment is based on whether the current domain name is similar to the central domain name, and the central domain name refers to a representative domain name in the aggregation class; The specific method for judging whether the current domain name is similar to the central domain name is: (1) If the two domain names are different only in the top-level domain and other parts are the same, the two domain names are similar; (2) If the top-level domains of two domain names are the same, when the length of the second-level domain is the same, the same position of the second-level domain does not differ by more than 2 characters; or the same position in the same position is different, the two domain names are similar; When the lengths of the second-level domains of the two domain names differ by 1 and the long domain name can be changed to a short domain name by removing one character, the two domain names are similar; (3) If neither of (1) and (2) is determined to be similar, the two domain names are not similar; Step 4, merge the current domain name into the i-th aggregation class, and extract the generation pattern generated during the matching process between the current domain name and the central domain name of this class, and continue to the next step; The generation mode is a wildcard string extracted from each domain name and the central domain name in the aggregation class; Step 5, generate possible similar domain names similar to the central domain name by enumerating in the generation mode, and filter out the illegal domain names that have been stored in the similar domain names, and continue to the next step; Step 6: Determine whether the similar domain names screened in Step 5 exist one by one by obtaining the domain name WHOIS information, if not, discard; otherwise, keep it and continue to the next step; Step 7: Detect whether the reserved domain name is illegal, if it is detected illegal, add it to the illegal domain name set; otherwise, add it to the unknown domain name set; continue to the next step; Step 8, determine whether the similar domain names screened in Step 5 have been detected, if the detection is completed, continue to the next step; otherwise, go to Step 6; Step 9, determine whether the illegal domain names of step 1 have been clustered, if the clustering has been completed, the algorithm ends; otherwise, go to step 1; Step 10, create a new class, set the current domain name as the central domain name of the class, and go to step 9. 2. the illegal domain name mining method based on domain name structure similarity according to claim 1, is characterized in that, in described step 4, described generation pattern uses wildcard to replace the difference part between two illegal domain names, uses instruction character to indicate the enumeration operation of the specified wildcard. 3. The method for mining illegal domain names based on the similarity of domain name structures according to claim 1, wherein in the step 7, the detection is performed through an authoritative third-party detection interface.

Images