CN111553155B - Password word segmentation system and method based on semantic structure - Google Patents

Abstract

A password word segmentation system and method based on semantic structure comprises: the system comprises a preprocessing module, an NLP semantic extraction module and a non-NLP semantic annotation module, wherein: the preprocessing module receives a password to be segmented, extracts special semantic factors which cannot be identified in the subsequent steps in the password, pre-segments the rest parts according to character types, outputs letter parts to the NLP semantic extraction module, and outputs non-letter parts to the non-NLP semantic labeling module; the NLP semantic extraction module divides words from the letter part of the password by using an NLP tool to obtain various semantic factors; the non-NLP semantic annotation module carries out semantic annotation on the part of the password, which cannot be segmented by the NLP tool. According to the invention, the password is segmented according to the semantic information contained in the password according to the corpus, the semantic structure of the password is identified, and the password set by the Chinese user and the English user can be accurately segmented.

Description

Password segmentation system and method based on semantic structure

Technical Field

The present invention relates to a technology in the field of computer security, in particular to a password segmentation system and method based on semantic structure.

Background Art

Text passwords are still widely used in user authentication and online services in computer systems due to their good security and usability. Since most passwords are defined by the users themselves, they often choose strings with specific semantics or rules as passwords for easy memorization. Therefore, the study of password semantic structure is of great significance to improving the security of user passwords.

Unlike natural languages, passwords do not have a fixed grammatical structure. When setting passwords, users can arbitrarily combine various semantic factors according to the rules of the website. Therefore, the word segmentation method for natural languages is not applicable to password segmentation.

In the past, most studies on the semantic structure of passwords were conducted on English user passwords. Since there are certain differences between the passwords set by English users and Chinese users, the word segmentation methods proposed for English user passwords often perform poorly on Chinese leaked databases. In recent years, several researchers have begun to study Chinese user passwords. Studies have shown that adding additional semantic information to the word segmentation system is effective, but what information to add and how to add it is still a subjective judgment, and there is no systematic method.

Summary of the invention

In view of the above-mentioned deficiencies in the prior art, the present invention proposes a password segmentation system and method based on semantic structure, which segments passwords according to the semantic information contained therein based on a corpus, identifies the semantic structure of passwords, and can accurately segment passwords set by Chinese users and English users.

The present invention is achieved through the following technical solutions:

The invention relates to a password segmentation system based on semantic structure, comprising: a preprocessing module, a natural language processing (NLP) semantic extraction module and a non-natural language processing (non-NLP) semantic annotation module, wherein: the preprocessing module receives a password to be segmented, extracts special semantic factors in the password that cannot be identified in subsequent steps and performs pre-segmentation on the remaining parts according to character types, outputs the letter part to the NLP semantic extraction module, and outputs the non-letter part to the non-NLP semantic annotation module; the NLP semantic extraction module performs segmentation on the letter part of the password using an NLP tool to obtain multiple semantic factors; and the non-NLP semantic annotation module performs semantic annotation on the part of the password that cannot be segmented using the NLP tool.

The special semantic factors include: keyboard structure, website, and email address.

The parts that cannot be segmented using NLP tools include: numbers and special characters.

The preprocessing module includes: a keyboard structure extraction unit, an email extraction unit, a website extraction unit and a character segmentation unit, wherein: the keyboard structure extraction unit extracts the part of the password related to the keyboard key distribution pattern, that is, extracts the keyboard structure in the password, the email extraction unit extracts the email address contained in the password, the website extraction unit extracts the website contained in the password, and the character segmentation unit segments the password according to different character types.

The NLP semantic extraction module includes: a word segmentation unit, a part-of-speech tagging (POS) unit and a semantic classification unit, wherein: the word segmentation unit uses a natural language toolkit (NLTK) to segment the letter part input from the preprocessing module, and outputs the result to the POS unit; the POS unit uses the POS module of NLTK to mark each input factor, and outputs the semantic factor that needs to be further classified to the semantic classification unit; the semantic classification unit uses a string matching method to further classify the named entity factors, marking them as place names, months, male names, female names, and Chinese name abbreviations, matches unrecognized factors in a pinyin list, and marks the matched factors as pinyin. When the unmatched factors meet the rule of "consonant letters with a length of more than 3 digits", they are marked as abbreviations, otherwise they are still marked as unrecognized factors.

The semantic factors that need to be further classified include: named entities and unrecognized segments.

The non-NLP semantic annotation module includes: a digital annotation module and a special character annotation module, wherein: the digital annotation module marks the digital fragments containing specific semantics accordingly, and marks the digital fragments with unknown semantics according to their length; the special character annotation unit marks the special character fragments according to their length.

The specific semantics include: date, year, and mobile phone number.

Technical Effects

The present invention solves the problem of word segmentation of passwords in different languages and different leaked libraries;

Compared with the prior art, the present invention extracts passwords from semantic factors including multiple character types such as keyboard structure, email address, and website before formally segmenting the passwords, thereby avoiding semantic loss caused by segmentation by character type and improving the accuracy of word segmentation. The keyboard structure contained in the password can be effectively extracted, thereby improving the accuracy of word segmentation. The present invention adds multiple semantic factors such as place names, Chinese name abbreviations, pinyin, abbreviations, mobile phone numbers, keyboard structures, website addresses, and email addresses to the word segmentation system, thereby improving the accuracy of word segmentation and realizing the word segmentation of Chinese website passwords by the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the system structure of the present invention.

DETAILED DESCRIPTION

As shown in Figure 1, this embodiment involves a password segmentation system based on semantic structure, including: a preprocessing module, an NLP semantic extraction module and a non-NLP semantic classification module, wherein: the preprocessing module is connected to the NLP semantic extraction module and transmits the letter part obtained by pre-segmentation during the preprocessing process, and the preprocessing module is connected to the non-NLP semantic annotation module and transmits the number and special character part obtained by pre-segmentation during the preprocessing process.

在键盘上相邻，且其<shift>键状态相同，则判定该子串

为键盘结构([KB])，具体的标签由其长度决定([KB4]，[KB5]，……)；网址提取单元通过前缀“www.”和“http://”检测口令中是否存在网址，当检测到“www.”或“http://”且该子串后匹配到一个在常用域名后缀列表中的子串，且两个子串之间为由“.”分隔开的一个或多个字符串，则判定从前缀到域名后缀的该字符串为网址([Website])；在电子邮箱提取单元中，以“‘@’+域名”的格式作为电子邮箱的格式，而将“@”之前的用户名作为普通的字符串保留，在之后的步骤中进行分词。当在字符串中匹配到“‘@’+域名”格式的字符串，则判定其为电子邮箱([email])。The preprocessing module predefines the extraction of three special semantic factors (keyboard structure, website, and email address). In the keyboard structure extraction unit, for a substring in the password,

If they are adjacent on the keyboard and their <shift> key status is the same, then the substring is determined

The structure is a keyboard ([KB]), and the specific label is determined by its length ([KB4], [KB5], ...); the URL extraction unit detects whether there is a URL in the password through the prefix "www." and "http://". When "www." or "http://" is detected and the substring is matched with a substring in the common domain name suffix list, and the two substrings are separated by one or more strings separated by ".", the string from the prefix to the domain name suffix is determined to be a URL ([Website]); in the email extraction unit, the format of "'@'+domain name" is used as the format of the email, and the user name before "@" is retained as a common string and segmented in the subsequent steps. When a string in the format of "'@'+domain name" is matched in the string, it is determined to be an email ([email]).

The NLP semantic extraction module includes: a word segmentation unit, a part-of-speech tagging (POS) unit and a semantic classification unit for identifying named entities, wherein: the word segmentation unit for identifying named entities specifically refers to: using a two-word segmentation algorithm, first adding a named entity list containing four semantic factors ([location], [month], [male_name], [female_name]) to the NLTK tool for word segmentation, and if there are still unrecognized segments after the first round of operation, then adding a named entity list containing five semantic factors ([location], [month], [male_name], [female_name] , [cn_name_abbr]) is added to the NLTK tool for secondary word segmentation; the semantic classification unit matches the named entity list by string matching, and marks the [NP] segment as one of the named entities ([location], [month], [male_name], [female_name], [cn_name_abbr]); for the [NN] segment, firstly judge by string matching: when it is in the pinyin list, it is marked as [PY], otherwise, when the length is greater than 3 and all are consonants, it is judged to be an English abbreviation ([abbr]), and when it is neither of the above two cases, keep the [NN] label unchanged.

This embodiment is based on the password segmentation method based on semantic structure of the above system, and the specific steps include:

S1) The preprocessing module reads the password P to be segmented.

在键盘上相邻，且其<shift>键状态相同，则判定该子串

键盘结构([KB])，具体的标签由其长度决定([KB4]，[KB5]，……)。S2) In the keyboard structure extraction unit, for a substring in the password

If they are adjacent on the keyboard and their <shift> key status is the same, then the substring is determined

Keyboard structure ([KB]), the specific label is determined by its length ([KB4], [KB5], ...).

S3) The website extraction unit detects whether there is a website in the password through the prefix "www." and "http://". When "www." or "http://" is detected and the substring is matched with a substring in the common domain name suffix list, and the two substrings are separated by one or more strings separated by ".", the string from the prefix to the domain name suffix is determined to be a website ([Website]).

S4) The email extraction unit uses the format of "'@' + domain name" as the format of the email, and retains the user name before "@" as a common string, and performs word segmentation in the subsequent steps. When a string in the format of "'@' + domain name" is matched in the string, it is determined to be an email ([email]).

S5) outputting the unmarked part of the password to the character segmentation unit, performing pre-segmentation according to the different character types (numbers, letters, special characters), and marking them as numbers ([number]), letters ([word]), and special characters ([special]), respectively.

After passing through the preprocessing module, the password 1qaziloveyou123@ becomes (1qaz, KB4), (iloveyou, word), (123, number), (@, special).

S6) Output the fragments marked as [word] to NLTK. The corpora used in the word segmentation process are the Brown corpus and the Web Text corpus, and add several named entity lists to them, which represent 5 semantic factors: four English semantic factors (([location], [month], [male_name], [female_name])) and Chinese name abbreviations ([cn_name_abbr]). First, do not add Chinese name abbreviations, only add four English named entities for word segmentation. When the word segmentation result contains unrecognized fragments, add Chinese name abbreviations for the second word segmentation.

S7) After word segmentation, the word segmentation results are output to the POS unit for semantic annotation, and are annotated as the following semantic factors with specific semantics: pronoun ([NOUN]), noun ([NOUN]), determiner ([DET]), adjective ([ADJ]), verb ([VERB]), preposition ([ADP]), adverb ([ADV]), particle ([PRT]), conjunction ([CONJ]), English word representing number ([NUM]), affix ([X]). In the POS tagging process, a sequential backward tagger is used for annotation, first using Browntrigram tagger, then bigtram tagger, and finally onegram tagger. The segment that appears in the named entity list is annotated as [NN], and the unrecognized segment is annotated as [NN].

S8) After POS tagging, the [NN] segment is further semantically classified. The [NP] segment is annotated as one of the named entities ([location], [month], [male_name], [female_name], [cn_name_abbr]) by string matching with the named entity list.

S9) For the [NN] segment, firstly judge by string matching: when it is in the pinyin list, it is marked as [PY]; otherwise, when the length is greater than 3 and all are consonants, it is judged to be an English abbreviation ([abbr]); when it is neither of the above two cases, keep the [NN] label unchanged.

S10) outputs the fragment marked as [number] in S5) to the digital semantic classification unit. For a digital fragment with a length of 4 digits, when it is between 1900 and 2020, it is considered to be a year and marked as [year]; for a fragment with a length of 6, when it satisfies the date format of YYMMDD, it is determined to be a date and marked as [YYMMDD]; for a fragment with a length of 8, when it satisfies the date format of YYYYMMDD, it is determined to be a date and marked as [YYYYMMDD]; for a fragment with 11 digits, when it satisfies the format of a mobile phone number, it is determined to be a mobile phone number and marked as [mobilephone]. The remaining digital segments are marked as [num1], [num2], ... according to their lengths.

S11) Input the [special] segments in S5) into special character marking units and mark them as [spec1], [spec2], ... according to their lengths.

S12) The labels of all the fragments are combined in order to form the semantic structure of the password.

In this embodiment, 13 leaked libraries are selected, including 6 Chinese libraries (CSDN, Tianya, Youku, 17173, Aipai, Dudu Niu) and 7 English libraries (LinkedIn, Zoosk, Myspace, Rockyou, MyHeritage, Gmail, Webhost), to test the word segmentation effect of the method. The specific test results are shown in Table 1.

Table 1

This test uses the absence of NN factors in the word segmentation results as the criterion for successful word segmentation of a password. It can be seen that this embodiment can achieve a high word segmentation success rate on both the Chinese leak database and the English leak database, especially the test word segmentation success rate on the Chinese leak database has reached more than 90%, which is sufficient to illustrate the effectiveness of this embodiment.

Compared with the prior art, this embodiment selected four leaked libraries for testing, including two Chinese libraries (17173 and Aipai) and two English libraries (LinkedIn and Gmail). The word segmentation success rates of this embodiment on the four leaked libraries were 92.22%, 91.24%, 79.37%, and 84.19%, respectively, which were significantly higher than 65.17%, 60.88%, 62.26%, and 67.14% of the prior art.

The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims (8)

1. A password segmentation system based on semantic structures, comprising: a preprocessing module, a Natural Language Processing (NLP) semantic extraction module and a non-NLP semantic annotation module, wherein: the preprocessing module receives a password to be segmented, extracts special semantic factors which cannot be identified in the subsequent steps in the password, pre-segments the rest parts according to character types, outputs letter parts to the NLP semantic extraction module, and outputs non-letter parts to the non-NLP semantic labeling module; the NLP semantic extraction module divides words from the letter part of the password by using an NLP tool to obtain various semantic factors; the non-NLP semantic annotation module carries out semantic annotation on the part of the password, which cannot be segmented by the NLP tool;

the preprocessing module comprises: the electronic mail box extraction device comprises a keyboard structure extraction unit, an electronic mail box extraction unit, a website extraction unit and a character segmentation unit, wherein: the keyboard structure extracting unit extracts a part of the password related to the distribution rule of the keyboard keys, namely, extracts the keyboard structure in the password, the electronic mailbox extracting unit extracts the electronic mailbox address contained in the password, the website extracting unit extracts the website contained in the password, and the character segmentation unit carries out word segmentation on the password according to different character types.

2. The system of claim 1, wherein the NLP semantic extraction module comprises: the system comprises a word segmentation unit, a part-of-speech labeling POS unit and a semantic classification unit, wherein: the word segmentation unit is used for segmenting the letter part input from the preprocessing module by using a Natural Language Tool Kit (NLTK) and outputting a result to the POS unit; the POS unit marks the input factors by using a POS module of NLTK, and outputs semantic factors needing further classification to the semantic classification unit; the semantic classification unit further classifies the named entity factors by using a character string matching method, marks the named entity factors as abbreviation categories of place names, months, male names, female names and Chinese names, matches unidentified factors in a pinyin list, marks the matched factors as pinyin, marks the unidentified factors as abbreviations when the unidentified factors accord with the rule of consonant letters with the length exceeding 3 bits, and otherwise marks the unidentified factors.

3. The system of claim 1, wherein the non-NLP semantic annotation module comprises: the system comprises a digital marking module and a special character marking module, wherein: the digital labeling module carries out corresponding labeling on the digital fragments containing the specific semantics, and labels the digital fragments with unknown semantics according to the length of the digital fragments; the special character labeling unit labels the special character segments according to the lengths of the special character segments.

4. The system according to claim 2, wherein the recognition of named entities by the word segmentation unit is specifically: and adding a named entity list containing four semantic factors [ location ], [ mole_name ] and [ mole_name ] into the NLTK tool for word segmentation for the first time by adopting a twice word segmentation algorithm, and adding a named entity list containing five semantic factors [ location ], [ mole_name ] and [ cn_name_abbr ] into the NLTK tool for secondary word segmentation when unrecognized segments still exist after the first round of operation.

5. The system of claim 2, wherein the semantic classification unit marks the [ NP ] segment as one of a named entity [ location ], [ mol ], [ mol_name ], [ fe_name ], [ cn_name_abbr ] by performing a string match with a named entity list; for [ NN ] segmentation, firstly, judging through character string matching: when in the pinyin list, labeled [ PY ], otherwise, when the length is greater than 3 and the consonants are all consonants, the judgment is probably the English abbreviation [ abbr ], and when the two conditions are not the same, the [ NN ] label is kept unchanged.

6. A method of password segmentation based on semantic structures based on the system of any one of the preceding claims, comprising the steps of:

s1, a preprocessing module reads a password P to be segmented;

s2, in the keyboard structure extraction unit, for one substring in the password

When->

And

adjacent on the keyboard, and it<shift>The key states are the same, the substring is determined +.>

Is a keyboard structure [ KB ]]And its label is determined by its length; />

S3, detecting whether a Website exists in the password through prefixes of ' www ' and ' http:// ', and when ' www ' or ' http:// ' is detected and the sub-string is matched with a sub-string in a common domain name suffix list, one or more character strings separated by ' are arranged between the two sub-strings, judging that the character string from the prefix to the domain name suffix is the Website [ Website ];

s4, the email box extracting unit takes the format of the '@' + domain name as the format of the email box, and takes the user name before the '@' as a common character string; when the character string is matched with the character string in the format of the '@' + domain name, judging that the character string is an electronic mailbox [ email ];

s5, outputting unlabeled parts in the password to a character word segmentation unit, and pre-segmenting according to different character types, numbers, letters and special characters, wherein the unlabeled parts are labeled as numbers, letters and special characters;

s6, outputting the fragments marked as word into NLTK, wherein the corpus used in the word segmentation process is a Brown corpus and a Web Text corpus, and a plurality of named entity lists are added in the Brown corpus and the Web Text corpus;

s7, outputting the word segmentation result to a POS unit for semantic annotation, wherein the annotation is as follows: pronouns [ NOUN ], NOUNs [ NOUN ], qualifiers [ DET ], adjectives [ ADJ ], VERBs [ VERB ], prepositions [ ADP ], adverbs [ ADV ], small article words [ PRT ], conjunctions [ CONJ ], english words representing numbers [ NUM ] and suffixes [ X ];

s8, after POS labeling, further semantic classification is carried out on the [ NN ] segments; by matching the character string with the named entity list, labeling the [ NP ] segment as one of named entities [ location ], [ mole ], [ mole_name ], [ fe_name ], [ cn_name_abbr ];

s9, for [ NN ] segmentation, firstly judging through character string matching: when the Chinese phonetic alphabet is in the phonetic alphabet list, marking as [ PY ], otherwise, when the Chinese phonetic alphabet is longer than 3 and is consonant, judging as English abbreviation [ abbr ], and when the Chinese phonetic alphabet is not in the two cases, keeping the [ NN ] label unchanged;

s10, outputting the segment marked as the number in S5 to a digital semantic classification unit, wherein for the digital segment with the length of 4 bits, the segment is regarded as year and marked as the year between 1900 and 2020; for a fragment of length 6, when the date format of YYMMDD is satisfied, the determination is a date, labeled [ YYMMDD ]; for a fragment of length 8, when the date format of YYYYMMDD is satisfied, the determination is a date, labeled [ YYYYMMDD ]; for the 11-bit segment, when the format of the mobile phone number is met, judging that the mobile phone number is the mobile phone number, marking the mobile phone as [ mobile ], and marking the rest digital segments according to the length of the digital segments;

s11, inputting the special segment in the S5 into a special character labeling unit, and labeling according to the length of the special character labeling unit;

s12, the labels of all the fragments are combined together in sequence to form the semantic structure of the password.

7. The password segmentation method according to claim 6, wherein the named entity list in step S6 includes: four English semantic factors [ location ], [ month ], [ rule_name ], and Chinese name abbreviation [ cn_name_abbr ], wherein the Chinese name abbreviation is not added first, only four English named entities are added for word segmentation, and when the word segmentation result contains unrecognized fragments, the Chinese name abbreviation is added for second word segmentation.

8. The method of claim 6, wherein step s7 is performed by using a sequence reversing labeler in the POS labeling process, wherein first using Brown trigram tagger, then using bigram tagger, and finally using onegar tagger, the segments appearing in the named entity list are labeled [ NN ], and the unidentified segments are labeled [ NN ].

Images