CN113434840A

CN113434840A - Mobile phone continuous identity authentication method and device based on feature map

Info

Publication number: CN113434840A
Application number: CN202110740280.XA
Authority: CN
Inventors: 何慧; 宋晨; 王焕然; 张伟哲; 方滨兴; 蒙俊雄
Original assignee: Harbin Institute of Technology
Current assignee: Harbin Institute of Technology
Priority date: 2021-06-30
Filing date: 2021-06-30
Publication date: 2021-09-24
Anticipated expiration: 2041-06-30
Also published as: CN113434840B

Abstract

A mobile phone continuous identity authentication method and device based on a feature diagram relates to the technical field of identity authentication and is used for solving the problem that the existing mobile phone identity authentication method is not enough to effectively protect the privacy and the safety of a mobile phone owner. The technical points of the invention comprise: the method comprises the steps that information generated by each touch screen operation of a user is constructed into a specific touch characteristic diagram, the touch characteristic diagram comprises a single-touch characteristic diagram, an interval characteristic diagram and a sliding gesture characteristic diagram, and the authentication and identification of the user are realized through the identification of the touch characteristic diagram; decision strategies are further devised to more accurately and efficiently identify users. The invention can continuously and effectively protect the mobile phone privacy and property safety of the mobile phone owner under the condition that the mobile phone is lost or the password is leaked. The method and the device are suitable for continuous identity authentication of the smart phone.

Description

Mobile phone continuous identity authentication method and device based on feature map

Technical Field

The invention relates to the technical field of identity authentication, in particular to a method and a device for continuously authenticating a mobile phone identity based on a feature diagram.

Background

The mobile phone has become an important device for people to communicate, entertain and pay, and especially, the importance of the mobile phone as a payment means for replacing cash and maintaining the safety of the mobile phone is self evident. The existing main modes aiming at the security of the mobile phone comprise a screen locking pattern and a screen locking password, and payment software comprises a payment password, however, once the mobile phone is lost or the password is leaked, a non-authentication user may break through the identity authentication when the mobile phone is unlocked or the payment is carried out by resetting the password application, directly inputting the password and the like, and the property and the privacy of the mobile phone owner are threatened. For example, some mobile phone users like to record passwords on notebooks and small paper slips, and the paper carriers recording the passwords may be acquired by non-authenticated users; when a user uses the mobile phone, a non-authentication user beside the user can directly see a sliding gesture pattern or an input password used when the user unlocks the mobile phone, and once the non-authentication user who obtains the password obtains the opportunity of operating the mobile phone of the user, the safety of the mobile phone of the user is greatly threatened. For this situation, the existing mobile phone identity authentication method cannot effectively identify the owner and other users.

Disclosure of Invention

In view of the above problems, the present invention provides a method and an apparatus for mobile phone continuous identity authentication based on a feature diagram, so as to solve the problem that the existing mobile phone identity authentication method cannot effectively protect the mobile phone privacy security of the mobile phone owner.

According to one aspect of the present invention, a method for continuously authenticating an identity of a mobile phone based on a feature map is provided, the method comprising the following steps:

step one, constructing a touch characteristic diagram according to data information generated by the operation of touching a mobile phone screen by fingers of an authenticated user and a non-authenticated user, and using the touch characteristic diagram as training set characteristic data;

selecting a classification algorithm and training a classification model according to the feature data of the training set to obtain a trained classification model;

thirdly, constructing a corresponding touch characteristic diagram according to data information generated by the operation of touching the mobile phone screen by the fingers of the unknown user, inputting the touch characteristic diagram into a trained classification model for classification, and obtaining a classification result of the touch characteristic diagram of the unknown user;

analyzing the classification result of the touch characteristic diagram based on the judgment strategy to obtain the identity authentication result of the unknown user as an authenticated user or a non-authenticated user;

wherein the touch profile comprises a single-touch profile.

Further, the construction process of the single-touch characteristic map comprises the following steps: acquiring touch data and sensor data generated when a user finger touches a mobile phone screen; and processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain a single-touch characteristic gray-scale image, namely the single-touch characteristic image.

Further, the feature data included in the single-touch feature map is: touch point x coordinate, touch point y coordinate, current touch time, pressure, contact area, touch major axis, touch minor axis, x coordinate difference, y coordinate difference, time difference of two adjacent touch records, average value, variance of all time differences, maximum value, minimum value, average value, variance of all pressure, maximum value, minimum value, average value, variance of all touch major axes, maximum value, minimum value, average value, variance of all areas, maximum value, minimum value, average value, variance of all touch minor axes, acceleration, gyroscope, direction sensor data, difference of two adjacent acceleration sensor data, difference of two adjacent gyroscope sensor data, difference of two adjacent direction sensor data, average value, all acceleration, direction sensor data, and average value of all acceleration, And respectively calculating the variance of the data of the gyroscope and the direction sensor, the square sum of 3 axial data of each sensor and the letter key area of the touch point at the corresponding position.

Further, the touch characteristic diagram further comprises an interval characteristic diagram, and the interval characteristic diagram is constructed by the following steps: acquiring touch data and sensor data generated by a user finger continuously touching a mobile phone screen twice, wherein the time interval between the continuous two touches is smaller than a preset time threshold; processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain an interval characteristic gray-scale map, namely an interval characteristic map;

the characteristic data included in the interval characteristic diagram is as follows: the area number and the duration of the first touch, the area number and the duration of the second touch, the interval duration of the second touch, the time difference between the two touches, the ratio of the interval duration of the two touches to the duration of the first touch, the ratio of the interval duration of the two touches to the duration of the second touch, the ratio of the duration of the second touch to the duration of the first touch, the difference between the acceleration sensor data of the second touch and the first touch, the difference between the gyroscope sensor data of the second touch and the first touch, the difference between the direction sensor data of the second touch and the first touch, and the direction, the length and the speed from the first touch start coordinate to the second touch start coordinate.

Further, the touch characteristic diagram further comprises a slide gesture characteristic diagram, and the construction process of the slide gesture characteristic diagram is as follows: acquiring touch data and sensor data generated by sliding a mobile phone screen by a finger of a user; and processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain a sliding gesture characteristic gray-scale image, namely the sliding gesture characteristic image.

Further, the classification algorithm in the second step is a convolutional neural network algorithm.

Further, the touch feature map classification result in the fourth step is a positive example or a negative example, and a first decision strategy is designed based on the single-touch feature map classification result:

if more than x classification results in the n single-touch characteristic graphs are positive examples, the n single-touch characteristic graphs are determined as authenticated users, and otherwise, the n single-touch characteristic graphs are determined as non-authenticated users; wherein n is more than or equal to 3, x is a preset discrimination threshold, and x is more than or equal to 0 and less than or equal to n-1.

Further, the touch feature map classification result in the fourth step is a positive example or a negative example, and a second determination strategy is designed based on the single-touch feature map and the interval feature map classification result:

after a user is identified by using the strategy each time, a state mark is set, if the final authentication result is that the user is authenticated, the current state is marked as an authentication trend, if the final authentication result is that the user is not authenticated, the current state is marked as a non-authentication trend, the initial state is marked as an authentication trend by default, and when the user is identified by using the strategy, the state mark set based on the last final determination result is checked;

firstly, judging according to a first judgment strategy to obtain a primary identity authentication result of an unknown user;

if the current state is 'authentication trend': if the initial identity authentication result is the authenticated user, the authenticated user is finally identified; if the primary identity authentication result is a non-authentication user, judging according to the number of the latest n-1 interval feature map classification results as positive examples: when the number of the positive examples is less than or equal to x, finally, the user is regarded as a non-authentication user; otherwise, the terminal is finally determined as an authenticated user;

if the current state is 'non-authentication trend': if the initial identity authentication result is a non-authentication user, the user is finally identified as the non-authentication user; if the primary identity authentication result is the authenticated user, judging according to the number of the positive examples of the latest n-1 interval feature map classification results: when the number of the positive examples is larger than x, the user is finally determined as the authenticated user; otherwise, the user is finally identified as a non-authenticated user.

According to another aspect of the present invention, a device for continuously authenticating an identity of a mobile phone based on a feature map is provided, the device comprising:

the characteristic graph building module is used for building a touch characteristic graph as training set characteristic data according to data information generated by the operation that fingers of an authenticated user and a non-authenticated user touch the mobile phone screen;

the classification model training module is used for selecting a classification algorithm and training a classification model according to the training set characteristic data to obtain a trained classification model; wherein the classification algorithm is a convolutional neural network algorithm;

the characteristic diagram classification module is used for constructing a corresponding touch characteristic diagram according to data information generated by operation of touching a mobile phone screen by fingers of an unknown user, inputting the touch characteristic diagram into a trained classification model for classification, and obtaining a classification result of the touch characteristic diagram of the unknown user;

the identity authentication module is used for analyzing the touch characteristic diagram classification result based on the judgment strategy and obtaining the identity authentication result of the unknown user as an authenticated user or a non-authenticated user;

the touch characteristic diagram comprises a single-touch characteristic diagram, an interval characteristic diagram and a sliding gesture characteristic diagram, and the construction process of the single-touch characteristic diagram comprises the following steps: acquiring touch data and sensor data generated when a user finger touches a mobile phone screen; processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain a single-touch characteristic gray-scale image, namely a single-touch characteristic image; the construction process of the interval characteristic diagram comprises the following steps: acquiring touch data and sensor data generated by a user finger continuously touching a mobile phone screen twice, wherein the time interval between the continuous two touches is smaller than a preset time threshold; processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain an interval characteristic gray-scale map, namely an interval characteristic map; the construction process of the sliding gesture feature map comprises the following steps: acquiring touch data and sensor data generated by sliding a mobile phone screen by a finger of a user; and processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain a sliding gesture characteristic gray-scale image, namely the sliding gesture characteristic image.

Further, the process of analyzing the touch characteristic diagram classification result based on the decision strategy in the identity authentication module to obtain the identity authentication result of the unknown user as an authenticated user or a non-authenticated user is as follows: the touch characteristic diagram classification result is a positive example or a negative example, and a first judgment strategy is designed based on the single touch characteristic diagram classification result: if more than x classification results in the n single-touch characteristic graphs are positive examples, the n single-touch characteristic graphs are determined as authenticated users, and otherwise, the n single-touch characteristic graphs are determined as non-authenticated users; wherein n is more than or equal to 3, x is a preset discrimination threshold, and x is more than or equal to 0 and less than or equal to n-1;

designing a second judgment strategy based on the single-touch characteristic diagram and the interval characteristic diagram classification result:

The beneficial technical effects of the invention are as follows:

the invention utilizes the related information generated by touching the screen when the user uses the mobile phone to identify whether the user currently using the mobile phone is an authenticated user, so as to maintain the privacy and property safety of the authenticated user in the scene of mobile phone loss or password leakage. The invention is characterized in that the invention constructs the information generated by the operation of touching the screen by the user each time (from pressing the finger to lifting the finger) into a specific touch characteristic diagram, and further realizes the identification of the user by identifying the characteristic diagram; meanwhile, a judgment strategy is designed to further accurately and effectively identify the user. For application scenes that a user uses a mobile phone to type or input letter codes, inputs digital codes through mobile payment, slides to unlock a screen and the like, the method can continuously and effectively protect the privacy and property safety of the mobile phone owner under the condition that a non-authenticated user knows the codes.

Drawings

The invention may be better understood by referring to the following description in conjunction with the accompanying drawings, in which like reference numerals are used throughout the figures to indicate like or similar parts. The accompanying drawings, which are incorporated in and form a part of this specification, illustrate preferred embodiments of the present invention and, together with the detailed description, serve to further explain the principles and advantages of the invention.

FIG. 1 is a flow chart of a method for continuous identity authentication of a mobile phone based on a feature diagram according to the present invention;

FIG. 2 is a diagram illustrating a difference between pressing the "a" key and the "p" key by 10 different persons in the present invention;

FIG. 3 is an example of a single-touch profile constructed by different 5-person keys "p" in the present invention;

FIG. 4 is a schematic diagram of the timing characteristics of two consecutive key presses according to the present invention;

FIG. 5 is a schematic view of the change in orientation of two consecutive touches in the present invention;

FIG. 6 is an example of a spacing profile for pressing "p" after pressing "i" for a different 5-person key in the present invention;

FIG. 7 is an exemplary sliding gesture feature diagram constructed by the user sliding pattern "S" in the present invention;

FIG. 8 is a schematic diagram of the convolutional neural network structure of the present invention;

FIG. 9 is a graph of different recognition strategies FRR, FAR of the convolutional neural network as a function of threshold in the present invention;

FIG. 10 is a schematic structural diagram of a device for continuously authenticating identity of a mobile phone based on a feature diagram according to the present invention;

fig. 11 is a block diagram of a mobile phone persistent identity authentication apparatus according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the disclosure, exemplary embodiments or examples of the disclosure are described below with reference to the accompanying drawings. It is obvious that the described embodiments or examples are only some, but not all embodiments or examples of the invention. All other embodiments or examples obtained by a person of ordinary skill in the art based on the embodiments or examples of the present invention without any creative effort shall fall within the protection scope of the present invention.

The continuous identity authentication is an implicit process of capturing the behavior attributes of a user currently operating a mobile phone continuously or regularly by utilizing resources of the mobile device and a built-in sensor, and verifying whether the behavior attributes are the authenticated user or not by analyzing the behavior attributes. Different users generally have unique behavior patterns when using the mobile phone, and whether the current user is the authenticated user of the equipment can be judged based on the behavior patterns. The continuous identity authentication of the mobile phone can still effectively protect the security of the mobile phone under the condition that the password is known by the non-authenticated user.

Based on the above, the main content of the invention is to identify whether the user currently using the mobile phone is an authenticated user by using the related information generated by the touch screen when the user uses the mobile phone, so as to maintain the privacy and property safety of the authenticated user in the scene of mobile phone loss or password leakage. The invention is characterized in that the invention constructs the information generated by the operation (from pressing the finger to lifting the finger) of the user touching the screen each time as a specific touch characteristic diagram, and further realizes the identification of the user through the identification of the characteristic diagram. Meanwhile, the user can be effectively identified under the condition that a plurality of characteristic graphs (corresponding to the condition that the user touches the screen for a plurality of times) are used at the same time. As shown in fig. 1, a method for continuously authenticating a mobile phone identity based on a feature map includes the following steps:

step one, constructing a touch characteristic diagram according to data information generated by the operation of touching a mobile phone screen by fingers of an authenticated user and a non-authenticated user, and using the touch characteristic diagram as training set characteristic data; selecting a classification algorithm and training a classification model according to the feature data of the training set to obtain a trained classification model; thirdly, constructing a corresponding touch characteristic diagram according to data information generated by the operation of touching the mobile phone screen by the fingers of the unknown user, inputting the touch characteristic diagram into a trained classification model for classification, and obtaining a classification result of the touch characteristic diagram of the unknown user; analyzing the classification result of the touch characteristic diagram based on the judgment strategy to obtain the identity authentication result of the unknown user as an authenticated user or a non-authenticated user; the touch characteristic diagram comprises a single-touch characteristic diagram, an interval characteristic diagram and a sliding gesture characteristic diagram. The persistent authentication method will be described in detail below.

1. Constructing feature maps

When a finger touches a screen once, the change of the sensor generated along with the touch is an integral process, and the integral relevant data of the process is written into a picture, wherein the picture includes the characteristic of the touch. Compared with most other researches which adopt a time slicing mode, the mode can exclude the data interference which can exist in the time of a non-touch screen.

1.1 construction of Single-touch profiles

The single-touch characteristic diagram is used for characterizing the characters of letters, numerical passwords and all short-time touch screen scenes input by a user by using a mobile phone. A touch will produce several records, divided into touch records and sensor records, and table 1 defines the relevant characters and their meanings for use in the present invention.

TABLE 1

The data selected and the interpretation of the write-once-touch profile are shown in table 2.

TABLE 2

The last one in table 2 is explained additionally: different people use the same mobile phone, and the touch points of the different people pressing specific keys have own characteristics due to different palm sizes and input habits, and fig. 2 shows the difference of 10 different people when clicking the letters "a" and "p", so that the touch points of the same user are connected for convenient observation to show the habit touch areas.

And calculating and identifying a key corresponding to the current touch position through touch coordinates (x, y), and calculating relative coordinates on the key to obtain one item of data of the characteristic diagram.

The data to be written into the signature has the following characteristics: the negative number is small, the number is mostly floating point number from 0 to 2040, the maximum number is 10 digits after decimal point, and the traditional data normalization effect is not ideal. Therefore, the data of each type is determined to be amplified 10 according to the actual situation of the dataⁿDepending on the number of decimal places of such data, n is changed to an integer and then converted to a binary write profile.

The construction process of the characteristic diagram comprises the following steps: and (3) creating 300-40 all-zero matrixes, processing the data in the table 2, writing the processed data line by line, and finally storing the processed data as a gray picture. The number of the relevant data of the table 2 generated by one conventional letter and number input or sliding operation is generally not more than 300, and the picture height is selected to be 300 in order to ensure that the constructed characteristic picture occupies small space as much as possible and can store the data to the maximum extent; the picture width is set to 40, one datum is represented by a 40-bit binary number, the first 5 bits are specified as negative sign bits, and the last 35 bits are specified as binary numbers, so that the aim of representing and converting the datum is achieved as thoroughly as possible and efficiently. The constructed single-touch characteristic diagram is shown in the following by way of example, and the characteristic diagram constructed according to the above idea for different 5-person keys "p" is shown in fig. 3 (in order to express the characteristic of the characteristic diagram, the data are uniformly multiplied by 255 and shown here, otherwise, the gray scale diagram with all data of 0 and 1 is visually seen as a completely black picture).

In fig. 3, the white portion represents that the corresponding position data in the picture is 1, the black portion represents that the corresponding position data in the picture is 0, the top portion of the picture is related raw data recorded as "down", "move", and "up", the middle portion is related raw data of three sensors, and the bottom portion is data that needs to be calculated additionally, such as maximum value, minimum value, average value, and the like. The longer the key press time is, the more data is generated, and the more white part area of the corresponding feature map is. The difference of 5 people touching the same key feature map can be seen from fig. 3, and then the feature maps are classified by using a classification algorithm to achieve the purpose of distinguishing authenticated users from non-authenticated users.

Constructing a single-touch characteristic diagram performance analysis: for one touch of a user, a program can generally form a corresponding single-touch characteristic diagram within 0.03 second, the image storage format is a bmp type image, and each single-touch characteristic diagram occupies 16kb of a hard disk space.

1.2 constructing a touch interval feature map

The touch interval characteristic diagram can effectively record the habit characteristics of the user in continuous input and is used for assisting the single-touch characteristic diagram to classify the user. The 2 consecutive touch time characteristics during touching the screen can be abstracted as shown in fig. 4.

Based on the time profile shown in fig. 4, data of the selectable write interval profile is shown in table 3 below.

TABLE 3

The last 3 items in table 3 are explained additionally, when the screen is touched continuously (for example, when the screen is typed continuously), the finger may have its own speed and direction rules when clicking to different directions, therefore, a coordinate system is constructed by using the first touch point as the origin, the coordinate system is divided into 8 areas, and finally the characteristics of the direction, the movement length and the speed of the two touches are written into the picture, the schematic diagram of the coordinates is shown in fig. 5, and the direction 2 corresponds to the two touches in the picture.

The data is written into the matrix of 31 x 24 according to the single-touch characteristic diagram data processing idea, and the data in table 3 can be represented by 31 rows, and the data calculated in table 3 is small in whole, and can be represented by 24-bit binary, and the data is defined as data represented by the first 4 bits as negative flag bits and the last 20 bits as binary. Finally, the matrix is stored as a picture, and fig. 6 is an interval feature graph formed by pressing "p" after pressing "i" by 5 different people (the data is displayed by multiplying 255 in the same single-touch feature graph data processing thought).

In fig. 6, the white part represents that the corresponding position data in the picture is 1, the black part represents that the corresponding position data in the picture is 0, the upper part of the picture is the calculated information such as the position, the time difference ratio and the like, and the lower part of the picture is the information such as the difference value of two adjacent touch sensors. Similarly to the single-touch characteristic diagram, the difference that 5 persons press the "p" key after pressing the "i" key can be visually seen. This profile will be used hereinafter to assist in determining, identifying, authenticated users and non-authenticated users.

The interval characteristic of two continuous touches is more meaningful only under the condition of continuous input, because the data reflects the habitual characteristics of speed, key pressing force and the like when a person continuously inputs specific two letters, the interval time is too long, the characteristic is weakened or even completely loses meaning, and the subsequent classification result can be interfered, so that a time threshold value is set, and an interval characteristic graph is generated only when the continuous two touches are less than the threshold value. The collected data was analyzed and the threshold was set to 1300 ms, which covers substantially all of the continuous input scenario.

Constructing an interval characteristic diagram performance analysis: for a single touch of a user, if the time interval between the touch and the last touch is smaller than a threshold value, the program may generally construct a single-touch feature map and a corresponding interval feature map within 0.05 second, where the picture storage format is a bmp-type picture, and each interval feature map occupies 4kb of hard disk space.

1.3 constructing a slide gesture feature map

When a user slides a screen to unlock the mobile phone, one sliding gesture is coherent, and the related information in the whole process is written into one picture. The invention focuses on the Sudoku sliding unlocking which is widely applied, and the common sliding unlocking can also be regarded as a part of a partial Sudoku sliding unlocking gesture, so that the method is also applicable to research results. After the user slides the unlocking pattern once, no matter whether the screen is successfully unlocked or not, the next operation is irrelevant to the operation, so that a touch interval characteristic diagram corresponding to a sliding gesture does not need to be made.

The swipe gesture feature map can be performed according to the concept of constructing a single-touch feature map, and the features in table 2, except for the last "region feature by specific letter", can be used for the swipe gesture feature map.

In the data set of the present invention, young people, all ages 22-30, typically touch an alpha (numeric) key once for no more than 100 milliseconds, but slide an unlock pattern for as much as 2000 milliseconds or more. This also results in a large increase in the "move" information and sensor information returned by the system. Therefore, the signature is changed to 1700 × 40, writing the returned "move" information of the system as completely as possible; meanwhile, certain sampling writing is carried out on the sensor information according to the actual situation, for example, one piece is written in every n pieces, and the data of each type of sensor does not exceed 7 pieces at most, so that the key information is not lost, the data storage efficiency can be improved, and the next classification and identification are facilitated. In the present invention, not all sensor data is written into the picture, also because too much raw sensor data will reduce the recognition accuracy through testing.

The characteristic diagram of the slide gesture constructed according to the user slide pattern "S" is shown in fig. 7, and since the diagram is too high, fig. 7 is shown rotated 90 degrees to the left.

Also like the single-touch characteristic diagram, the white part represents that the corresponding position data in the picture is 1, the black part represents that the corresponding position data in the picture is 0, the left side represents the relevant raw data recorded by "down", "up" and a large number of "move", and the rightmost side represents the three sensor relevant raw data and the data which needs additional calculation. It can be seen that a large number of areas with data of 0 exist in the middle of the picture, and the purpose of the design is to make the generated feature picture as large as possible and make the picture fixed in position store fixed relevant information so as to improve the subsequent picture recognition efficiency is considered to leave space for more 'move' records possibly generated by touching the screen for a longer time.

Constructing a sliding gesture feature map performance analysis: for a slide gesture of a user, a program can generally form a corresponding slide gesture feature map within 0.1 second, the picture storage format is a bmp type picture, and each slide gesture feature map occupies 68kb of hard disk space.

2. Feature map classification algorithm: convolutional neural network

The invention aims to achieve the purpose of identifying whether a user is an authenticated user or not through the three feature graphs constructed in the way, and a convolutional neural network is constructed by combining the characteristics of the feature graphs and experimental data volume.

Referring to the design idea of deep learning VGG-19 network structure, the convolution part of the convolutional neural network also adopts a continuous small convolution kernel mode; by combining the characteristics that the picture size of the characteristic diagram constructed by the invention is not large and the characteristic diagram is a single-channel diagram, no pooling layer is arranged, and data is kept as much as possible under the premise of being allowed by computing power so as to improve the precision. The convolutional neural network structure with the best effect of classifying the characteristic diagram of the invention is obtained by adjusting parameters and testing for multiple times, as shown in fig. 8, firstly, a picture extracts characteristics through two layers of convolution kernels with 3 × 3 convolution kernels and step length of 1, and an activation function selects 'ReLu'; the data was subsequently flattened using the flatten function; then passing through a full connection layer of 128 nodes in 2 layers and a random zero setting layer of 0.1 probability in 1 layer; eventually reaching the output level of 10 nodes.

3. The technical effect of the invention is verified through experiments

The experimental data set is characterized in that 70 persons of data are collected, wherein 58 men and 12 women are college students who are aged 20-30 and often use mobile phones in daily life, and each participant needs to use the experimental mobile phone to input a fixed 19-digit letter string and a fixed 6-digit number string for 2 times according to prompts, so that the login and payment scene of known internet bank login passwords and WeChat payment passwords is simulated; then inputting 1 short English sentence "The quick brown fox jumps over The lazy dog" and "pack my box with five park riser dogs" capable of covering 26 English letters, aiming at collecting data of clicking different screen positions as much as possible for further analysis and research; and finally, the participant is required to slide 6 specific mobile phone sliding unlocking screen patterns according to the prompt. The data collection mode is used for simulating three scenes of daily typing or inputting letter codes, inputting digital codes through mobile payment and unlocking a screen through sliding of the mobile phone.

In the collected 70 persons data, 1 person is set as the identity of the authenticated user, and the collected data is defined as the regular data; the remaining 69 persons were set as non-authenticated users, and the collected data was defined as counterexample data. According to the training set and the test set 8: 2, randomly selecting 14 persons from the non-authenticated users, using the corresponding feature maps as a test set counter example, and using the remaining feature maps of 55 persons as a training set counter example. The authenticated user additionally performs a plurality of times of data acquisition, so that the number of the feature graphs and the number of counter examples reach one magnitude. The data collected by the authenticated users are performed in batches on different 4 days, and the data similarity generated when the same person operates the mobile phone in a certain time period is very high, so that the experimental result is only that the identification accuracy is high, and the experimental result is not practical.

Classifying the data to obtain: 7336 positive example pictures and 7812 negative example pictures of the training set of the single-touch characteristic map; the test set comprises 2038 positive examples and 1993 negative examples. 7150 positive example pictures and 6725 negative example pictures of the interval feature map training set; test set 1970 and test set 1616.

Another purpose of the above categorized data is to make the training set and test set data completely non-coincident, corresponding to the fact that the network is trained with the known 1 authenticated user data and 55 non-authenticated user data to identify the new data for the authenticated user and the new 14 non-authenticated user data. The training set and the test set are designed in such a way to correspond to the actual situation: non-authenticated users to be detected in real life are generally not present in the training set.

The classification effect of different characteristic graphs is tested in sequence in a simulation experiment, and the influence of different classification strategies on the accuracy of the user identification is researched.

3.1 Classification Effect of Individual different feature maps

Table 4 below shows the recognition accuracy of the single-touch feature map and the interval feature map using a common classification algorithm SVM, logistic regression, decision tree, gradient boosting decision tree algorithm, and convolutional neural network.

TABLE 4

In the above several common algorithms, the best expression is the convolutional neural network, and the data in table 4 shows that, in the case of considering only one single-touch characteristic diagram (corresponding to the actual situation, that is, the user touches the screen 1 time), the recognition rate for the authenticated user is 93.82%, and the recognition rate for the non-authenticated user is 94.02%; when only one interval feature map is considered, the recognition rate of the authenticated user is 72.89%, and the recognition rate of the non-authenticated user is 67.01%.

The swipe gesture feature recognition effect is shown in table 5 below.

TABLE 5

As can be seen from table 5, both the logistic regression algorithm and the convolutional neural network achieve better recognition effect. Corresponding to the actual situation, namely, the recognition accuracy of the slide gesture of the authenticated user is 95%, and the recognition accuracy of the slide gesture of the non-authenticated user is 96.02%.

3.2 Classification Effect of multiple feature maps

In actual use, it may be considered to use multiple feature maps simultaneously to determine whether the user is an authenticated user. In a scene of actually inputting a password, the length of the password is generally greater than or equal to 6 bits, so this section focuses on how to effectively identify a user in a process of less than or equal to 6 touches.

In the field of mobile phone continuous certification research, 2 common evaluation and identification effect indexes are as follows: the error rejection rate FRR (false Reject rate), the error acceptance rate FAR (false Accept rate), and the calculation formula are shown in formula (1):

(1) using only single touch profiles

The user can obtain a single-touch characteristic diagram by touching the screen once, and the classification result of the touch characteristic diagram can be obtained through the classification of section 3.1: 0 or 1. When the user touches the mobile phone screen for 6 times continuously, a continuous recognition result can be obtained, for example, a user touches the mobile phone screen for 6 times continuously to obtain a single-touch feature map recognition result of "110101", and then, it is to be discussed, specifically, the most recent feature map results (corresponding to the cases of "101", "0101", "10101" and "110101" 4 as examples) are used, and when more than one feature map result is identified as an authenticated user, the lowest FRR and FAR can be obtained. The following 4 strategies (i.e., the first decision strategy) are designed based on this.

And observing the single-touch characteristic diagram formed by the current operation and a plurality of single-touch characteristic diagrams generated recently, if:

strategy 1: more than x pictures (preset discrimination threshold) in the 6 pictures are identified as positive examples, namely, the positive examples are identified as the authenticated users.

Strategy 2: and if more than x pictures (preset discrimination threshold values) in the 5 pictures are recognized as positive examples, the positive examples are considered as the authenticated users.

Strategy 3: more than x pictures (preset discrimination threshold) in the 4 pictures are identified as positive examples, namely, the pictures are identified as authenticated users.

Strategy 4: and if more than x pictures (preset discrimination threshold values) in the 3 pictures are recognized as positive examples, the positive examples are considered as the authenticated users.

By the strategy, according to the single-touch characteristic diagram identification result, change curves of FRR and FAR can be drawn, for example, as shown in a graph (a) corresponding to a strategy 1, as a judgment threshold value is increased, an FRR curve is continuously raised, and corresponding reality means that an authenticated user is more easily identified as a non-authenticated user by mistake, and the normal use of a mobile phone by the user is possibly influenced; the FAR curve is constantly dropping, and the corresponding reality means that non-authenticated users will be more easily detected and safer. The intersection point of the two curves corresponds to the actual situation, namely the most balanced point which can not only take care of user experience, but also block non-authentication users to the maximum extent.

(2) Single touch profile and spacing profile for use in conjunction with each other

Similar to the above thought, the interval feature diagram auxiliary judgment is added on the basis of the most balanced scheme of each FRR and FAR, and at most n-1 interval feature diagrams can be generated by n times of touch, so that at most n-1 interval feature diagram results can be found by observing the n single-touch feature diagram results. Still based on the above example, assuming that 5 interval feature maps are obtained by 6 touches, the recognition result is "11001", and the following discussion is to be made, and based on the previous results, the minimum FRR and FAR can be obtained by specifically using the latest interval feature map results (i.e., "11001", "1001", "001", and "01" in 4 cases) and using what strategy to assist the determination when the number of 1 in the interval feature map results is more than or equal to that.

Firstly, setting a state flag by using a judgment result of the current strategy used last time, and if the user is judged to be authenticated last time, assuming that the current state is an 'authentication trend': the operation at this time is likely to authenticate the user; if the user is determined to be the unauthenticated user last time, the current state is assumed to be the unauthenticated trend': the operation of this time is likely to be an unauthenticated user. Wherein the initial state flag defaults to "authentication tendency". Based on this assumption, the following strategy (i.e., the second decision strategy) can be designed and analyzed:

strategy 5: firstly, a strategy 1 most balanced scheme (more than 3 pictures are positive examples, namely the authenticated user) is used for identification, and a preliminary identification result is obtained.

If the current state is an authentication trend, the initial identification result is a non-authentication user and does not accord with the trend, checking the number of the latest 5 interval characteristic graphs identified as positive examples, and if the number is less than or equal to x (a judgment threshold), finally identifying the current state as the non-authentication user, and meanwhile, modifying the current state into a non-authentication trend; if the number of the authentication keys is larger than x, the authentication key is finally determined as the authenticated user. And if the preliminary identification result is the authenticated user and is consistent with the trend, the authenticated user is finally identified.

If the current situation is a non-authentication trend, the initial identification result is an authentication user, and the trend is not consistent with the authentication user, then checking the number of the latest 5 interval characteristic graphs identified as positive examples, if the number is more than x, finally identifying the authentication user, and simultaneously modifying the current state into an authentication trend; if the number of the users is less than x, the users are finally identified as non-authenticated users. And if the preliminary identification result is that the user is the non-authentication user and the trend is consistent, the user is finally identified as the non-authentication user.

Strategy 6: and 5 single-touch characteristic graphs and 4 interval characteristic graphs corresponding to the single-touch characteristic graphs are observed, the number of positive and negative examples of the corresponding interval characteristic graphs is observed on the basis of the identification result of the strategy 2 most balanced scheme, the range of the threshold value x is [0,1,2,3,4], and the idea of the strategy 5 is compared for testing.

Strategy 7: and 4 single-touch characteristic graphs and 3 interval characteristic graphs corresponding to the single-touch characteristic graphs are observed, the number of positive and negative examples of the corresponding interval characteristic graphs is observed on the basis of the strategy 3 most balanced scheme identification result, the threshold value x is in the value range of [0,1,2 and 3], and the idea is tested by analogy with the strategy 5.

Strategy 8: and observing 3 single-touch characteristic graphs and 2 interval characteristic graphs corresponding to the single-touch characteristic graphs, observing the number of positive and negative examples of the corresponding interval characteristic graphs on the basis of the identification result of the strategy 4 most balanced scheme, wherein the range of the threshold value x is [0,1,2], and performing test by analogy with the strategy 5.

In addition, the additional test observes the classification effect of 10 feature map cases. The most balanced FRR, FAR of the convolutional neural network using the above strategy is shown in table 6 below.

TABLE 6

Fig. 9 shows graphs of changes of FRR and FAR with respect to threshold values in the cases of observing the policy 1, the policy 5, and the 10 pictures corresponding to the 6 pictures. As can be seen from fig. 9, when only the single-touch feature map is used, the recognition effect of FRR 1.23% and FAR 4.03% can be obtained by observing 6 pictures, and further, in combination with the interval feature map, the recognition effect of FRR 0.84% and FAR 0.40% can be obtained. The more the number of the observed characteristic pictures is, the better the effect of the corresponding most balanced strategy is.

It should be noted that, in the two feature map combination strategy, when 12 pictures are observed, in addition to the positive example of observing more than 6 feature maps of 12 pictures, that is, when the user is determined to be authenticated, when the interval feature map auxiliary determination threshold x is 6, the effect that FRR is 0 and FAR is 0 can be obtained. This result also means that, for practical purposes, the user can be identified with an accuracy of FRR 0.84% and FAR 0.40% (average identification accuracy 99.38% after calculation) after 6 clicks on the screen, and in theory, the user can be identified accurately with more than 12 clicks.

According to another embodiment of the present invention, a device for continuously authenticating a mobile phone based on a feature diagram is provided, as shown in fig. 10 and 11, the device includes:

the characteristic diagram constructing module 110 is configured to construct a touch characteristic diagram according to data information generated by operation of touching a mobile phone screen by fingers of an authenticated user and a non-authenticated user, and the touch characteristic diagram is used as training set characteristic data;

according to the embodiment of the invention, the mobile phone comprises a data collection and uploading sub-module 1110, which is deployed at a mobile phone end and is mainly responsible for collecting and uploading data generated by a user touch screen, and information generated by the user touch screen is continuously acquired by calling a relevant system function of the mobile phone. Each time the user touches the screen, the screen is moved from being pressed down to being lifted up to be a complete action, information and data types (common actions/sliding gestures) collected by the system function during the complete action are stored in a file with the finger lifting moment as a time label, and the file is sent to the information processing submodule 1120. And after the file is sent for several seconds, sending a query result request with the number of the time tag. The training stage corresponds to the situation that a user uses a system for the first time or supplements training data, the recognition stage corresponds to the situation that a mobile phone is protected by normalized operation, when the training stage and the recognition stage of the mobile phone end of the user are switched, a message is sent to a server, and the server correspondingly switches response states.

And the information processing sub-module 1120 is deployed on a website server, the website server is connected with a MySQL database, and the database stores the identification result of a certain touch screen of a user by using a ResultRecord table. The main tasks of the information processing module are to respond to the following requests: and the user uploads the information request. When a file uploading request of a user is received, the file is agreed and received, and the received file named by the time tag is stored in a system log folder. The result is a return request. And after receiving the query result request with the time tag number, the server is connected with the database, accesses the ResultRecord table in which the identification result is stored, searches the identification result corresponding to the current time tag and returns the result to the mobile phone terminal.

The feature map making sub-module 1130 is deployed on a website server and has a main task of making various feature maps. And continuously monitoring a system log folder, if a new file appears, making a single-touch characteristic diagram or a sliding gesture characteristic diagram according to the data type (common touch/sliding gesture) according to the content of the newly added file in the folder, and if the difference value of the time labels of the last single-touch characteristic diagram and the single-touch characteristic diagram to be made at this time is less than 1300 milliseconds, additionally making an interval characteristic diagram. The profiles are named with the time labels of the corresponding files. According to the server state: the training or recognition phase, the generated feature maps are forwarded to the model training module 120 or the classification module 130.

The model training module 120 is used for selecting a classification algorithm and training a classification model according to the training set feature data to obtain a trained classification model; wherein, the classification algorithm is a convolutional neural network algorithm;

according to the embodiment of the present invention, the model training module 120, i.e., the data training module, is deployed on the website server, and the main task is to train the convolutional neural network by using the generated touch feature map. When the server is in the training phase, the model training module 120 continuously receives feature images of different classes and stores the feature images in a classified manner, and when a training phase end message is received, the received feature images and the existing counterexample feature images are used for training corresponding classifiers: a single-touch feature map classifier, an interval feature map classifier and a sliding gesture classifier. And after the training is finished, creating or replacing the existing classifier of the data recognition module.

The classification module 130 is configured to construct a corresponding touch feature map according to data information generated by an operation of touching a mobile phone screen by a finger of an unknown user, and input the touch feature map into a trained classification model for classification to obtain a classification result of the touch feature map of the unknown user;

according to the embodiment of the present invention, the data recognition sub-module 1310 is deployed on the website server, and the main task is to recognize the feature pattern transmitted from the feature pattern making sub-module 1130 in the data recognition stage. And when the single-touch characteristic diagram, the interval characteristic diagram or the sliding gesture characteristic diagram is received, operating the single-touch characteristic diagram, the interval characteristic diagram or the sliding gesture characteristic diagram in a corresponding classifier to obtain a recognition result. And saves the "time tag + identification result" to the corresponding field in the ResultRecord table of the database. And simultaneously calling out the classification result of the latest n single-touch characteristic graphs of the time label characteristic graph and the classification result of the corresponding interval characteristic graph according to a set judgment strategy respectively to obtain a judgment result: and storing the identification result into the identification result field of the same time label of the ResultRecord table by a legal user or an illegal user.

The identity authentication module 140 is configured to analyze the touch characteristic map classification result based on the decision policy, and obtain an identity authentication result of the unknown user as an authenticated user or a non-authenticated user;

the touch characteristic diagram comprises a single-touch characteristic diagram, an interval characteristic diagram and a sliding gesture characteristic diagram;

the construction process of the single-touch characteristic diagram comprises the following steps: acquiring touch data and sensor data generated when a user finger touches a mobile phone screen; processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain a single-touch characteristic gray-scale image, namely a single-touch characteristic image;

the construction process of the interval characteristic graph comprises the following steps: acquiring touch data and sensor data generated by a user finger continuously touching a mobile phone screen twice, wherein the time interval between the continuous two touches is smaller than a preset time threshold; processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain an interval characteristic gray graph, namely an interval characteristic graph;

the construction process of the sliding gesture feature map comprises the following steps: acquiring touch data and sensor data generated by sliding a mobile phone screen by a finger of a user; and processing the touch data and the sensor data, and writing the touch data and the sensor data into the all-zero matrix to obtain a sliding gesture characteristic gray-scale image, namely the sliding gesture characteristic image.

Further, the process of analyzing the touch characteristic diagram classification result based on the decision policy in the identity authentication module 140 to obtain the identity authentication result of the unknown user as the authenticated user or the non-authenticated user is as follows: the classification result of the touch characteristic diagram is a positive example or a negative example, and a first judgment strategy is designed based on the classification result of the single-touch characteristic diagram: if more than x classification results in the n single-touch characteristic graphs are positive examples, the n single-touch characteristic graphs are determined as authenticated users, and otherwise, the n single-touch characteristic graphs are determined as non-authenticated users; wherein n is more than or equal to 3, x is a preset discrimination threshold, and x is more than or equal to 0 and less than or equal to n-1;

designing a second judgment strategy based on the single-touch characteristic diagram and the interval characteristic diagram classification result: after a user is identified by using the strategy each time, a state mark is set, if the final authentication result is that the user is authenticated, the current state is marked as an authentication trend, if the final authentication result is that the user is not authenticated, the current state is marked as a non-authentication trend, the initial state is marked as an authentication trend by default, and when the user is identified by using the strategy, the state mark set based on the last final determination result is checked;

The functions of the device for continuously authenticating the identity of the mobile phone based on the feature diagram according to the embodiment of the present invention can be described by the method for continuously authenticating the identity of the mobile phone based on the feature diagram, so that the detailed description of the device embodiment is omitted, and the detailed description thereof is omitted.

While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as described herein. The present invention has been disclosed in an illustrative rather than a restrictive sense, and the scope of the present invention is defined by the appended claims.

Claims

1. A mobile phone continuous identity authentication method based on a feature map is characterized by comprising the following steps:

wherein the touch profile comprises a single-touch profile.

2. The method for mobile phone continuous identity authentication based on the feature map as claimed in claim 1, wherein the construction process of the single-touch feature map is as follows: acquiring touch data and sensor data generated when a user finger touches a mobile phone screen; and processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain a single-touch characteristic gray-scale image, namely the single-touch characteristic image.

3. The method for mobile phone continuous identity authentication based on the feature map as claimed in claim 2, wherein the feature data included in the single-touch feature map is: touch point x coordinate, touch point y coordinate, current touch time, pressure, contact area, touch major axis, touch minor axis, x coordinate difference, y coordinate difference, time difference of two adjacent touch records, average value, variance of all time differences, maximum value, minimum value, average value, variance of all pressure, maximum value, minimum value, average value, variance of all touch major axes, maximum value, minimum value, average value, variance of all areas, maximum value, minimum value, average value, variance of all touch minor axes, acceleration, gyroscope, direction sensor data, difference of two adjacent acceleration sensor data, difference of two adjacent gyroscope sensor data, difference of two adjacent direction sensor data, average value, all acceleration, direction sensor data, and average value of all acceleration, And respectively calculating the variance of the data of the gyroscope and the direction sensor, the square sum of 3 axial data of each sensor and the letter key area of the touch point at the corresponding position.

4. The method for mobile phone continuous identity authentication based on the feature map of claim 3, wherein the touch feature map further comprises an interval feature map, and the construction process of the interval feature map is as follows: acquiring touch data and sensor data generated by a user finger continuously touching a mobile phone screen twice, wherein the time interval between the continuous two touches is smaller than a preset time threshold; processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain an interval characteristic gray-scale map, namely an interval characteristic map;

5. The feature graph-based mobile phone continuous identity authentication method according to claim 4, wherein the touch feature graph further comprises a slide gesture feature graph, and the slide gesture feature graph is constructed by the following steps: acquiring touch data and sensor data generated by sliding a mobile phone screen by a finger of a user; and processing the touch data and the sensor data, and writing the touch data and the sensor data into an all-zero matrix to obtain a sliding gesture characteristic gray-scale image, namely the sliding gesture characteristic image.

6. The method as claimed in claim 5, wherein the classification algorithm in step two is a convolutional neural network algorithm.

7. The method according to claim 6, wherein the touch feature map classification result in step four is positive or negative, and the first decision policy is designed based on the single-touch feature map classification result:

8. The method according to claim 7, wherein the touch feature map classification result in step four is a positive example or a negative example, and the second decision strategy is designed based on the single-touch feature map and the interval feature map classification result:

9. A mobile phone continuous identity authentication device based on a feature map is characterized by comprising:

the model training module is used for selecting a classification algorithm and training a classification model according to the training set characteristic data to obtain a trained classification model; wherein the classification algorithm is a convolutional neural network algorithm;

the classification module is used for constructing a corresponding touch characteristic diagram according to data information generated by operation of touching a mobile phone screen by fingers of an unknown user, inputting the touch characteristic diagram into a trained classification model for classification, and obtaining a classification result of the touch characteristic diagram of the unknown user;

10. The continuous mobile phone identity authentication device based on the feature map as claimed in claim 9, wherein the identity authentication module analyzes the touch feature map classification result based on the decision policy, and the process of obtaining the identity authentication result of the unknown user as the authenticated user or the non-authenticated user comprises: the touch characteristic diagram classification result is a positive example or a negative example, and a first judgment strategy is designed based on the single touch characteristic diagram classification result: if more than x classification results in the n single-touch characteristic graphs are positive examples, the n single-touch characteristic graphs are determined as authenticated users, and otherwise, the n single-touch characteristic graphs are determined as non-authenticated users; wherein n is more than or equal to 3, x is a preset discrimination threshold, and x is more than or equal to 0 and less than or equal to n-1;