WO2018161893A1

WO2018161893A1 - User identification method and device

Info

Publication number: WO2018161893A1
Application number: PCT/CN2018/078139
Authority: WO
Inventors: 张军平; 黄晨宇; 张黔
Original assignee: 华为技术有限公司
Priority date: 2017-03-06
Filing date: 2018-03-06
Publication date: 2018-09-13
Also published as: CN108536314A

Abstract

Disclosed are a user identification method and device. The method comprises: constructing a trajectory for a user gesture according to an angular rotational speed measured in real time by a gyroscope of a terminal device, wherein the trajectory consists of three-dimensional coordinates of an arm of a user at each moment (S101); performing writing stroke segmentation processing on the trajectory and extracting feature information from each segment of a writing stroke (S102); and performing user identification according to the extracted feature information from each of the segments of the writing stroke and feature information of each segment of a writing stroke corresponding to a trajectory in at least one group of pre-stored feature information templates, wherein each group of pre-stored feature information templates comprise feature information of each segment of a writing stroke corresponding to at least one trajectory (S103). As a result, user identification is not limited by a handheld terminal device with respect to gestures recognized, thus increasing the accuracy of identification and enhancing the user experience.

Description

User identification method and device

The present application claims the priority of the Chinese Patent Application, filed on Jan. 6, 2017, the entire disclosure of which is hereby incorporated by reference.

Technical field

The present application relates to the field of communications technologies, and in particular, to a user identity identification method and apparatus.

Background technique

With the versatility and power of smart devices, most homes have more and more smart devices, such as smart watches, smart bracelets, smart oxygen meters, and smart blood pressure monitors. In these smart devices, it is sometimes necessary to record part of sensitive data (such as its own medical data, motion data, etc.), and the user does not want the data to be seen by others, so the smart device is required to recognize the identity of the current user. However, unlike mobile phone input user passwords, most smart devices do not have a touch screen, and most smart devices do not choose to add additional expensive sensors (such as fingerprint readers) for identification in order to reduce costs, and Most smart devices do not have strong computing power.

In the related art, AirSig is an identity authentication method based on gesture recognition. AirSig can input a password for a smart device lacking a touch screen, no expensive sensors and limited computing resources for user identification, and the user holds the smart device. Initiating an air gesture, the smart device can identify the identity of the air gesture initiator. AirSig identifies the user's identity by installing an accelerometer and a gyroscope in the smart device. When the user holds the smart device to initiate an air gesture, the real-time data output by the accelerometer and the gyroscope is compared with the pre-stored feature information template data.

In the above related art, the real-time data outputted by the accelerometer and the gyroscope is directly compared with the pre-stored feature information template data to identify the user identity. Therefore, the user's hand holding the smart device corresponding to the real-time data needs the user hand corresponding to the pre-stored feature information template. The gesture of holding the smart device is consistent to ensure the accuracy of the recognition, which is limited by the posture of the user holding the smart device.

Summary of the invention

The present application provides a user identification method and apparatus to solve the problem that the posture of the user holding the smart device is limited in the related art.

In a first aspect, the application provides a user identity identification method, including:

Constructing a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, the trajectory is composed of three-dimensional coordinate points of the user's arm at each moment; segmenting the stroke of the trajectory, and performing features on each stroke Information extraction; performing user identification according to the extracted feature information of each stroke of each stroke and the feature information of each stroke corresponding to one of the at least one set of pre-stored feature information templates, each set of pre-stored feature information templates including at least one track corresponding to each Characteristic information of a stroke.

The trajectory of the user gesture is constructed according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, the stroke segmentation processing is performed on the trajectory, and the feature information is extracted for each stroke, and finally, according to the extracted feature information of each stroke The feature information of each stroke corresponding to one track in at least one set of pre-stored feature information templates is used for user identification. Since the feature information is extracted based on the user gesture track and compared with the pre-stored feature information template, the user identity is not recognized. It is limited by the posture of the user holding the terminal device. Users and terminal devices can still identify the user in various postures, such as standing, standing, and lying, improving the accuracy of recognition and user experience.

In one possible design, the feature information includes shape information and speed information, or the feature information includes length information, angle information, and speed information, or the feature information includes shape information, speed information, and acceleration information. .

In a possible design, the trajectory of the user gesture is constructed according to the rotational angular velocity measured by the gyroscope in real time, including:

Calculating a rotation matrix C _{t of the} attitude change of the terminal device from the previous moment to the current moment according to the rotational angular velocity of the current time;

According to the formula _{_{P t = P t - * C}} t calculated three-dimensional coordinates at the current time point of the user's arm P _t, to obtain three-dimensional coordinates of each point of time of the user's arm;

Where P _t - is the three-dimensional coordinate point of the previous moment of the current time, and the three-dimensional coordinate point at which the user gesture starts is the origin.

In a possible design, the segmenting processing of the trajectory includes:

Determining a segmentation point in the trajectory according to the three-dimensional curvature to obtain at least one stroke;

Size normalization and rotation normalization for each stroke.

In one possible design, the size of each stroke is normalized, including:

Dividing the size of each stroke by the length of the trajectory;

The rotation normalizes each stroke, including:

Rotating the axis of each stroke to be parallel to the X axis of the initial absolute coordinate system, the axis of each stroke is a line segment from the starting point to the ending point, the initial absolute coordinate system and the terminal device of the user gesture start time The three coordinate axes of the coordinate system are the same, and the origin of the initial absolute coordinate system is a three-dimensional coordinate point of the position where the user's elbow is located.

In a possible design, the feature information extraction is performed for each stroke, including:

Extracting a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke;

A sequence of first derivative of the three-dimensional position of each stroke of the middle distance interval with respect to time is calculated, and the velocity information of each stroke is obtained.

Calculating a first-order derivative sequence of the three-dimensional position of each stroke of the medium-distance interval with respect to time, and obtaining speed information of each stroke;

The acceleration information of each stroke is calculated according to the speed information of each stroke.

In a possible design, the segmenting processing of the trajectory includes:

Rotating normalization and size normalization of the trajectory;

The segmentation points in the trajectory after the rotation normalization and the size normalization are determined according to the two-dimensional curvature, and at least one stroke is obtained.

In one possible design, the rotation normalization of the trajectory includes:

Determining a two-dimensional coordinate sequence [u(i), v(i)], i=1...N according to three-dimensional coordinate points constituting the trajectory, wherein u(i) is a three-dimensional coordinate point constituting the trajectory projected The value of the Y-axis on the YZ plane, v(i) is the value of the Z-axis projected on the YZ plane of the three-dimensional coordinate point constituting the trajectory, and N is the number of three-dimensional coordinate points constituting the trajectory;

Finding a rotation axis of the minimum moment of inertia of the trajectory, rotating the trajectory to a position where the minimum moment of inertia rotation axis is parallel to the Y axis projected to the Y-Z plane;

Calculating the center of gravity of the track

Calculating the covariance matrix of the trajectory

among them,

Multiplying all three-dimensional coordinate points constituting the trajectory by I;

The normalizing the size of the trajectory includes:

The width W and height H of the trajectory are calculated, u(i) in the two-dimensional coordinate sequence is divided by W, and V(i) in the two-dimensional coordinate sequence is divided by H.

Calculate the length of each stroke and obtain the length information of each stroke;

Calculating an angle between two consecutive strokes, wherein the angle is an angle between respective minimum moments of inertia axes of the two strokes, and obtaining angle information between two consecutive strokes;

In a possible design, the user identity is determined according to the feature information of each of the extracted strokes and the feature information of each of the corresponding strokes in the at least one set of pre-stored feature information templates, including:

For each set of pre-stored feature information templates, calculating a dynamic time-rounded DTW distance of the extracted feature information of each segment of the stroke and the feature information of each segment of the stroke corresponding to one of the pre-stored feature information templates;

Determining whether to accept the user who initiated the user gesture is based on the calculated DTW distance and a preset threshold.

In a possible design, the preset threshold is an average of DTW distances between feature information of each segment of strokes corresponding to all the tracks in a set of pre-stored feature information templates, and all of the set of pre-stored feature information templates. The sum of the standard deviations of the DTW distances between the feature information of each stroke corresponding to the track.

In a possible design, each set of pre-stored feature information templates carries a user identifier.

In a second aspect, the application provides a user identity identification device, including:

a trajectory construction module, configured to construct a trajectory of the user gesture according to a real-time measured rotational angular velocity of the gyroscope in the terminal device, where the trajectory is composed of three-dimensional coordinate points of the user's arm at each moment; a stroke segmentation processing module is configured to The trajectory performs segmentation processing of the stroke; the information extraction module is configured to extract feature information for each segment of the stroke; and the identification module is configured to: according to the extracted feature information of each segment of the stroke and each of the at least one set of pre-stored feature information templates The feature information of a stroke is used for user identification, and each set of pre-stored feature information templates includes feature information of each stroke corresponding to at least one track.

In a possible design, the trajectory building module is specifically configured to:

In a possible design, the stroke segmentation processing module includes:

a first determining unit, configured to determine a segment point in the trajectory according to the three-dimensional curvature, to obtain at least one stroke;

The first normalization unit is used for size normalization and rotation normalization of each stroke.

In a possible design, the first normalization unit is specifically configured to:

Dividing the size of each stroke by the length of the trajectory;

In a possible design, the information extraction module is specifically configured to:

In a possible design, the stroke segmentation processing module includes:

a second normalization unit, configured to perform rotation normalization and size normalization on the trajectory;

And a second determining unit, configured to determine a segmentation point in the trajectory after the rotation normalization and the size normalization according to the two-dimensional curvature, to obtain at least one stroke.

In a possible design, the second normalization unit is specifically configured to:

Calculating the center of gravity of the track

Calculating the covariance matrix of the trajectory

among them,

In a possible design, the identification module is specifically used to:

In a third aspect, the application provides a user identity recognition apparatus, including:

Memory and processor;

The memory is used to store program instructions;

The processor is configured to perform the user identification method in any of the possible aspects of the first aspect and the first aspect.

In a fourth aspect, the present application provides a readable storage medium, where an execution instruction is stored, and when at least one processor of the user identification device executes the execution instruction, the user identification device performs the first aspect and the On the one hand, any possible design of the user identification method.

In a fifth aspect, the present application provides a program product comprising an execution instruction stored in a readable storage medium. At least one processor of the user identification device can read the execution instructions from a readable storage medium, the at least one processor executing the execution instructions such that the user identification device implements the first aspect and any of the possible aspects of the first aspect User identification method.

DRAWINGS

1 is a flowchart of Embodiment 1 of a method for identifying a user of the present application;

2 is a schematic diagram of a training process;

Figure 3 is a schematic diagram of the authentication identification process;

4 is a flowchart of Embodiment 2 of a method for identifying a user of the present application;

FIG. 5 is a flowchart of Embodiment 3 of a method for identifying a user identity according to the present application;

6 is a flowchart of Embodiment 4 of a method for identifying a user identity according to the present application;

FIG. 7 is a schematic structural diagram of Embodiment 1 of a user identity identification apparatus according to the present application;

8 is a schematic structural diagram of Embodiment 2 of a user identity identification apparatus according to the present application;

FIG. 9 is a schematic structural diagram of Embodiment 3 of a user identity identification apparatus according to the present application.

detailed description

The user identification method and device provided by the application can be applied to various terminal devices, such as a mobile phone, a smart watch, an intelligent blood pressure meter, etc., for identifying a user identity, the terminal device does not need a touch screen, and the user holds the terminal device. When the air gesture is initiated, the terminal device can identify the identity of the air gesture initiator. The present application draws the user gesture track by using the gyroscope installed in the terminal device, extracts the feature information based on the user gesture track, and compares it with the pre-stored feature template. The identity of the user is recognized, and the real-time data of the accelerometer and the gyroscope output is directly compared with the pre-stored feature information template data to identify the user identity, which is not limited by the posture of the user holding the terminal device. The user and the terminal device can still recognize the user identity in various postures, such as standing, standing, and lying. The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a flowchart of Embodiment 1 of a user identification method of the present application. As shown in FIG. 1 , the method in this embodiment may include:

S101. Construct a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, and the trajectory is composed of three-dimensional coordinate points of the user arm at each moment.

Specifically, the terminal device is equipped with a gyro sensor, and the user can hold the hand device or the terminal device with the gyroscope on the wrist and then write in the air as an input of the terminal device, and the gyroscope can acquire the hand. The trajectory of each user is different even if they do the same gesture trajectory, because according to the principle of kinesiology, when a person wants to make a gesture, the brain will first segment the gesture, and each segment is an arc. For each segment, ie each arc, each segment is not a perfect arc and has a perfect arc due to the different ways in which the nerves control the muscles, resulting in a normal distribution of velocity from start to finish in each stroke. Personal characteristics, people writing is also based on this principle.

The trajectory of the user gesture is constructed, and the two coordinate systems, the device coordinate system and the initial absolute coordinate system are designed, and the device coordinate system is the coordinate system of the terminal device itself, which is defined by the terminal device itself. The initial absolute coordinate system is the same as the three coordinate axes of the terminal device coordinate system at the start of the user gesture, and the origin of the initial absolute coordinate system is the three-dimensional coordinate point of the position where the user's elbow is located. In order to construct the trajectory of the user's gesture, it is first necessary to model the human arm. The human arm is divided into an upper arm and a forearm, the upper arm connects the elbow and the shoulder, and the forearm connects the wrist and the elbow. When the user writes in the air, the movement is not very large. The main part of the displacement is the movement of the wrist relative to the elbow. This process can be regarded as the rigid body (forearm) moving around the elbow (the fulcrum). The gyroscope records the angular velocity of the rotation, so the motion trajectory can be calculated as long as the length of the rigid body is known.

The trajectory of the user gesture is constructed according to the rotational angular velocity measured by the gyroscope in real time, which may be:

The rotation angular velocity before the current time to the current time point is calculated a time change in posture of the terminal device rotation matrix C _t, according to the formula _{_{P t = P t - * C}} t calculated three-dimensional coordinates at the current time point of the user's arm P _t ,, to obtain the user's arm Three-dimensional coordinate points at each moment. Where P _t - is the three-dimensional coordinate point of the previous moment of the current time, and the three-dimensional coordinate point at which the user gesture starts is the origin.

Under the premise that the user's elbow gesture is not very large, the user's forearm can be regarded as a rigid body moving around the elbow, that is, a straight line moves around the origin in the initial absolute coordinate system. The gyroscope can obtain the angular velocity of the current device rotation, and thus can calculate the rotation matrix of the device posture change from the previous moment to the current moment, and this rotation matrix also describes the posture change of the user's arm. The current rotation matrix C _t is:

Where (ω _x , ω _y , ω _z ) is the rotational angular velocity of the current time t obtained by the gyroscope, C _t− is the rotation matrix of the previous time t ⁻ , and I is the unit matrix.

The three-dimensional coordinate point of each moment of the user's arm is obtained, that is, the trajectory of the user's gesture is obtained

S102: Perform stroke segmentation processing on the trajectory, and perform feature information extraction on each stroke.

Specifically, after obtaining the trajectory of the user's gesture, according to the kinematics principle, when the person writes in the air, the entire trajectory segment is written in order, and each segment is approximated by an arc, due to each person's own The muscles and nerves react differently, so each person draws a stroke and the curve has a difference from the arc in the brain, and the difference is in a normal distribution. Therefore, the trajectory needs to be segmented, and the principle of segmentation is that each pen approximates an arc. Differentiating between different people requires extracting the features hidden in each pen. For a trajectory, the segmentation point is the junction of the two strokes, which is the turning point between the two strokes. According to the shape feature of the trajectory, find the turning point of each pen and segment the stroke. After the segmentation of the strokes, it is considered that the size and speed of the gestures made by the user may not be completely consistent. In this embodiment, the normalization process is performed on the basis of each trajectory, and each of the trajectories is normalized and extracted. Personal characteristics of the pen.

As an implementable manner, the stroke processing is performed on the trajectory, which may specifically be:

The segmentation points in the trajectory are determined according to the three-dimensional curvature, at least one stroke is obtained, and then each of the strokes is subjected to size normalization and rotation normalization.

Wherein, since the curvature of the stroke segment is much larger than the curvature of the peripheral point, in this embodiment, a threshold is set to determine whether the point is a segmentation point. The size of each stroke is normalized, which can be: the size of each stroke is divided by the length of the track. Thus the track length of the entire gesture becomes unity. Rotate the normalization of each stroke, which can be: rotate the axis of each stroke to be parallel to the X axis of the initial absolute coordinate system. The axis of each stroke is a line segment from the starting point to the ending point. The coordinate system is the same as the three coordinate axes of the terminal device coordinate system at the start of the user gesture, and the origin of the initial absolute coordinate system is the three-dimensional coordinate point of the position where the user's elbow is located.

The feature information includes shape information and speed information, or the feature information includes length information, angle information, and speed information, or the feature information includes shape information, speed information, and acceleration information.

1. When the feature information includes shape information and speed information, feature information is extracted for each stroke, which may be:

The three-dimensional coordinate sequence of each stroke is extracted at a medium time interval, and the shape information of each stroke is obtained; the first derivative sequence of the three-dimensional position of each stroke with respect to time is calculated, and the speed information of each stroke is obtained.

For the current location,

For the position of the previous moment, t is the current moment, t ^- is the previous moment, then the speed information

The calculation formula is:

2. When the feature information includes shape information, speed information, and acceleration information, feature information is extracted for each stroke, which may be:

Extracting a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke; calculating a first derivative sequence of the three-dimensional position of each stroke with respect to time, and obtaining speed information of each stroke; according to each stroke The velocity information calculates the acceleration information for each stroke.

For the speed of the current moment,

Speed for the previous moment, acceleration information

The calculation formula is:

3. When the feature information includes length information, angle information, and speed information, feature information is extracted for each stroke, which may be:

Calculate the length of each stroke and obtain the length information of each stroke; calculate the angle between two consecutive strokes, the angle is the angle between the minimum moments of inertia of the two strokes, and obtain the continuous between the two strokes The angle information; the first derivative sequence of the three-dimensional position of each stroke interval with respect to time is calculated, and the velocity information of each stroke is obtained.

As another implementable manner, the stroke processing of the trajectory may be specifically:

Rotate normalization and size normalization of the trajectory; determine the segmentation points in the trajectory after rotation normalization and size normalization according to the two-dimensional curvature, and obtain at least one stroke.

Wherein, the rotation of the trajectory is normalized, which may be:

First, a two-dimensional coordinate sequence [u(i), v(i)], i=1...N is determined according to the three-dimensional coordinate points constituting the trajectory, wherein u(i) is a three-dimensional coordinate point constituting the trajectory projected on the YZ plane The value of the upper Y-axis, v(i) is the value of the Z-axis projected on the YZ plane of the three-dimensional coordinate point constituting the trajectory, and N is the number of three-dimensional coordinate points constituting the trajectory. Because the plane in which people write in the air is parallel to the human body plane in front of the body, that is, the Y-Z plane of the initial absolute coordinate system, the three-dimensional coordinate points are projected on the Y-Z two-dimensional coordinate plane.

Next, the rotation axis of the minimum moment of inertia of the trajectory is searched, and the trajectory is rotated to a position where the rotation axis of the minimum moment of inertia is parallel to the Y axis projected to the Y-Z plane.

Calculate the track center of gravity

Calculate the covariance matrix of the trajectory

among them,

Finally, all the three-dimensional coordinate points that make up the trajectory are multiplied by I.

Wherein, the size of the trajectory is normalized, which may be:

Calculate the width W and height H of the trajectory, divide u(i) in the two-dimensional coordinate sequence by W, and divide V(i) in the two-dimensional coordinate sequence by H.

On the normalized trajectory, according to the curvature parting stroke, the feature information of the stroke is extracted for each stroke.

S103. Perform user identification according to the extracted feature information of each stroke of each stroke and the feature information of each stroke corresponding to one track of the at least one set of pre-stored feature information templates, where each set of pre-stored feature information templates includes each segment corresponding to at least one track. Characteristic information of strokes.

Specifically, the pre-stored feature information template includes feature information of each segment of the stroke corresponding to the at least one track. After obtaining the feature information of the track of the user gesture, the user identity is performed. If the terminal device has not been trained by the user gesture before, the terminal The device will prompt the user to perform multiple trainings (for example, 5 to 10 times). When the user is in the training state, the terminal device extracts the feature information corresponding to each user's gesture track according to the process of extracting the feature information, and stores the information multiple times. The feature information of the gesture track is used as a pre-stored feature information template. When the training is completed, the device can enter the authentication state. A group of pre-stored feature information templates can be used to distinguish different users. When a user uses multiple users, multiple sets of pre-stored feature information templates can be stored. Each group of pre-stored feature information templates can carry user identifiers to distinguish different users.

The user identity is performed according to the extracted feature information of each segment of the stroke and the feature information of each segment of the corresponding segment of the pre-stored feature information template, which may be:

For each set of pre-stored feature information templates, calculate the dynamic time warping (DTW) distance of the feature information of each segment of the extracted strokes and the feature information of each segment of the strokes corresponding to one of the pre-stored feature information templates. A user is determined to accept the user gesture based on the calculated DTW distance and the preset threshold.

Taking the feature information including the shape information and the speed information as an example, after obtaining the feature information of each stroke corresponding to the gesture track, the DTW distance between the shape information and the velocity information between each stroke and the template is calculated separately. For each feature information, since there are multiple strokes, it is necessary to calculate the DTW distance of the gesture track and the corresponding stroke in the template, and add the DTW distances of all the strokes as the DTW distance of one feature of the two tracks. It is worth noting that the two feature information are speed information and shape information, respectively, and their physical units are different. Therefore, when calculating the overall DTW distance, the DTW distances of the two feature information need to be normalized to make their values. The range is between 0-1 and then added. After obtaining the DTW distance between all the gesture trajectories in a set of templates and the currently recognized gesture trajectory, each gesture trajectory template determines whether to accept the current user according to the preset threshold, when more than half of the templates are currently being authenticated. The user is accepted when the trajectory DTW distance is less than a preset threshold. The preset threshold is set, for example, to the average value of the DTW distance between the feature information of each stroke corresponding to all the tracks in the pre-stored feature information template plus one standard deviation.

Taking the feature information including the shape information, the speed information and the acceleration information as an example, after obtaining the feature information of each stroke corresponding to the gesture track, respectively calculating the DTW between the three feature information of each stroke and the three feature information corresponding to the template. The distance is used as the distance in the k nearest neighbor (KNN), and the current specific user is determined according to the KNN (k=3) algorithm.

In this embodiment, the user can start the gesture recognition process of the terminal device in an active or passive manner. After the terminal device prompts the recognition process to start, the user can start to make a gesture, and the terminal device starts to record the gyroscope data in the gesture process. Users can customize the style of gestures, such as five-pointed stars, Chinese characters, and so on. After the user completes the gesture, the recognition may be ended in an active or passive manner. When the device recognizes that the gesture ends, the trajectory construction, personal user feature information extraction and recognition are started, and if the user exists in the database, it is finally displayed on the display screen. Prompt the identified user and accept the user to log in to the terminal device. If the user does not exist in the database, the login is forbidden.

The user identification method provided in this embodiment constructs a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, performs segmentation processing on the trajectory, and extracts feature information for each segment of the stroke, and finally extracts the feature information according to the extraction. The feature information of each stroke of each stroke is identified by the feature information of each stroke corresponding to one track in at least one set of pre-stored feature information templates, because the feature information is extracted on the basis of the user gesture track and the pre-stored feature information template is extracted. The comparison to identify the user's identity is not limited by the user's posture of holding the terminal device. The user and the terminal device can still recognize the user identity in various postures, such as standing, standing, and lying. Improved recognition accuracy and user experience.

The technical solutions of the method embodiment shown in FIG. 1 are described in detail below by using several specific embodiments.

For user identification, there are two processes. One is that the training process stores the feature information of the user's gesture track as a verification template, and the other is to start recognition for the received gesture. Figure 2 is a schematic diagram of the training process, as shown in Figure 2, including:

S201. After determining that the user gesture ends, obtain a rotation angular velocity measured by the gyroscope.

S202. Construct a trajectory of the user gesture according to the rotation angular velocity measured by the gyro, perform segmentation processing on the trajectory, and perform feature information extraction on each stroke.

S203. Store the extracted feature information.

Generally, the terminal device prompts the user to perform multiple trainings (for example, 5 to 10 times), and stores feature information of multiple gesture trajectories as a pre-stored feature information template.

Figure 3 is a schematic diagram of the authentication identification process, as shown in Figure 3, including:

S301. Detect whether the user triggers an authentication process. If it is executed S202.

S302. After determining that the user gesture ends, obtain a rotation angular velocity measured by the gyroscope.

S303. Construct a trajectory of the user gesture according to the rotational angular velocity measured by the gyro, perform segmentation processing on the trajectory, and perform feature information extraction on each stroke.

S304. Perform user identification according to the feature information of each stroke of the extracted stroke and the feature information of each stroke corresponding to one track in the pre-stored feature information template, and determine whether it matches the pre-stored feature information template. If yes, execute S305, otherwise, the process ends.

S305. The authenticated user succeeds and executes the corresponding command.

The user identification method of the present application is described in detail in three different usage scenarios.

The first step is a single user identification scenario. FIG. 4 is a flowchart of Embodiment 2 of the user identification method of the present application. As shown in FIG. 4, the method in this embodiment may include:

S401. Construct a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, and the trajectory is composed of three-dimensional coordinate points of the user's arm at each moment.

Specifically, the rotation matrix C _{t of the} terminal device posture change from the previous moment to the current moment is calculated according to the rotational angular velocity of the current time, and the three-dimensional coordinate point P _{t of the} current moment of the user arm is calculated according to the formula P _t =P _t −*C _t The three-dimensional coordinate point of the user's arm at each moment. Where P _t - is the three-dimensional coordinate point of the previous moment of the current time, and the three-dimensional coordinate point at which the user gesture starts is the origin.

S402. Determine a segmentation point in the trajectory according to the three-dimensional curvature, and obtain at least one stroke.

S403, then size normalize and rotate normalize each stroke.

Specifically, the size of each stroke is divided by the length of the trajectory. Thus the track length of the entire gesture becomes unity. Rotate the axis of each stroke to be parallel to the X axis of the initial absolute coordinate system. The axis of each stroke is a line segment from the starting point to the ending point.

S404: Extract a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke; calculate a first derivative sequence of the three-dimensional position of each stroke with respect to time, and obtain speed information of each stroke.

For the current location,

The calculation formula is:

S405. Perform user identification according to the feature information of each segment of the extracted strokes and the feature information of each segment of the stroke corresponding to each of the pre-stored feature information templates.

Specifically, the DTW distance between the shape information and the velocity information between each stroke and the template is calculated separately. For each feature information, since there are multiple strokes, it is necessary to calculate the DTW distance of the gesture track and the corresponding stroke in the template, and add the DTW distances of all the strokes as the DTW distance of one feature of the two tracks. It is worth noting that the two feature information are speed information and shape information, respectively, and their physical units are different. Therefore, when calculating the overall DTW distance, the DTW distances of the two feature information need to be normalized to make their values. The range is between 0-1 and then added. After obtaining the DTW distance between all the gesture trajectories in a set of templates and the currently recognized gesture trajectory, each gesture trajectory template determines whether to accept the current user according to the preset threshold, when more than half of the templates are currently being authenticated. The user is accepted when the trajectory DTW distance is less than a preset threshold. The preset threshold is set, for example, to the average value of the DTW distance between the feature information of each stroke corresponding to all the tracks in the pre-stored feature information template plus one standard deviation.

This is followed by a multi-user identification scenario, which is how the same device can allow multiple users to identify the user. FIG. 5 is a flowchart of Embodiment 3 of the user identification method of the present application. As shown in FIG. 5, the method in this embodiment may include:

S501. Construct a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, and the trajectory is composed of three-dimensional coordinate points of the user's arm at each moment.

The specific process is the same as that of S401, and will not be described here.

S502: Perform rotation normalization and size normalization on the trajectory.

Wherein, the rotation of the trajectory is normalized, which may be:

Calculate the track center of gravity

Calculate the covariance matrix of the trajectory

among them,

Wherein, the size of the trajectory is normalized, which may be:

S503. Determine a segmentation point in the trajectory after the rotation normalization and the size normalization according to the two-dimensional curvature, to obtain at least one stroke.

S504, calculating the length of each stroke, obtaining the length information of each stroke; calculating the angle between the two consecutive strokes, the angle is the angle between the minimum moments of inertia of the two strokes, and obtaining two consecutive strokes The angle information between each segment; the first derivative sequence of the three-dimensional position of each stroke interval with respect to time is calculated, and the velocity information of each stroke is obtained.

S505. For each group of pre-stored modules in the plurality of sets of pre-stored feature information templates, perform user identification according to the feature information of each segment of the extracted strokes and the feature information of each segment of the stroke corresponding to each of the set of pre-stored feature information templates.

Specifically, the DTW distance between the templates corresponding to each user is calculated according to the length information, the angle information, and the speed information, and finally, the current user is one of the owners and determines the current possession by means of a template majority vote of each user. Which one is it? In this embodiment, each group of pre-stored feature information templates may carry a user identifier for distinguishing different users.

Followed by a multi-user scenario under the shared terminal device, the scenario is a weak security scenario, which aims to distinguish different users by gesture trajectory, for example, distinguishing users who share the blood pressure meter from the home, and when the user uses the blood pressure meter, determine the current user and extract the user. User history. FIG. 6 is a flowchart of Embodiment 4 of the user identification method of the present application. As shown in FIG. 6, the method in this embodiment may include:

S601: Construct a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, and the trajectory is composed of three-dimensional coordinate points of the user arm at each moment.

S602. Determine a segmentation point in the trajectory according to the three-dimensional curvature, and obtain at least one stroke.

S603, then size normalize and rotate normalize each stroke.

S604. Extract a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke; calculate a first derivative sequence of the three-dimensional position of each stroke with respect to time, and obtain speed information of each stroke.

For the current location,

The calculation formula is:

For the speed of the current moment,

Speed for the previous moment, acceleration information

The calculation formula is:

S605. For each group of pre-stored modules in the plurality of sets of pre-stored feature information templates, perform user identification according to the feature information of each segment of the extracted strokes and the feature information of each segment of the stroke corresponding to each of the set of pre-stored feature information templates.

Specifically, calculating a DTW distance of the feature information of each stroke of each set of pre-stored feature information templates and the feature information of the corresponding stroke of the currently recognized gesture, and using the distance as the distance in the KNN classification algorithm, according to the KNN The (k=3) algorithm determines the current specific user.

FIG. 7 is a schematic structural diagram of Embodiment 1 of a user identity identification apparatus according to the present application. The user identity identification apparatus may be implemented as part or all of a terminal device by using software, hardware, or a combination of the two. As shown in FIG. 7, the embodiment of the present invention is as shown in FIG. The apparatus may include: a trajectory construction module 11, a stroke segmentation processing module 12, an information extraction module 13, and an identification module 14, wherein

The trajectory construction module 11 is configured to construct a trajectory of the user gesture according to the rotational angular velocity measured by the gyro in the terminal device in real time, and the trajectory is composed of three-dimensional coordinate points of the user's arm at each moment.

The stroke segmentation processing module 12 is configured to perform stroke segmentation processing on the trajectory.

The information extraction module 13 is configured to perform feature information extraction for each piece of strokes.

The identification module 14 is configured to perform user identification according to the extracted feature information of each segment of the stroke and the feature information of each segment of the stroke corresponding to one of the at least one set of pre-stored feature information templates, where each set of pre-stored feature information templates includes at least one track corresponding Characteristic information of each stroke.

Further, the trajectory construction module 11 is specifically configured to:

A front rotation angular velocity of the current time point is calculated at a time to the current time the posture change of the terminal device rotation matrix C _t, according to the formula _{_{P t = P t - * C}} t calculated three-dimensional coordinates of the points the user's arm at the current time P _t, to obtain the user's arm per A three-dimensional coordinate point at a time. Where P _t - is the three-dimensional coordinate point of the previous moment of the current time, and the three-dimensional coordinate point at which the user gesture starts is the origin.

The device in this embodiment may be used to implement the technical solution of the method embodiment shown in FIG. 1 , and the implementation principle is similar, and details are not described herein again.

The user identity recognition apparatus provided in this embodiment constructs a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in the terminal device in real time, performs segmentation processing on the trajectory of the trajectory, and extracts feature information for each segment of the stroke, and finally extracts according to the feature. The feature information of each stroke of each stroke is identified by the feature information of each stroke corresponding to one track in at least one set of pre-stored feature information templates, because the feature information is extracted on the basis of the user gesture track and the pre-stored feature information template is extracted. The comparison to identify the user's identity is not limited by the user's posture of holding the terminal device. The user and the terminal device can still recognize the user identity in various postures, such as standing, standing, and lying. Improved recognition accuracy and user experience.

FIG. 8 is a schematic structural diagram of Embodiment 2 of the user identity identification apparatus of the present application. As shown in FIG. 8, the apparatus of this embodiment is based on the apparatus structure shown in FIG. 7. Further, the stroke segmentation processing module 12 includes the first a determining unit 121 and a first normalizing unit 122, the first determining unit 121 is configured to determine a segment point in the trajectory according to the three-dimensional curvature, to obtain at least one stroke, and the first normalization unit 122 is configured to perform each stroke Size normalization and rotation normalization.

Further, the first normalization unit 122 is specifically configured to:

Divide the size of each stroke by the length of the trajectory, and rotate the axis of each stroke to be parallel to the X axis of the initial absolute coordinate system. The axis of each stroke is a line segment from the starting point to the ending point, and the initial absolute coordinates. It is the same as the three coordinate axes of the terminal device coordinate system at the start of the user gesture, and the origin of the initial absolute coordinate system is the three-dimensional coordinate point of the position where the user's elbow is located.

Optionally, the information extraction module 13 is specifically configured to:

The three-dimensional coordinate sequence of each stroke is extracted at a medium time interval, and the shape information of each stroke is obtained, and the first derivative sequence of the three-dimensional position of each stroke interval with respect to time is calculated, and the velocity information of each stroke is obtained.

Optionally, the information extraction module 13 is specifically configured to:

Extract the three-dimensional coordinate sequence of each stroke in the middle time interval, obtain the shape information of each stroke, calculate the first derivative sequence of the three-dimensional position of each stroke with respect to time, and obtain the speed information of each stroke, according to each stroke The velocity information calculates the acceleration information for each stroke.

FIG. 9 is a schematic structural diagram of Embodiment 3 of the user identity identification apparatus of the present application. As shown in FIG. 9, the apparatus of this embodiment is based on the apparatus structure shown in FIG. 7, and further, the stroke segmentation processing module 12 includes a second. The normalization unit 123 and the second determining unit 124 are configured to perform rotation normalization and size normalization on the trajectory. The second determining unit 124 is configured to determine a segmentation point in the trajectory of the rotation normalization and the size normalization according to the two-dimensional curvature, to obtain at least one stroke.

Further, the second normalization unit 123 is specifically configured to:

A two-dimensional coordinate sequence [u(i), v(i)], i=1...N is determined according to the three-dimensional coordinate points constituting the trajectory, wherein u(i) is a three-dimensional coordinate point constituting the trajectory projected on the YZ plane The value of the Y-axis, v(i) is the value of the Z-axis projected on the YZ plane of the three-dimensional coordinate point constituting the trajectory, and N is the number of three-dimensional coordinate points constituting the trajectory;

Finding the rotation axis of the minimum moment of inertia of the trajectory, rotating the trajectory to a position where the rotation axis of the minimum moment of inertia is parallel to the Y axis projected to the Y-Z plane;

Calculate the track center of gravity

Calculate the covariance matrix of the trajectory

among them,

Optionally, the information extraction module 13 is specifically configured to:

Calculate the length of each stroke, obtain the length information of each stroke, calculate the angle between two consecutive strokes, and the angle is the angle between the minimum moments of inertia of the two strokes, and obtain the continuous between the two strokes. The angle information is used to calculate a first-order derivative sequence of the three-dimensional position of each stroke of the medium-distance interval with respect to time, and obtain the velocity information of each stroke.

In the above embodiment, the identification module 14 is specifically configured to:

For each set of pre-stored feature information templates, the dynamic time-timed DTW distance of the extracted feature information of each stroke of each stroke and the feature information of each stroke of a set of pre-stored feature information templates is calculated according to the calculated DTW distance. And the preset threshold determines whether the user who initiated the user gesture is accepted. The preset threshold is an average value of the DTW distance between the feature information of each stroke corresponding to all the tracks in the pre-stored feature information template plus a standard deviation.

Further, each set of pre-stored feature information templates carries a user identifier.

The application also provides a user identity recognition device, including: a memory and a processor;

The memory is used to store program instructions;

The processor is configured to execute the user identity identification method in the foregoing method embodiment.

The application further provides a readable storage medium, where the execution instruction is stored, and when the at least one processor of the user identification device executes the execution instruction, the user identification device performs the user identity in the foregoing method embodiment. recognition methods.

The application also provides a program product comprising an execution instruction stored in a readable storage medium. At least one processor of the user identification device can read the execution instruction from a readable storage medium, and the at least one processor executes the execution instruction such that the user identification device implements the user identification method in the above method embodiment.

One of ordinary skill in the art will appreciate that all or part of the steps to implement the various method embodiments described above may be accomplished by hardware associated with the program instructions. The aforementioned program can be stored in a computer readable storage medium. The program, when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Claims

A user identification method, comprising:

Constructing a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in real time in the terminal device, the trajectory being composed of three-dimensional coordinate points of the user's arm at each moment;

Performing segmentation processing on the trajectory and extracting feature information for each stroke;

Performing user identification according to the feature information of each piece of stroke extracted and the feature information of each piece of stroke corresponding to one track in at least one set of pre-stored feature information templates, each set of pre-stored feature information templates including each piece of stroke corresponding to at least one track Feature information.
The method of claim 1 wherein

The feature information includes shape information and speed information, or

The feature information includes length information, angle information, and speed information, or

The feature information includes shape information, speed information, and acceleration information.
The method according to claim 1, wherein the constructing a trajectory of the user gesture according to the rotational angular velocity measured by the gyroscope in real time comprises:

Calculating a rotation matrix C t of the attitude change of the terminal device from the previous moment to the current moment according to the rotational angular velocity of the current time;

According to the formula P t = P t - * C t calculated three-dimensional coordinates at the current time point of the user's arm P t, to obtain three-dimensional coordinates of each point of time of the user's arm;

Where P t - is the three-dimensional coordinate point of the previous moment of the current time, and the three-dimensional coordinate point at which the user gesture starts is the origin.
The method according to claim 3, wherein said performing segmentation processing on said trajectory comprises:

Determining a segmentation point in the trajectory according to the three-dimensional curvature to obtain at least one stroke;

Size normalization and rotation normalization for each stroke.
The method according to claim 4, wherein said normalizing each segment of the stroke comprises:

Dividing the size of each stroke by the length of the trajectory;

The rotation normalizes each stroke, including:

Rotating the axis of each stroke to be parallel to the X axis of the initial absolute coordinate system, the axis of each stroke is a line segment from the starting point to the ending point, the initial absolute coordinate system and the terminal device of the user gesture start time The three coordinate axes of the coordinate system are the same, and the origin of the initial absolute coordinate system is a three-dimensional coordinate point of the position where the user's elbow is located.
The method according to claim 4 or 5, wherein the extracting feature information for each stroke comprises:

Extracting a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke;

A sequence of first derivative of the three-dimensional position of each stroke of the middle distance interval with respect to time is calculated, and the velocity information of each stroke is obtained.
The method according to claim 4 or 5, wherein the extracting feature information for each stroke comprises:

Extracting a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke;

Calculating a first-order derivative sequence of the three-dimensional position of each stroke of the medium-distance interval with respect to time, and obtaining speed information of each stroke;

The acceleration information of each stroke is calculated according to the speed information of each stroke.
The method according to claim 3, wherein said performing segmentation processing on said trajectory comprises:

Rotating normalization and size normalization of the trajectory;

The segmentation points in the trajectory after the rotation normalization and the size normalization are determined according to the two-dimensional curvature, and at least one stroke is obtained.
The method of claim 8 wherein said normalizing said trajectory of rotation comprises:

Determining a two-dimensional coordinate sequence [u(i), v(i)], i=1...N according to three-dimensional coordinate points constituting the trajectory, wherein u(i) is a three-dimensional coordinate point constituting the trajectory projected The value of the Y-axis on the YZ plane, v(i) is the value of the Z-axis projected on the YZ plane of the three-dimensional coordinate point constituting the trajectory, and N is the number of three-dimensional coordinate points constituting the trajectory;

Finding a rotation axis of the minimum moment of inertia of the trajectory, rotating the trajectory to a position where the minimum moment of inertia rotation axis is parallel to the Y axis projected to the Y-Z plane;

Calculating the center of gravity of the track

Calculating the covariance matrix of the trajectory
among them,

Multiplying all three-dimensional coordinate points constituting the trajectory by I;

The normalizing the size of the trajectory includes:

The width W and height H of the trajectory are calculated, u(i) in the two-dimensional coordinate sequence is divided by W, and V(i) in the two-dimensional coordinate sequence is divided by H.
The method according to claim 6 or 7, wherein the extracting feature information for each stroke comprises:

Calculate the length of each stroke and obtain the length information of each stroke;

Calculating an angle between two consecutive strokes, wherein the angle is an angle between respective minimum moments of inertia axes of the two strokes, and obtaining angle information between two consecutive strokes;

A sequence of first derivative of the three-dimensional position of each stroke of the middle distance interval with respect to time is calculated, and the velocity information of each stroke is obtained.
The method according to claim 1, wherein the user identity is determined according to the feature information of each of the extracted strokes and the feature information of each of the corresponding strokes in the at least one set of pre-stored feature information templates, including:

For each set of pre-stored feature information templates, calculating a dynamic time-rounded DTW distance of the extracted feature information of each segment of the stroke and the feature information of each segment of the stroke corresponding to one of the pre-stored feature information templates;

Determining whether to accept the user who initiated the user gesture is based on the calculated DTW distance and a preset threshold.
The method according to claim 11, wherein the preset threshold is an average value of DTW distances between feature information of each segment of strokes corresponding to all tracks in a set of pre-stored feature information templates and the set of pre-stored The sum of the standard deviations of the DTW distances between the feature information of each stroke corresponding to all the tracks in the feature information template.
The method according to claim 1, wherein each of the pre-stored feature information templates carries a user identifier.
A user identity recognition device, comprising:

a trajectory construction module, configured to construct a trajectory of the user gesture according to a real-time measured rotational angular velocity of the gyroscope in the terminal device, where the trajectory is composed of three-dimensional coordinate points of the user's arm at each moment;

a stroke segmentation processing module, configured to perform stroke segmentation processing on the track;

An information extraction module, configured to extract feature information for each stroke;

The identification module is configured to perform user identification according to the extracted feature information of each segment of the stroke and the feature information of each segment of the stroke corresponding to one of the at least one set of pre-stored feature information templates, where each set of pre-stored feature information templates includes at least one track corresponding Characteristic information of each stroke.
The device of claim 14 wherein:

The feature information includes shape information and speed information, or

The feature information includes length information, angle information, and speed information, or

The feature information includes shape information, speed information, and acceleration information.
The device according to claim 14, wherein the trajectory building module is specifically configured to:

Calculating a rotation matrix C t of the attitude change of the terminal device from the previous moment to the current moment according to the rotational angular velocity of the current time;

According to the formula P t = P t - * C t calculated three-dimensional coordinates at the current time point of the user's arm P t, to obtain three-dimensional coordinates of each point of time of the user's arm;

Where P t - is the three-dimensional coordinate point of the previous moment of the current time, and the three-dimensional coordinate point at which the user gesture starts is the origin.
The apparatus according to claim 16, wherein the stroke segmentation processing module comprises:

a first determining unit, configured to determine a segment point in the trajectory according to the three-dimensional curvature, to obtain at least one stroke;

The first normalization unit is used for size normalization and rotation normalization of each stroke.
The device according to claim 17, wherein the first normalization unit is specifically configured to:

Dividing the size of each stroke by the length of the trajectory;

Rotating the axis of each stroke to be parallel to the X axis of the initial absolute coordinate system, the axis of each stroke is a line segment from the starting point to the ending point, the initial absolute coordinate system and the terminal device of the user gesture start time The three coordinate axes of the coordinate system are the same, and the origin of the initial absolute coordinate system is a three-dimensional coordinate point of the position where the user's elbow is located.
The device according to claim 17 or 18, wherein the information extraction module is specifically configured to:

Extracting a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke;

A sequence of first derivative of the three-dimensional position of each stroke of the middle distance interval with respect to time is calculated, and the velocity information of each stroke is obtained.
The apparatus according to claim 17 or 18, wherein said information extraction module is specifically for:

Extracting a three-dimensional coordinate sequence of each stroke of a medium time interval to obtain shape information of each stroke;

Calculating a first-order derivative sequence of the three-dimensional position of each stroke of the medium-distance interval with respect to time, and obtaining speed information of each stroke;

The acceleration information of each stroke is calculated according to the speed information of each stroke.
The apparatus according to claim 16, wherein the stroke segmentation processing module comprises:

a second normalization unit, configured to perform rotation normalization and size normalization on the trajectory;

And a second determining unit, configured to determine, according to the two-dimensional curvature, a segmentation point in the trajectory of the rotation normalization and the size normalization, to obtain at least one stroke.
The device according to claim 21, wherein the second normalization unit is specifically configured to:

Determining a two-dimensional coordinate sequence [u(i), v(i)], i=1...N according to three-dimensional coordinate points constituting the trajectory, wherein u(i) is a three-dimensional coordinate point constituting the trajectory projected The value of the Y-axis on the YZ plane, v(i) is the value of the Z-axis projected on the YZ plane of the three-dimensional coordinate point constituting the trajectory, and N is the number of three-dimensional coordinate points constituting the trajectory;

Finding a rotation axis of the minimum moment of inertia of the trajectory, rotating the trajectory to a position where the minimum moment of inertia rotation axis is parallel to the Y axis projected to the Y-Z plane;

Calculating the center of gravity of the track

Calculating the covariance matrix of the trajectory
among them,

Multiplying all three-dimensional coordinate points constituting the trajectory by I;

The width W and height H of the trajectory are calculated, u(i) in the two-dimensional coordinate sequence is divided by W, and V(i) in the two-dimensional coordinate sequence is divided by H.
The device according to claim 19 or 20, wherein the information extraction module is specifically configured to:

Calculate the length of each stroke and obtain the length information of each stroke;

Calculating an angle between two consecutive strokes, wherein the angle is an angle between respective minimum moments of inertia axes of the two strokes, and obtaining angle information between two consecutive strokes;

A sequence of first derivative of the three-dimensional position of each stroke of the middle distance interval with respect to time is calculated, and the velocity information of each stroke is obtained.
The device according to claim 14, wherein the identification module is specifically configured to:

For each set of pre-stored feature information templates, calculating a dynamic time-rounded DTW distance of the extracted feature information of each segment of the stroke and the feature information of each segment of the stroke corresponding to one of the pre-stored feature information templates;

Determining whether to accept the user who initiated the user gesture is based on the calculated DTW distance and a preset threshold.
The device according to claim 24, wherein the preset threshold is an average value of DTW distances between feature information of each segment of strokes corresponding to all tracks in a set of pre-stored feature information templates and the set of pre-stored The sum of the standard deviations of the DTW distances between the feature information of each stroke corresponding to all the tracks in the feature information template.
The device according to claim 14, wherein each of the pre-stored feature information templates carries a user identifier.