WO2020103291A1 - Unlocking method, apparatus and device based on eye movement trajectory, and storage medium - Google Patents

Abstract

Provided are an unlocking method and apparatus based on an eye movement trajectory, a computer device and a storage medium. The method comprises: acquiring an ambient illumination intensity, and selecting infrared rays or visible light according to the ambient illumination intensity to irradiate the eyeballs of a user with light rays; acquiring a reflected light signal formed after the infrared rays or the visible light are reflected by the eyeballs of the user, tracking a moving trajectory of the reflected light signal, and establishing a correlation between the moving trajectory and the position of a display screen; and acquiring, according to the correlation, a strong reflection point formed on the display screen by the reflected light signal, and comparing the strong reflection point with an unlocking password. According to the method, accurate determination of a user identity can be realized.

Description

Unlocking method, device, equipment and storage medium based on eye movement trajectory

This application requires the priority of the Chinese patent application submitted to the China Patent Office on November 20, 2018, with the application number 201811381597.3 and the invention titled "Unlocking method, device, equipment and storage medium based on eye movement trajectory". Incorporated by reference in this application.

Technical field

The present application relates to the technical field of information security, and in particular to an unlocking method, device, device and storage medium based on eye movement trajectories.

Background technique

The password protection of the smart terminal is to use the smart terminal owner to input different forms of security passwords in the process of using the smart terminal to obtain the safe use of the smart terminal. The user needs to input the correct password when using it in order to use the smart terminal normally. Among them, the types of security passwords include digital passwords, nine-grid passwords, voiceprint passwords, fingerprint passwords.

When unlocking the smart terminal, the user is easy to be peeped by others when entering the digital password or the nine-space password to cause the password to leak; the voiceprint password is easy to be imitated by a voice changer, and the recognition accuracy of the voiceprint password is not high; Replaced by fingerprint stickers. Therefore, the above security password protection methods have certain security risks, which may easily cause property damage to the owner of the smart terminal.

In addition, for the user of the transfer device, when using the above security code to unlock, a finger action or sound is required, which is not convenient for operation in special occasions such as meetings. Moreover, when the smart terminal frequently performs voiceprint or fingerprint recognition, it will affect the service life of the processor and other devices.

Summary of the invention

Based on this, it is necessary to provide an unlocking method, device, computer equipment and storage medium based on the eye movement trajectory for the problem that the existing smart terminal unlocking method is low in security and easily used by others.

An unlocking method based on eye movement trajectory includes:

Obtain the ambient light intensity, select infrared or visible light to irradiate the user's eyeball according to the ambient light intensity; obtain the user's eyeball to reflect the infrared light or the visible light to form a reflected light signal, and track the reflection The walking track of the optical signal, establishing the correspondence between the walking track and the position of the display screen;

According to the corresponding relationship, obtain the strong reflection point formed by the reflected light signal on the display screen, and compare the strong reflection point with the unlock password, and if the verification result is consistent, then perform the unlock operation, if inconsistent, proceed Re-emit infrared or visible light to re-enter the password in the eyes of the user, and the unlock password is pre-stored in the database.

An unlocking device based on eye movement trajectory includes the following modules:

The optical signal transmitting module is set to obtain the ambient light intensity, and select infrared or visible light to irradiate the eyes of the user according to the ambient light intensity;

The optical signal tracking module is configured to acquire the infrared light or the visible light reflected by the user's eyeballs to form a reflected light signal, track the walking path of the reflected light signal, and establish a correspondence between the walking path and the position of the display screen relationship;

The data processing module is configured to obtain the strong reflection point formed on the display screen by the reflected light signal according to the corresponding relationship, and compare the strong reflection point with the unlock password, and if the verification result is consistent, proceed If the unlocking operation is inconsistent, re-emit infrared rays or visible light to re-enter the password in the eyes of the user, and the unlocking password is pre-stored in the database.

A computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor causes the processor to perform the above-mentioned unlocking method based on eye movement trajectory A step of.

A storage medium storing computer-readable instructions, when the computer-readable instructions are executed by one or more processors, causes the one or more processors to perform the steps of the above-described unlocking method based on eye movement trajectories.

The above method, device, computer equipment and storage medium for unlocking based on eye movement trajectory include obtaining ambient light intensity, selecting infrared or visible light to irradiate the user's eyeball according to the ambient light intensity; obtaining the user's eyeball alignment After the infrared rays or the visible light is reflected, a reflected light signal is formed, the walking track of the reflected light signal is tracked, and the corresponding relationship between the walking track and the position of the display screen is established; according to the corresponding relationship, the reflected light signal is acquired in The strong reflection point formed on the display screen compares the strong reflection point with the unlock code, if the verification result is consistent, the unlock operation is performed, and if it is inconsistent, re-emission of infrared rays or visible light is re-entered in the user's eyeball Password, the unlock password is pre-stored in the database. This technical solution aims at unlocking based on eye movement trajectories when an intelligent terminal is unlocked and protected, so as to realize accurate judgment of the user's identity, and prevent non-intelligent terminal owners from using security unlocking vulnerabilities to threaten the owner's personal information and property.

BRIEF DESCRIPTION

By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments, and are not considered as limitations to the present application.

FIG. 1 is an overall flowchart of an unlocking method based on eye movement trajectories of this application;

2 is a schematic diagram of a process of establishing a mapping relationship in an unlocking method based on eye movement trajectories of the present application;

FIG. 3 is a structural diagram of unlocking based on eye movement trajectories of the present application.

detailed description

In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be described in further detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

Those skilled in the art can understand that unless specifically stated, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the word "comprising" used in the specification of the present application refers to the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and / or their groups.

FIG. 1 is a flowchart of an unlocking method based on eye movement trajectory in an embodiment of the present application. As shown in FIG. 1, an unlocking method based on eye movement trajectory includes:

S1. Obtain the ambient light intensity, and select infrared or visible light to irradiate the eyes of the user according to the ambient light intensity;

When using eye tracking technology, using only one band of light as the light beam in the eye tracking process will cause light wave interference due to the influence of ambient light, causing the intelligent terminal receiver to misrecognize during the recognition of the eye trajectory and will cause use. When using the eye-tracking technology to unlock, the person cannot proceed normally. Therefore, in this step, light waves of two frequencies are used to track the eye trajectory. Infrared can be used as a light source for eye tracking when the light is sufficient during the day to avoid visible light interference, and visible light is used to increase the eye trajectory when the light is insufficient at night. Tracking accuracy.

S2: Obtain the infrared light or the visible light reflected by the user's eyeball to form a reflected light signal, track the walking track of the reflected light signal, and establish a correspondence between the walking track and the position of the display screen;

Specifically, in the process of converting the optical signal into an electrical signal, the optical signal is normally distributed to establish a histogram, the strongest electrical signal is output according to the position of the maximum value, and is established according to the strongest electrical signal and the physical position of the display screen Mapping relations.

S3. Obtain the strong reflection point formed on the display screen by the reflected light signal according to the corresponding relationship, and compare the strong reflection point with the unlock password. If the verification result is consistent, then perform the unlock operation, which is inconsistent Then, re-emit infrared or visible light to re-enter the password in the eyes of the user, and the unlock password is pre-stored in the database.

Specifically, the position information on the photosensitive panel corresponding to the reflected light signal is compared with the information on each position of the photosensitive panel in the database of the smart terminal. The dimensions of the comparison are the position of the reflected light and the intensity of the reflected light. When the location is consistent with the location of a password in the database, then compare the intensity of the optical signal generated under the light of this band to prevent the position of the optical signal generated by infrared and visible light from being reflected;

In this embodiment, infrared or visible light is used as the emitted light during eye tracking according to different environments, and the infrared or visible light is absorbed and converted into an electrical signal using a photosensitive panel, and then analyzed by the processor and compared with the password in the database to achieve To control the eye tracking process accurately. The above-mentioned eye tracking method can apply the method of eye movement in various environments to unlock the smart terminal, which effectively improves the accuracy of unlocking the eye movement method.

In one embodiment, the acquiring ambient light intensity, and selecting infrared or visible light to irradiate the eye of the user according to the ambient light intensity includes:

Collecting ambient light and photoelectrically converting the ambient light to obtain the ambient light intensity of the ambient light;

Comparing the light intensity with a preset light intensity threshold, if the ambient light intensity is greater than the light intensity threshold, infrared light is used to emit an optical signal to the user's eyeball; the ambient light intensity is less than When the light intensity threshold is used, visible light is used to emit an optical signal to the user's eyeball, and the light intensity threshold is obtained from historical data;

Specifically, a light intensity threshold is set in the database of the smart terminal, usually set to 200lx, when the ambient light is stronger than 200lx, infrared light is used as the emitted light to emit to the user's eyeball, and when the ambient light intensity is less than 200lx, the visible light is used as The emitted light is emitted toward the eyes of the user.

According to the ambient light intensity, an emission angle for emitting visible light or infrared light to the user's eyeball is determined, and visible light or infrared light is emitted to the user according to the emission angle.

Specifically, the emission angle range is generally within the range of 45 ° to 135 °. If the angle of light irradiating the eyeball exceeds this range, scattering will occur and affect the test effect.

In this embodiment, the intensity and angle of incident infrared or visible light are limited by setting a light intensity threshold and a preset angle of illumination, so that the accuracy of the light receiver when receiving the reflected light from the eyeball is improved to operate the eyeball The trajectory makes accurate judgments.

FIG. 2 is a schematic diagram of a process of establishing a mapping relationship in an embodiment of the present application. As shown in the figure, the acquired eyeballs reflect the infrared rays or the visible light to form a reflected light signal, and track the reflection The walking track of the optical signal to establish the correspondence between the walking track and the position of the display screen includes:

S201: Capture the reflected light signal formed by the reflected light of the user's eyeballs, and convert the reflected light signal into a reflected electrical signal according to the position and residence time of the reflected light signal;

Specifically, capturing the reflected light requires a signal receiver on the smart terminal to receive the optical signal. In this embodiment, the signal receiver uses a photosensitive panel provided on the display, and the photosensitive panel is coated with a semiconductor-type photosensitive medium In this kind of medium, the energy level transition will occur under the irradiation of infrared rays or visible light to generate electrical signals. The intensity of the infrared or visible light absorbed by different positions of the photosensitive panel is different and the intensity of the generated electrical signal is also different.

S202. Collect the reflected electrical signals to form an electrical signal induction point;

Specifically, an electrical signal concentrator is provided on the side of the photosensitive panel facing away from the eyeball, and the electrical signal concentrator compares the electrical signal threshold with the electrical signal threshold stored in the database according to the electrical signal strength, when the electrical signal strength is greater than the electrical At the signal threshold, aggregation is performed, otherwise aggregation is not performed. The electrical signal thresholds are two cases corresponding to visible light as the emission light source and infrared light as the emission light source, respectively.

When the position and strength of the electrical signal both correspond to the value of a certain password stored in the database, the position identification of the next primary electrical signal or secondary electrical signal will begin;

When the passwords identified by all the main electrical signals or secondary electrical signals are consistent with the passwords pre-stored in the database, the smart terminal performs the unlocking operation.

Because the photosensitive medium on the photosensitive panel will deteriorate with use, the threshold of the electrical signal needs to be corrected according to the condition of the photosensitive medium on the photosensitive panel. The formula applied is:

erf (x [i]) = a * 2 / π ^1/2 ∫ _x ⁰ e ^- 2tdt,

In the formula, a is the correction factor to correct the error function, and its value range is 0.8 to 1.0. The value of a is used to control the value of the error function to not exceed 5; as the value of the correction factor a of the aging of the photosensitive medium increases, when the correction factor is less than 0.8, the photosensitive panel needs to be replaced to meet the needs of light collection.

S203. Acquire position information of the electrical signal sensing point to obtain a walking track of the reflected light signal;

Specifically, a rectangular coordinate system is established on the screen, and the lower left corner of the screen can be used as the origin of coordinates, and then coordinates can be given to each point on the screen. position.

S204. Record the position information of the sensing point of the electrical signal, and establish a mapping relationship between the sensing information and the physical position coordinates of the display screen.

Specifically, the processor arranges the electrical signals collected by the electrical signal concentrator in a normal manner, and extracts the value of the main electrical signal corresponding to the maximum value as the effective value of eye tracking this time to measure the effect of this reflected light capturing.

If the reflection angle of the eyeball corresponding to the value of the main electrical signal is within the range of 80 ° to 100 °, it means that the reflected light is captured correctly this time;

If the reflection angle corresponding to the value of the main electrical signal is not within the range of 80 ° ～ 100 °, the value of the electrical signal corresponding to the secondary maximum value on the normal distribution diagram is extracted. If the reflection angle corresponding to the auxiliary electrical signal is within 80 ° ～ Within the range of 100 °, the value of the auxiliary electrical signal is used as the basis for judging the reflection position. If the value of the auxiliary electrical signal is still not within the range of 80 ° to 100 °, the smart terminal needs to be checked.

In this embodiment, the optical signal is converted into an electrical signal through the photosensitive panel, and the electrical signal is optimized to improve the efficiency and accuracy of eye tracking.

In one embodiment, the strong reflection point formed on the display screen by the reflected light signal is obtained according to the correspondence, and the strong reflection point is compared with the unlock password. If the verification result is consistent, Then perform the unlocking operation, and if it is inconsistent, re-emit infrared or visible light to re-enter the password in the eyes of the user, and the unlocking password is pre-stored in the database, including:

Obtain the angle between the strong reflection point on the display screen and each point on the curved surface of the user's eyeball according to the corresponding relationship;

Summarize all the included angles and determine whether any of the included angles is less than the preset reflection angle threshold. When all of the included angles are less than the reflection angle threshold, adjust the incident infrared or visible light To the angle of the user's eyeball;

When the angle between the reflection strong point and the eyeball is greater than or equal to the reflection angle threshold, the reflection strong point is compared with the unlock password, and if the verification result is consistent, the unlock operation is performed, and if it is inconsistent, re-emission of infrared rays Or re-enter the password with visible light in the eyes of the user.

Specifically, the reflection angle threshold is an interval threshold. Generally, the range of values under visible light is 60 ° to 120 °, and the range of values under infrared is 70 ° to 110 °. The range of the range of infrared rays is relative to Visible light should be narrow, because infrared light is prone to diffuse reflection and affect the test results.

When the angle between the reflection point and the eyeball is smaller than the reflection angle threshold, adjust the angle at which the infrared or visible light enters the user's eyeball;

When the reflection angle is greater than the reflection angle threshold, the processor checks the reflection point against the password pre-stored in the database of the smart terminal, and if the verification result is consistent, an unlock operation is performed, which is inconsistent Then, re-emit infrared rays or visible light to re-enter the password in the eyes of the user.

In this embodiment, the matching degree between the reflected light and the password is improved by removing the light greater than the reflection angle threshold.

In one embodiment, the recording the position information of the electrical signal sensing point and establishing the mapping relationship between the sensing information and the physical position coordinates of the display screen includes:

A coordinate system is established by taking the center point of the area where the corresponding unlocked number on the display screen is located as the coordinate origin, the left-right direction as the X axis, and the up and down direction as the Y axis, and the distance between two adjacent numbers or patterns is A;

Using the origin of the coordinates as the center of the circle, draw a circle with radius R, where 3 / 4A≤R≤A;

Map the electrical signal sensing points to the pixels on the display, the coordinates of the two pixels farthest from the origin of the coordinates are recorded as (X ₁ , Y ₁ ) and (X ₂ , Y ₂ ) respectively The connection between these two points, the mathematical expression is: y = kx + b, X ₁ ≤ x ≤ X ₂ , Y ₁ ≤ y ≤ Y ₂ ;

If the average distance d of the pixels is [(X ₂ -X ₁ ) ² + (Y ₂ -Y ₁ ) ² ] ^1/2 / 2 is 75% ～ 90% of the radius R, it means that the electrical signal is The corresponding digits match, otherwise the mapping position of the electrical signal needs to be re-analyzed if they do not match.

Specifically, a few explanations on the above calculation method:

(1) When infrared light is used as the incident light, the value of R is close to A, because the longer wavelength of infrared light has a wider range of reflected energy distribution; when the wavelength is longer, the value of R is closer to A;

(2) When visible light is used as the incident light, the value of R is close to 3 / 4A, the wavelength of visible light is shorter than that of infrared, and the energy of the emitted light is relatively concentrated; when the wavelength is shorter, the value of R is closer to 3 / 4A;

(3) For the formula y = kx + b, the value range of k is preferably [1/2, 2], and the slope within this range can effectively improve the accuracy of password recognition.

In one embodiment, the strong reflection point formed on the display screen by the reflected light signal is obtained according to the correspondence, and the strong reflection point is compared with the unlock password. If the verification result is consistent, Then perform an unlocking operation, and if it is inconsistent, re-emit infrared rays or visible light to the user's eyeball and re-enter the password, and the unlocking password is pre-stored in the database. After entering the password, the operation steps when it still cannot be unlocked include:

Obtain user's iris information and extract iris feature points;

Compare the iris feature point with the pre-stored iris image information, and if the comparison is the same, unlock, otherwise lock the screen and upload the operation process to the cloud server.

Specifically, in the process of iris recognition, infrared emitters may be used to obtain eye information through infrared projection, blink recognition, and gaze recognition techniques. The infrared receiver authenticates user information by collecting the user's blinking motion; the iris device can use iris recognition technology for identity authentication. The iris device may include an iris camera, and the iris camera may acquire iris information and the acquired iris information.

In this embodiment, dynamic iris recognition can be used to better record the reflection trajectory of infrared rays during the movement of the eyeball, and to compare with the historical data of the infrared rays reflected by the user's eyes stored in the database.

In one embodiment, the strong reflection point formed on the display screen by the reflected light signal is obtained according to the correspondence, and the strong reflection point is compared with the unlock password. If the verification result is consistent, An unlock operation is performed, and if it is inconsistent, re-emit infrared or visible light is re-entered into the user's eyeball, and the unlock password is pre-stored in the database. The re-emission of infrared or visible light is re-entered into the user's eyeball The operation steps after the password include:

After re-entering, if the password recognized by the strong reflection point is still inconsistent with the pre-stored password, then a lock screen instruction is issued to prohibit the user from performing the unlock operation and send an error message;

Specifically, when unlocking for the second time, the infrared emitter or the visible light emitter emits light beams of different wavelengths to the user's eyeballs as compared with the first unlocking, to prevent erroneous judgment of the reflected light walking path due to surrounding environmental factors, For example, in the first identification, 40 μm of visible light is used, in the second identification, 60 μm of visible light is used, and the length of the light wave used twice is as large as possible; or the first use of visible light as the source of incident light, the second Secondly, infrared light is used as the light source of the incident light, and the difference between the two light sources during the emission process is used for comprehensive identification to avoid errors in the single light source identification process.

If the password recognized by the reflection point again is inconsistent with the pre-stored password stored in the database, the processor issues a lock screen instruction to the display, and the user will not be able to operate the smart terminal;

Receive the error report information, perform error analysis on the error report information, and perform an unlock operation if the error report information is incorrect, otherwise collect face images of the user and collect them in the database.

In this embodiment, the cloud identification of the user's identity effectively avoids the identification error generated by the smart terminal and prevents the user from using the smart terminal.

In one embodiment, an unlocking device based on eye movement trajectories, as shown in FIG. 3, includes:

The optical signal transmitting module is set to obtain the ambient light intensity, and select infrared or visible light to irradiate the eyes of the user according to the ambient light intensity;

The optical signal tracking module is configured to acquire the infrared light or the visible light reflected by the user's eyeballs to form a reflected light signal, track the walking path of the reflected light signal, and establish a correspondence between the walking path and the position of the display screen relationship;

The data processing module is configured to obtain the strong reflection point formed on the display screen by the reflected light signal according to the corresponding relationship, and compare the strong reflection point with the unlock password, and if the verification result is consistent, proceed If the unlocking operation is inconsistent, re-emit infrared rays or visible light to re-enter the password in the eyes of the user, and the unlocking password is pre-stored in the database.

In one embodiment, the optical signal transmission module includes:

The light intensity capturing module is set to collect ambient light and perform photoelectric conversion on the ambient light to obtain the ambient light intensity of the ambient light;

An infrared module, configured to compare the light intensity with a preset light intensity threshold, and if the ambient light intensity is greater than the light intensity threshold, use infrared to emit a light signal to the user's eyeball;

The visible light module is set such that when the ambient light intensity is less than the light intensity threshold, the visible light is used to emit an optical signal to the user's eyeball, and the light intensity threshold is obtained from historical data;

The emission angle module is configured to determine an emission angle for emitting visible light or infrared light to the user's eye according to the ambient light intensity, and emit visible light or infrared light to the user according to the emission angle.

In one embodiment, the optical signal tracking module includes:

The electrical signal module is configured to capture the reflected light signal formed by the reflected light of the user's eyeballs, and convert the reflected light signal into a reflected electrical signal according to the position and residence time generated by the reflected light signal;

An induction point module configured to collect the reflected electrical signals to form an electrical signal induction point;

A walking trajectory module, configured to obtain the position information of the electrical signal sensing point to obtain the walking trajectory of the reflected light signal;

The mapping relationship module is configured to record the position information of the sensing point of the electrical signal and establish a mapping relationship between the sensing information and the physical position coordinates of the display screen.

In one embodiment, the emission angle module includes:

The included angle generating module is configured to obtain the included angle between the strong reflection point on the display screen and each point on the curved surface of the user's eyeball according to the corresponding relationship;

The angle adjustment module is set to summarize all the included angles, and determine whether any of the included angles is less than a preset reflection angle threshold, and adjust when all of the included angles are less than the reflection angle threshold The angle at which the infrared or visible light is incident on the user's eyeball;

The password comparison module is configured to, when the angle between the strong reflection point and the eyeball is greater than or equal to the reflection angle threshold, compare the strong reflection point with the unlock password, and perform an unlock operation if the verification result is consistent , If it is inconsistent, re-emit infrared or visible light to the user's eyeball and re-enter the password.

In one embodiment, the mapping relationship module includes:

The coordinate establishment module is set to establish a coordinate system by using the center point of the area where the corresponding unlocked number on the display screen is located as the coordinate origin, the left and right directions as the X axis, and the up and down directions as the Y axis, for two adjacent numbers or The distance is A; take the origin of the coordinates as the center of the circle and draw a circle with radius R, where 3 / 4A≤R≤A;

The position analysis module is configured to map the electrical signal sensing points to the pixels on the display, and the coordinates of the two pixels farthest from the origin of the coordinates are recorded as (X ₁ , Y ₁ ) and (X ₂ , Y ₂ ) and the connection between these two points, the mathematical expression is: y = kx + b, X ₁ ≤ x ≤ X ₂ , Y ₁ ≤ y ≤ Y ₂ ; if the pixel The average distance d = [(X ₂ -X ₁ ) ² + (Y ₂ -Y ₁ ) ² ] ^1/2 / 2 is 75% ～ 90% of the radius R, which means that the electrical signal matches the corresponding number, Otherwise, if there is no match, the mapping position of the electrical signal needs to be re-analyzed.

In one embodiment, the password comparison module further includes a re-unlocking module, which is configured to obtain the user's iris information when the infrared rays or visible light is re-entered into the user's eyeball and the password is re-entered, but the user still cannot unlock , Extract the iris feature points; compare the iris feature points with the pre-stored iris image information, and if the comparison is the same, unlock, otherwise lock the screen and upload the operation process to the cloud server; the unlock again module includes: :

The extraction module is set to obtain user's iris information and extract iris feature points;

The comparison module is configured to compare the iris feature points with the pre-stored iris image information, and if the comparison is the same, unlock, otherwise lock the screen and upload the operation process to the cloud server.

In one embodiment, the password comparison module further includes an error reporting module configured to issue a lock screen instruction if the password identified by the strong reflection point is still inconsistent with the pre-stored password after re-entry. Prohibit the user from unlocking operation and send error message; receive the error message, perform error analysis on the error message, if the error message is incorrect, perform unlock operation, otherwise collect face image on the user , Income into the database; the error reporting module, including:

The error reporting module is configured to, after re-entering, if the password identified by the strong reflection point is still inconsistent with the pre-stored password, then issue a lock screen instruction to prohibit the user from unlocking and send an error message;

The error reporting and analysis module is configured to receive the error reporting information, perform error analysis on the error reporting information, and perform an unlocking operation if the error reporting information is incorrect, otherwise collect face images of the user and collect them in the database.

In one embodiment, a computer device is proposed. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the computer device The processor executes the steps of the above-mentioned unlocking method based on eye movement trajectory.

In one embodiment, a storage medium storing computer-readable instructions is proposed. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the above-mentioned eye movement trajectory-based Steps of unlocking method. The storage medium may be a non-volatile storage medium.

A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the foregoing embodiments may be completed by instructing relevant hardware through a program. The program may be stored in a computer-readable storage medium, and the storage medium may include: Read only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

The technical features of the above-mentioned embodiments can be combined arbitrarily. To simplify the description, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, All should be considered within the scope of this description.

The above-mentioned embodiments only express some exemplary embodiments of the present application, and their descriptions are more specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which all fall within the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be subject to the appended claims.

Claims (20)

1. An unlocking method based on eye movement trajectory includes:

   Obtain the ambient light intensity, and select infrared or visible light to irradiate the eyes of the user according to the ambient light intensity;

   Acquiring the infrared light or the visible light of the user's eyeball to form a reflected light signal, tracking the walking track of the reflected light signal, and establishing the correspondence between the walking track and the position of the display screen;

   According to the corresponding relationship, obtain the strong reflection point formed by the reflected light signal on the display screen, and compare the strong reflection point with the unlock password, and if the verification result is consistent, then perform the unlock operation, if inconsistent Re-emit infrared or visible light to re-enter the password in the eyes of the user, and the unlock password is pre-stored in the database.
2. The unlocking method based on eye movement trajectory according to claim 1, wherein the acquiring ambient light intensity and selecting infrared or visible light to irradiate the user's eyeball according to the ambient light intensity include:

   Collecting ambient light and photoelectrically converting the ambient light to obtain the ambient light intensity of the ambient light;

   Compare the light intensity with a preset light intensity threshold, and if the ambient light intensity is greater than the light intensity threshold, use infrared to emit a light signal to the user's eyeball;

   When the ambient light intensity is less than the light intensity threshold, visible light is used to emit an optical signal to the user's eyeball, and the light intensity threshold is obtained from historical data;

   According to the ambient light intensity, an emission angle for emitting visible light or infrared light to the user's eyeball is determined, and visible light or infrared light is emitted to the user according to the emission angle.
3. The unlocking method based on eye movement trajectory according to claim 1, wherein the acquiring the eyeballs of the user to reflect the infrared rays or the visible light forms a reflected light signal, and tracks a walking trajectory of the reflected light signal To establish the correspondence between the walking track and the position of the display screen, including:

   Capture the reflected light signal formed by the reflected light of the user's eyeballs, and convert the reflected light signal into a reflected electrical signal according to the position and residence time of the reflected light signal;

   Collect the reflected electrical signals to form electrical signal induction points;

   Acquiring the position information of the sensing point of the electrical signal to obtain the walking trajectory of the reflected light signal;

   Record the position information of the sensing point of the electrical signal, and establish a mapping relationship between the sensing information and the physical position coordinates of the display screen.
4. The unlocking method based on eye movement trajectory according to claim 1, wherein, according to the corresponding relationship, acquiring the strong reflection point formed on the display screen by the reflected light signal, and comparing the strong reflection point with The unlock passwords are compared. If the verification results are consistent, the unlock operation is performed. If they are inconsistent, re-emit infrared or visible light to reenter the password in the eyes of the user. The unlock password is pre-stored in the database, including

   Obtain the angle between the strong reflection point on the display screen and each point on the curved surface of the user's eyeball according to the corresponding relationship;

   Summarize all the included angles and determine whether any of the included angles is less than the preset reflection angle threshold. When all of the included angles are less than the reflection angle threshold, adjust the incident infrared or visible light To the angle of the user's eyeball;

   When the angle between the reflection strong point and the eyeball is greater than or equal to the reflection angle threshold, the reflection strong point is compared with the unlock password, and if the verification result is consistent, the unlock operation is performed, and if it is inconsistent, re-emission of infrared rays Or re-enter the password with visible light in the eyes of the user.
5. The unlocking method based on eye movement trajectory according to claim 3, wherein the recording of the position information of the sensing point of the electrical signal to establish a mapping relationship between the sensing information and the physical position coordinates of the display screen includes:

   A coordinate system is established by taking the center point of the area where the corresponding unlocked number on the display screen is located as the coordinate origin, the left-right direction as the X axis, and the up and down direction as the Y axis, and the distance between two adjacent numbers or patterns is A;

   Using the origin of the coordinates as the center of the circle, draw a circle with radius R, where 3 / 4A≤R≤A;

   Map the electrical signal sensing points to the pixels on the display, the coordinates of the two pixels farthest from the origin of the coordinates are recorded as (X ₁ , Y ₁ ) and (X ₂ , Y ₂ ) respectively The connection between these two points, the mathematical expression is: y = kx + b, X ₁ ≤ x ≤ X ₂ , Y ₁ ≤ y ≤ Y ₂ ;

   If the average distance d of the pixels is [(X ₂ -X ₁ ) ² + (Y ₂ -Y ₁ ) ² ] ^1/2 / 2 is 75% ～ 90% of the radius R, it means that the electrical signal is The corresponding digits match, otherwise the mapping position of the electrical signal needs to be re-analyzed if they do not match.
6. The unlocking method based on eye movement trajectory according to claim 1, wherein, according to the corresponding relationship, acquiring the strong reflection point formed on the display screen by the reflected light signal, and comparing the strong reflection point with The unlock passwords are compared. If the verification results are consistent, the unlock operation is performed. If they are inconsistent, re-emit infrared or visible light to re-enter the password in the eyes of the user. The unlock password is pre-stored in the database and included in the re-transmission The operation steps when infrared or visible light cannot be unlocked after re-entering the password in the eyes of the user include:

   Obtain user's iris information and extract iris feature points;

   Compare the iris feature point with the pre-stored iris image information, and if the comparison is the same, unlock, otherwise lock the screen and upload the operation process to the cloud server.
7. The unlocking method based on eye movement trajectory according to claim 1, wherein, according to the corresponding relationship, acquiring the strong reflection point formed on the display screen by the reflected light signal, and comparing the strong reflection point with The unlock passwords are compared. If the verification results are consistent, the unlock operation is performed. If they are inconsistent, re-emit infrared rays or visible light to re-enter the password in the eyes of the user. Or the visible light operation steps after the user reenters the password, specifically including:

   After re-entering, if the password recognized by the strong reflection point is still inconsistent with the pre-stored password, then a lock screen instruction is issued to prohibit the user from performing the unlock operation and send an error message;

   Receive the error report information, perform error analysis on the error report information, and perform an unlock operation if the error report information is incorrect, otherwise collect face images of the user and collect them in the database.
8. A device based on eye movement trajectory, which includes the following modules:

   The optical signal transmitting module is set to obtain the ambient light intensity, and select infrared or visible light to irradiate the eyes of the user according to the ambient light intensity;

   The optical signal tracking module is configured to acquire the infrared light or the visible light reflected by the user's eyeballs to form a reflected light signal, track the walking path of the reflected light signal, and establish a correspondence between the walking path and the position of the display screen relationship;

   The data processing module is configured to obtain the strong reflection point formed on the display screen by the reflected light signal according to the corresponding relationship, and compare the strong reflection point with the unlock password, and if the verification result is consistent, proceed If the unlocking operation is inconsistent, re-emit infrared rays or visible light to re-enter the password in the eyes of the user, and the unlocking password is pre-stored in the database.
9. The device for eye movement track according to claim 8, wherein the optical signal transmitting module comprises:

   The light intensity capturing module is set to collect ambient light and perform photoelectric conversion on the ambient light to obtain the ambient light intensity of the ambient light;

   An infrared module, configured to compare the light intensity with a preset light intensity threshold, and if the ambient light intensity is greater than the light intensity threshold, use infrared to emit a light signal to the user's eyeball;

   The visible light module is set such that when the ambient light intensity is less than the light intensity threshold, the visible light is used to emit an optical signal to the user's eyeball, and the light intensity threshold is obtained from historical data;

   The emission angle module is configured to determine an emission angle to emit visible light or infrared rays to the user's eye according to the ambient light intensity, and emit visible light or infrared rays to the user according to the emission angle.
10. The device for eye movement track according to claim 8, wherein the optical signal tracking module comprises:

    The electrical signal module is configured to capture the reflected light signal formed by the reflected light of the user's eyeballs, and convert the reflected light signal into a reflected electrical signal according to the position and residence time generated by the reflected light signal;

    An induction point module configured to collect the reflected electrical signals to form an electrical signal induction point;

    A walking trajectory module, configured to obtain the position information of the electrical signal sensing point to obtain the walking trajectory of the reflected light signal;

    The mapping relationship module is configured to record the position information of the sensing point of the electrical signal and establish a mapping relationship between the sensing information and the physical position coordinates of the display screen.
11. The device for eye movement track according to claim 8, wherein the launch angle module comprises:

    The included angle generating module is configured to obtain the included angle between the strong reflection point on the display screen and each point on the curved surface of the user's eyeball according to the corresponding relationship;

    The angle adjustment module is set to summarize all the included angles, and determine whether any of the included angles is less than a preset reflection angle threshold, and adjust when all of the included angles are less than the reflection angle threshold The angle at which the infrared or visible light is incident on the user's eyeball;

    The password comparison module is configured to, when the angle between the strong reflection point and the eyeball is greater than or equal to the reflection angle threshold, compare the strong reflection point with the unlock password, and perform an unlock operation if the verification result is consistent , If it is inconsistent, re-emit infrared or visible light to the user's eyeball and re-enter the password.
12. The device for eye movement track according to claim 10, wherein the mapping relationship module comprises:

    The coordinate establishment module is set to establish a coordinate system by using the center point of the area where the corresponding unlocked number on the display screen is located as the coordinate origin, the left and right directions as the X axis, and the up and down directions as the Y axis, for two adjacent numbers or The distance is A; take the origin of the coordinates as the center of the circle and draw a circle with radius R, where 3 / 4A≤R≤A;

    The position analysis module is configured to map the electrical signal sensing points to the pixels on the display, and the coordinates of the two pixels farthest from the origin of the coordinates are recorded as (X ₁ , Y ₁ ) and (X ₂ , Y ₂ ) and the connection between these two points, the mathematical expression is: y = kx + b, X ₁ ≤ x ≤ X ₂ , Y ₁ ≤ y ≤ Y ₂ ; if the pixel The average distance d = [(X ₂ -X ₁ ) ² + (Y ₂ -Y ₁ ) ² ] ^1/2 / 2 is 75% ～ 90% of the radius R, which means that the electrical signal matches the corresponding number, Otherwise, if there is no match, the mapping position of the electrical signal needs to be re-analyzed.
13. The device for eye movement trajectory according to claim 11, wherein the password comparison module further includes a re-unlocking module configured to re-emit infrared or visible light to the user's eyeball and re-enter the password, but still cannot When unlocking, obtain the user's iris information, extract the iris feature points; compare the iris feature points with the pre-stored iris image information, and if the comparison is the same, unlock, otherwise lock the screen and upload the operation process to the cloud Server; the unlocking module again includes:

    The extraction module is set to obtain user's iris information and extract iris feature points;

    The comparison module is configured to compare the iris feature points with the pre-stored iris image information, and if the comparison is the same, unlock, otherwise lock the screen and upload the operation process to the cloud server.
14. The device for eye movement trajectory according to claim 11, wherein the password comparison module further includes an error reporting module configured to, after re-entry, if the password recognized by the strong reflection point is still If the passwords are inconsistent, issue a lock screen instruction to prohibit the user from unlocking and send error messages; receive the error messages, perform error analysis on the error messages, and perform unlock operations if the error messages are incorrect, Otherwise, the user's face images are collected and collected into the database; the error reporting module includes:

    The error reporting module is configured to, after re-entering, if the password identified by the strong reflection point is still inconsistent with the pre-stored password, then issue a lock screen instruction to prohibit the user from unlocking and send an error message;

    The error reporting and analysis module is configured to receive the error reporting information, perform error analysis on the error reporting information, and perform an unlocking operation if the error reporting information is incorrect, otherwise collect face images of the user and collect them in the database.
15. A computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor is caused to perform the following steps:

    Obtain the ambient light intensity, and select infrared or visible light to irradiate the eyes of the user according to the ambient light intensity;

    Acquiring the infrared light or the visible light of the user's eyeball to form a reflected light signal, tracking the walking track of the reflected light signal, and establishing the correspondence between the walking track and the position of the display screen;

    According to the corresponding relationship, obtain the strong reflection point formed by the reflected light signal on the display screen, and compare the strong reflection point with the unlock password, if the verification result is consistent, then perform the unlock operation, if not consistent, proceed Re-emit infrared or visible light to re-enter the password in the eyes of the user, and the unlock password is pre-stored in the database.
16. A storage medium storing computer-readable instructions, which when executed by one or more processors, causes the one or more processors to perform the following steps:

    Obtain the ambient light intensity, and select infrared or visible light to irradiate the eyes of the user according to the ambient light intensity;

    Acquiring the infrared light or the visible light of the user's eyeball to form a reflected light signal, tracking the walking track of the reflected light signal, and establishing the correspondence between the walking track and the position of the display screen;

    According to the corresponding relationship, obtain the strong reflection point formed by the reflected light signal on the display screen, and compare the strong reflection point with the unlock password, and if the verification result is consistent, then perform the unlock operation, if inconsistent Re-emit infrared or visible light to re-enter the password in the eyes of the user, and the unlock password is pre-stored in the database.
17. A storage medium storing computer readable instructions according to claim 16, wherein said acquiring ambient light intensity and selecting infrared or visible light to irradiate the eyes of the user according to the ambient light intensity, such that all The processor performs the following steps:

    Collecting ambient light and photoelectrically converting the ambient light to obtain the ambient light intensity of the ambient light;

    Compare the light intensity with a preset light intensity threshold, and if the ambient light intensity is greater than the light intensity threshold, use infrared to emit a light signal to the user's eyeball;

    When the ambient light intensity is less than the light intensity threshold, visible light is used to emit an optical signal to the user's eyeball, and the light intensity threshold is obtained from historical data;

    According to the ambient light intensity, an emission angle for emitting visible light or infrared light to the user's eyeball is determined, and visible light or infrared light is emitted to the user according to the emission angle.
18. A storage medium storing computer readable instructions according to claim 16, wherein said acquiring said user's eyeball reflects said infrared rays or said visible light to form a reflected light signal to track said reflected light When the walking path of the signal establishes the correspondence between the walking path and the position of the display screen, the processor is caused to perform the following steps:

    Capture the reflected light signal formed by the reflected light of the user's eyeballs, and convert the reflected light signal into a reflected electrical signal according to the position and residence time of the reflected light signal;

    Collect the reflected electrical signals to form electrical signal induction points;

    Acquiring the position information of the sensing point of the electrical signal to obtain the walking trajectory of the reflected light signal;

    Record the position information of the sensing point of the electrical signal, and establish a mapping relationship between the sensing information and the physical position coordinates of the display screen.
19. A storage medium storing computer readable instructions according to claim 16, wherein, according to the corresponding relationship, the strong reflection point formed on the display screen by the reflected light signal is acquired, and the The strong reflection point is compared with the unlock password. If the verification result is consistent, the unlock operation is performed. If they are inconsistent, re-emit infrared or visible light to re-enter the password in the eyes of the user. When the unlock password is pre-stored in the database, the The processor performs the following steps:

    Obtain the angle between the strong reflection point on the display screen and each point on the curved surface of the user's eyeball according to the corresponding relationship;

    Summarize all the included angles and determine whether any of the included angles is less than the preset reflection angle threshold. When all of the included angles are less than the reflection angle threshold, adjust the incident infrared or visible light To the angle of the user's eyeball;

    When the angle between the reflection strong point and the eyeball is greater than or equal to the reflection angle threshold, the reflection strong point is compared with the unlock password, and if the verification result is consistent, the unlock operation is performed, and if it is inconsistent, re-emission of infrared rays Or re-enter the password with visible light in the eyes of the user.
20. A storage medium storing computer readable instructions according to claim 18, wherein, when the position information of the electrical signal sensing point is recorded, the mapping relationship between the sensing information and the physical position coordinates of the display screen is established, Causing the processor to perform the following steps:

    A coordinate system is established by taking the center point of the area where the corresponding unlocked number on the display screen is located as the coordinate origin, the left-right direction as the X axis, and the up and down direction as the Y axis, and the distance between two adjacent numbers or patterns is A;

    Using the origin of the coordinates as the center of the circle, draw a circle with radius R, where 3 / 4A≤R≤A;

    Map the electrical signal sensing points to the pixels on the display, the coordinates of the two pixels farthest from the origin of the coordinates are recorded as (X ₁ , Y ₁ ) and (X ₂ , Y ₂ ) respectively The connection between these two points, the mathematical expression is: y = kx + b, X ₁ ≤ x ≤ X ₂ , Y ₁ ≤ y ≤ Y ₂ ;

    If the average distance d of the pixels is [(X ₂ -X ₁ ) ² + (Y ₂ -Y ₁ ) ² ] ^1/2 / 2 is 75% ～ 90% of the radius R, it means that the electrical signal is The corresponding digits match, otherwise the mapping position of the electrical signal needs to be re-analyzed if they do not match.

Images