WO2021185110A1

WO2021185110A1 - Method and device for eye tracking calibration

Info

Publication number: WO2021185110A1
Application number: PCT/CN2021/079596
Authority: WO
Inventors: 张朕; 路伟成
Original assignee: 北京七鑫易维信息技术有限公司
Priority date: 2020-03-18
Filing date: 2021-03-08
Publication date: 2021-09-23
Also published as: CN113495613B; JP2023517380A; CN113495613A

Abstract

A method and device for eye tracking calibration, calibration being completed in a scenario applying sight positioning interaction, obviating the need for a separate calibration segment. The method comprises: starting an eye tracking function (S0); if an interaction scheme selected by a user is a non-line-of-sight positioning interaction scheme, starting a background calibration process (S1); during the background calibration process, acquiring ocular feature information of the user (S2); acquiring position coordinates produced by employing the non-line-of-sight positioning interaction scheme for positioning to serve as calibration point coordinates (S3); and calculating to produce a current personal calibration coefficient of the user on the basis of the ocular feature information and of the calibration point coordinates acquired (S4). Hence, in the method, if the user selects the non-line-of-sight positioning interaction scheme for positioning, the background calibration process is started at the same time, during the background calibration process, the position coordinates produced by using the non-line-of-sight positioning interaction scheme for positioning serve as the calibration point coordinates to produce by calculation the personal calibration coefficient. The background calibration process of the method is hidden from the user, and the need to exit from a current scenario is obviated.

Description

Eye tracking calibration method and device

Technical field

The present invention relates to the technical field of eye tracking, in particular to an eye tracking calibration method and device.

Background technique

With the development of science and technology, eye tracking technology has gained more and more widespread applications. For example, it can be used in virtual reality (VR), augmented reality (AR), eye-controlled tablet computers and other terminal devices that involve line-of-sight positioning (referred to as line-of-sight). In the positioning device), the positioning interaction is performed through the line of sight.

Because there are some differences in the physiological structure of each user’s eyes, in the prior art, the user usually needs to enter the calibration process before using the gaze positioning device with eye tracking function. At the same time of multiple calibration points, the user's eye feature information is obtained, and then the user's personal calibration coefficient is obtained by correlation calculation based on the calibration point coordinates and the eye feature information. After the calibration is completed, a point set screen will be displayed, allowing the user to evaluate the calibration effect by himself. If the user subjectively believes that the calibration effect cannot meet the sight positioning requirements, recalibration is required.

After the calibration is completed, it will enter the scene where the line of sight positioning is used. If the user finds that the line of sight is inaccurate during use, or the relative position of the head-mounted display and the eye changes due to adjustment of the head-mounted display position, etc., they need to exit the use The scene re-enters the calibration link, which brings inconvenience to users.

Summary of the invention

At least some embodiments of the present invention provide an eye tracking calibration method and device, so as to complete the calibration in a scene where the gaze positioning interaction is applied, without requiring a separate calibration link.

In one of the embodiments of the present invention, an eye tracking calibration method is provided, including:

In an interactive operation, if the interactive mode selected by the user is a non-line-of-sight interactive mode, start the background calibration process;

In the background calibration process, obtaining the user's eye characteristic information;

Acquiring the position coordinates obtained by positioning in the non-line-of-sight positioning interactive manner as the calibration point coordinates;

According to the acquired eye feature information and calibration point coordinates, the current personal calibration coefficient of the user is calculated.

In an optional embodiment, before starting the background calibration process, the method further includes: acquiring user's eye characteristic information; calculating the user's eye characteristic information and target calibration coefficient according to the acquired eye characteristic information Gaze point coordinates; the target calibration coefficients include: system default calibration coefficients, or personal calibration coefficients associated with the user ID of the user; display the interaction effect of the functional area to which the coordinates of the gaze point belong; if monitoring When the non-line-of-sight positioning interactive device is used and the non-line-of-sight positioning interactive device is positioned to other functional areas, it is determined that the interactive mode used in this interactive operation is the non-line-of-sight positioning mode.

In an optional embodiment, the user ID of the user is a target user ID; before the use of the target calibration coefficient to calculate the gaze point coordinates of the user, the method further includes: if the target user ID is associated with a personal calibration Coefficient, the personal calibration coefficient associated with the target user ID is determined as the target calibration coefficient; if the user ID of the user is not associated with a personal calibration coefficient, the system default calibration coefficient is determined as the target calibration coefficient.

In an optional embodiment, after the current personal calibration coefficient is calculated, the method further includes: if the target user identification has been associated with a personal calibration coefficient, updating the personal calibration coefficient associated with the target user identification to all The current personal calibration coefficient; if the target user identification is not associated with a personal calibration coefficient, the current personal calibration coefficient is associated with the target user identification.

In an optional embodiment, the user identification is bound to the biometric feature; before the use of the target calibration coefficient to calculate the user's gaze point coordinates, the method further includes: acquiring the user's biometric feature; Match the biometric features of the user ID with the biometric features of the established user ID; if the matching is successful, the user ID that is successfully matched is determined as the user ID of the user; if the matching fails, the user ID is created for the user, And bind the established user identification with the acquired biometric features.

In one of the embodiments of the present invention, there is also provided an eye tracking calibration method calibration device, including:

The background calibration start unit is set to start the background calibration process if the interactive mode selected by the user is a non-line-of-sight interactive mode during an interactive operation;

An acquisition unit, configured to acquire the user's eye characteristic information in the background calibration process;

The calibration unit is configured to obtain the position coordinates obtained by the non-line-of-sight positioning interactive method as the calibration point coordinates; and calculate the current personal calibration coefficient of the user according to the obtained eye feature information and the calibration point coordinates.

In an optional embodiment, the above-mentioned device further includes: a calculation unit configured to obtain the user's eye characteristic information before starting the background calibration process; and according to the obtained eye characteristic information and the target calibration coefficient The coordinates of the gaze point of the user are calculated; the target calibration coefficient includes: a system default calibration coefficient, or a personal calibration coefficient associated with the user ID of the user; the display unit is set to check the gaze point The interactive effect of the functional area to which the coordinate belongs is displayed; the monitoring unit is set to determine the interactive operation used in this interactive operation if it is detected that the non-line-of-sight positioning interactive device is used, and when the non-line-of-sight positioning interactive device is positioned to other functional areas The interactive mode is a non-line-of-sight positioning mode.

In an optional embodiment, the user identification of the user is the target user identification; before the gaze point coordinates of the user are calculated using the target calibration coefficient, the calculation unit is further configured to: A personal calibration coefficient is already associated, and the personal calibration coefficient associated with the target user ID is determined as the target calibration coefficient; and if the user’s user ID is not associated with a personal calibration coefficient, the system default calibration coefficient is determined as The target calibration coefficient.

In an optional embodiment, after the current personal calibration coefficient is calculated, the calibration unit is further configured to update the current personal calibration coefficient if the target user identification has been associated with a personal calibration coefficient. The personal calibration coefficient associated with the target user identification; and if the target user identification is not associated with a personal calibration coefficient, the current personal calibration coefficient is associated with the target user identification.

In an optional embodiment, the user identifier is bound to the biometric feature; the device further includes: a biometric unit configured to obtain the coordinates of the gaze point of the user before the user's gaze point coordinates are calculated using the target calibration coefficient The biometric characteristics of the user; matching the obtained biometric characteristics with the biometric characteristics of the established user ID; if the matching is successful, the user ID that is successfully matched is determined as the user ID of the user; and if the matching fails Establish a user identity for the user, and bind the established user identity with the acquired biometric characteristics.

In one of the embodiments of the present invention, there is also provided an eye tracking calibration method and equipment, including: a processor, a memory, and a computer program stored in the memory and running on the processor, and the processor executes the program When realizing the steps of the eye tracking calibration method described in any one of the above.

In one of the embodiments of the present invention, a storage medium is also provided. The storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by a processor, the implementation of any of the above Eye tracking calibration method steps.

In at least some embodiments of the present invention, in an interactive scene using eye tracking, if the user selects the non-line-of-sight positioning interactive mode for positioning, the background calibration process will be initiated at the same time, and the non-line-of-sight positioning interactive mode will be used in the background calibration process The position coordinates obtained by positioning are used as the calibration point coordinates to calculate the personal calibration coefficient. The background calibration process in the embodiment of the present invention is hidden from the user, and the user does not perceive it, nor does it need to exit the current scene to perform calibration again like the prior art, which can save calibration time and improve user experience.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work.

Fig. 1 is a flowchart of an eye tracking calibration method according to one embodiment of the present invention.

Fig. 2a is a schematic diagram of obtaining the coordinates of a gaze point in an interactive way of gaze positioning according to one of the embodiments of the present invention.

Fig. 2b is a schematic diagram of expanding the functional area to which the coordinates of the gaze point belong according to one embodiment of the present invention.

Fig. 3 is a flowchart of another eye tracking calibration method according to one embodiment of the present invention.

Fig. 4 is a flowchart of yet another eye tracking calibration method according to one of the embodiments of the present invention.

Fig. 5 is a flowchart of still another eye tracking calibration method according to one of the embodiments of the present invention.

Fig. 6 is a flowchart of still another eye tracking calibration method according to one of the embodiments of the present invention.

Fig. 7 is a structural block diagram of an eye tracking calibration device according to one of the embodiments of the present invention.

Fig. 8 is a structural block diagram of another eye tracking calibration device according to one embodiment of the present invention.

Fig. 9 is a structural block diagram of still another eye tracking calibration device according to one of the embodiments of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

The terms "first" and "second" in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific sequence. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not set in the listed steps or units, but may include steps or units that are not listed.

The eye tracking calibration method and the eye tracking calibration device provided in the embodiments of the present invention are applied to the field of eye tracking. Eye tracking can also be called gaze tracking, which is a technique for estimating the line of sight and/or gaze point of the eye by measuring eye movement. .

The eye-tracking calibration device involved in the embodiment of the present invention can be deployed in terminal devices related to line-of-sight positioning (referred to as line-of-sight positioning equipment) such as VR systems, AR devices, eye-controlled tablet computers.

Among them, VR systems generally include multiple devices, for example, VR all-in-one machine + handle/remote control, PC side + VR head-mounted display (VR software is installed on the PC side to communicate with the head display) + handle/remote control/mouse , Smart mobile terminal + head-mounted display (VR software is installed on the smart mobile terminal to communicate with the head-mounted display), etc.

The above-mentioned eye tracking calibration device can be deployed in an eye-controlled tablet computer, etc., in order to collect the user's eye diagram, the eye-controlled tablet computer can also be deployed with an infrared light source and set as a camera for taking eye diagrams (such as environmental cameras, infrared cameras) . The eye diagram here refers to an image that includes eyes, such as a frontal or profile portrait, or an image that only includes eyes.

Alternatively, the eye tracking calibration device can be deployed on the aforementioned VR/AR head-mounted display. The head-mounted display is also equipped with a line-of-sight tracking device (or eye tracking device). In addition to the above-mentioned eye tracking device that uses a camera for capturing eye diagrams and an infrared light source, other eye tracking devices, such as MEMS micro-electromechanical systems, can also be used. Specifically, it can include MEMS infrared scanning mirrors, infrared light sources, and infrared receivers, which can detect eye movement by taking eye images; for another example, the gaze tracking device can also be a capacitive sensor, which passes through the capacitance between the eyeball and the capacitor plate. For example, the gaze tracking device can also be a muscle current detector, more specifically, by placing electrodes on the bridge of the nose, forehead, ears or earlobes to detect the pattern of muscle current signals to detect eye movement.

Hereinafter, the embodiments of the present invention will be further described in detail based on the common aspects related to the present invention in the above description.

Example one

In order to solve the existing calibration methods: if the user finds that the line of sight is inaccurate in the scene where the line of sight positioning interaction is applied, or the relative position of the head-mounted display and the eye changes due to the adjustment of the head-mounted display position, etc., you need to Exit the scene recalibration problem.

In the first embodiment of the present application, an eye tracking calibration method is provided to complete the calibration in a scene where the gaze positioning interaction is applied, without requiring a separate calibration link.

Please refer to Figure 1. The eye tracking calibration method includes:

S0: Start the eye tracking function.

In one example, the eye tracking function of the gaze positioning device is turned on by default.

Of course, the eye tracking function can also be activated according to the user's operation.

S1: In an interactive operation, if the interactive mode selected by the user is the non-line-of-sight positioning interactive mode, the background calibration process is started.

An interactive operation can refer to: the user inputs information and operation commands through the input and output system, the system receives post-processing, and displays the processing result through the input and output system.

Follow-up users can further input information and operation commands according to the processing results.

Taking the VR scene as an example, the general default is to obtain the coordinates of the gaze point using the gaze positioning interactive method (see Figure 2a).

Display the interactive effects such as expansion (see Figure 2b), color change, and brightening of the functional area to which the fixation point coordinates belong. The functional areas here include, but are not limited to: icon controls, spatial areas, virtual objects (for example, in a VR game, you can go to a designated spatial area through the line of sight or select the virtual object you want to grab) and so on.

The aforementioned mouse, remote control, handle, and even the keyboard on the PC side can also be used for human-computer interaction. Compared with the line-of-sight interactive mode, these interactive modes can be collectively referred to as the non-line-of-sight positioning interactive mode. Mouse, remote control , Handles, keyboards, etc. can be collectively referred to as non-line-of-sight positioning interactive devices.

If in an interactive operation, the user uses the non-line-of-sight positioning interactive method, then the background calibration process will be entered, which is hidden from the user and the user will not perceive it.

S2: In the background calibration process, obtain the user's eye feature information.

In one example, the eye feature information may include pupil position coordinates, pupil shape, iris position, iris shape, eyelid position, eye corner position, position coordinates of corneal reflection spots, and the like.

Specifically, if the eye tracking calibration device is deployed in a mobile terminal device, the eye diagram can be captured by a camera of the mobile terminal device, etc., and the eye diagram can be processed to obtain eye feature information.

If the eye-tracking calibration device is deployed on a VR/AR head-mounted display, the eye-tracking device on the VR/AR head-mounted display can capture eye images and process the eye images to obtain eye feature information.

S3: Obtain the position coordinates obtained by the non-line-of-sight positioning interactive method as the calibration point coordinates.

Take the handle as an example, if the user uses the handle to move the cursor to a certain position and clicks to confirm, the position coordinates obtained at this time are the calibration point coordinates.

Different from the prior art, the embodiment of the present invention does not show a fixed calibration point, nor does it require the user to focus on the fixed calibration point. It is based on the fact that the user can see it when using non-line-of-sight positioning interactive devices such as handles and mice. Therefore, the position coordinates obtained by positioning using the above-mentioned non-line-of-sight positioning interactive method are the points that the user is staring at.

S4: According to the acquired eye feature information and calibration point coordinates, calculate the user's current personal calibration coefficient.

The personal calibration coefficient is a parameter used in the gaze estimation algorithm to calculate the final result of the gaze, and is related to the user’s pupil radius, corneal curvature, and the angle difference between the visual axis and the optical axis.

The calculation principle of personal calibration coefficient is as follows:

According to the obtained eye feature information and calibration point coordinates, the personal calibration coefficient can be inversely solved by using the line-of-sight estimation algorithm. After the current personal calibration coefficient is obtained, the previous personal calibration coefficient can be overwritten or the default calibration coefficient can be replaced to make the eye tracking accuracy more accurate.

It can be seen that in the embodiment of the present invention, in an interactive scene using eye tracking, if the user selects the non-line-of-sight positioning interactive mode for positioning, the background calibration process will be started at the same time, and the non-line-of-sight positioning interactive mode will be used in the background calibration process. The position coordinates obtained by positioning are used as the calibration point coordinates to calculate the personal calibration coefficient. The background calibration process of the embodiment of the present invention can continuously update the user's personal calibration coefficients, making eye tracking more accurate. In addition, the calibration method is hidden from the user, and the user will not perceive it, and it does not need to be the same as in the prior art. You need to exit the current scene and perform calibration again, which can save calibration time and improve user experience.

Example two

Generally, the eye tracking technology has default calibration coefficients. The default calibration coefficient is the one with higher accuracy when most people use it.

Of course, due to individual differences such as the user's eyeball radius, using the default calibration coefficient may have the possibility of inaccurate positioning, which is also the reason for the calibration to obtain the personal calibration coefficient.

The definable target calibration coefficients include: system default calibration coefficients, or personal calibration coefficients associated with the user's user ID.

At the very beginning, the target calibration coefficient is the system default calibration coefficient. After at least one background calibration, the target calibration coefficient is updated to the latest personal calibration coefficient.

This embodiment will introduce an exemplary flow of eye tracking calibration based on target calibration coefficients in a scenario where the gaze positioning interaction mode is adopted by default. Please refer to FIG. 3, which may include:

S31: Obtain the user's eye feature information, and calculate the user's gaze point coordinates according to the acquired eye feature information and the target calibration coefficient.

For the acquisition method, refer to step S2 of the foregoing first embodiment.

There are many eye tracking algorithms for calculating the coordinates of the gaze point. For example, the eye tracking algorithm based on the shape of the pupil position calculates gaze information based on the principal axis direction of the pupil and the pupil position.

Another example is a certain feature vector fitting method, its working principle is:

Extract the center position of the pupil, the center position of the left eye corner, and the center position of the right eye corner;

Take the center position of the pupil minus the center position of the left eye corner as the feature vector A;

The center position of the pupil minus the center position of the right eye corner is used as the feature vector B.

Construct a mapping function (several equations) between vector A, vector B and the gaze point.

Based on the given eigenvectors A and B, and the known gaze information, a polynomial fitting is performed to obtain the unknown coefficients in the mapping function (the unknown coefficients can be solved in the calibration process).

After obtaining the unknown coefficients, input the currently extracted feature vector into the mapping function to obtain the current gaze point (this process is a tracking process).

Specifically, the feature data may include: pupil position, pupil shape, iris position, iris shape, eyelid position, eye corner position, spot position, and so on.

Different eye tracking algorithms may have different requirements for the types of feature data. For example, the aforementioned eye tracking algorithm based on the shape of the pupil position, the feature data that needs to be extracted include the main axis direction of the pupil, the pupil position, and the long axis of the pupil. Length, pupil minor axis length, etc.

For another example, in the aforementioned feature vector fitting method, the feature data that needs to be extracted include the center position of the pupil, the center position of the left eye corner, and the center position of the right eye corner.

Since there are many types of eye tracking algorithms, I will not repeat them here.

It should be noted that if the user is a new user, or the user ID of the user is not associated with a personal calibration coefficient, the target calibration coefficient is the default standard coefficient.

S32: Display the interactive effect of the functional area to which the coordinates of the fixation point belong.

Specifically, the function area to which the fixation point coordinates belong can be expanded (see Fig. 2b), color change, brightening, and other interactive effects display. The functional area here can be icon controls, space areas, virtual objects, and so on.

S31 to S32 are an interactive operation.

If the user thinks that the functional area displaying the interactive effect is the expected functional area, then follow-up will confirm to enter the functional area by means of continuous gaze, key press, etc.

If the user thinks that the functional area displaying the interactive effect is not the expected functional area (it means that the accuracy of the fixation point coordinate calculation at this time cannot meet the usage requirements), then the mouse, remote control, handle, and even the keyboard on the PC side can be used later. The line of sight locates the interactive device to move the focus on the screen to reposition it.

S33: Judge whether the background calibration start condition is met, if so, go to S34, otherwise, go to S31;

The background calibration start condition may include: detecting that the non-line-of-sight positioning interactive device is used, and positioning the interactive device to other functional areas according to the non-line-of-sight positioning interactive device.

When the background calibration is started, it is determined that the interactive method used in this interactive operation is the non-line-of-sight positioning method, the background calibration process will be executed later, and the latest personal calibration coefficient obtained will be assigned to the target calibration coefficient.

If the background calibration start is not satisfied, it indicates that the current line-of-sight positioning accuracy meets the needs of the user, the target calibration coefficient does not need to be changed, and the step S31 is returned directly.

S34: Start the background calibration process, and calculate the user's current personal calibration coefficient according to the acquired eye feature information and calibration point coordinates.

For specific introduction, please refer to the aforementioned S2-S4, which will not be repeated here.

S35: Use the calculated personal calibration coefficient to update the target calibration coefficient, and return to S31.

After the non-line-of-sight positioning interactive device is used for positioning, it will return to the line-of-sight positioning interactive mode again.

In this embodiment, in every interactive operation based on gaze positioning, the default calibration coefficient or the user's associated personal calibration coefficient is used to calculate the coordinates of the gaze point, and the functional area to which the coordinates of the gaze point belongs is highlighted.

If the highlighted functional area is the functional area expected by the user, the user can confirm to enter the functional area by means of continuous gaze, keystrokes, etc.

If the highlighted functional area is not the functional area expected by the user, the user will use a handle and other non-line-of-sight positioning interactive devices to reposition it, and at the same time, the background calibration will be started. Therefore, in addition to realizing human-computer interaction, this embodiment can also determine whether to start background calibration.

The user ID of the user is mentioned in the second embodiment. For the convenience of addressing, the user ID of the current user can be referred to as the target user ID.

Before using the target calibration coefficient to calculate the coordinates of the user's gaze point, the following steps may be further included:

If the target user ID has been associated with a personal calibration coefficient, the personal calibration coefficient associated with the target user ID is determined as the target calibration coefficient;

If the user ID of the user is not associated with a personal calibration coefficient, the system default calibration coefficient is determined as the target calibration coefficient.

In addition, after calculating the current personal calibration coefficient, you can perform the following operations:

If the target user ID has been associated with a personal calibration coefficient, update the personal calibration coefficient associated with the target user ID to the current personal calibration coefficient;

If the target user ID is not associated with a personal calibration coefficient, associate the current personal calibration coefficient with the target user ID.

The third embodiment will introduce how to perform eye tracking calibration when the target user identification is not associated with the personal calibration coefficient at the beginning. Please refer to Fig. 4, which may include:

S41: The target user ID is not associated with a personal calibration coefficient, and the system default calibration coefficient is determined as the target calibration coefficient.

S42-S45 are similar to the aforementioned S31-S34, and will not be repeated here.

S46: Associate the calculated current personal calibration coefficient with the target user identification.

S47: Use the calculated personal calibration coefficient to update the target calibration coefficient, and return to S41.

This embodiment can be implemented: when the user is not associated with a personal calibration coefficient, the default calibration coefficient is determined as the target calibration coefficient to calculate the fixation point coordinates, and the functional area to which the fixation point coordinates belong is highlighted. If the highlighted functional area is not the functional area expected by the user, the user will use a handle or other non-line-of-sight positioning interactive device to reposition it, and at the same time start the background calibration.

The personal calibration coefficient obtained by the background calibration will be associated with the target user ID, and the personal calibration coefficient will be used in the next interactive operation. In this way, the personal calibration obtained from the previous calculation can be used when the gaze positioning method is used for positioning next time The coefficients calculate the coordinates of the gaze point, which can realize the transition from using the default calibration coefficients to using more accurate personal calibration coefficients, and further provide users with more personalized and accurate gaze positioning services.

Example four

The fourth embodiment will specifically introduce the eye tracking calibration method according to the personal calibration coefficient associated with the target user identification at the beginning. Please refer to Figure 5, which may include:

S51: The target user identification has been associated with a personal calibration coefficient, and the personal calibration coefficient associated with the target user identification is determined as the target calibration coefficient.

S52-S55 are similar to the aforementioned S31-S34, and will not be repeated here.

S56: Update the personal calibration coefficient associated with the target user ID to the current personal calibration coefficient.

S57: Use the calculated personal calibration coefficient to update the target calibration coefficient, and return to S51.

This embodiment can realize that when the user has been associated with a personal calibration coefficient, the personal calibration coefficient associated with the user is determined as the target calibration coefficient, which can provide the user with more personalized and accurate line-of-sight positioning services.

In addition, if the background calibration is activated, the calculated personal calibration coefficient will be associated with the target user ID, and the personal calibration coefficient will be used in the next interactive operation, which can realize the self-adaptive update of the personal calibration coefficient and further provide users with more personality And accurate line-of-sight positioning services.

Example five

The user who uses the sight positioning device can be the same person by default. Or, considering that there may be multiple people using the same device, it is also possible to distinguish different users through technologies such as biometrics.

In specific implementation, the user ID can be bound to the biometric feature. Different biometric features correspond to different user IDs, and different user IDs represent different users, so that different users can be distinguished.

Please refer to Fig. 6, the eye tracking calibration method based on biometric features exemplarily includes the following process:

S60: Obtain the biometric characteristics of the current user;

Exemplary biometric features here may include at least one of iris, fingerprint, voiceprint, and even facial features.

S61: Match the acquired biometric features with the established biometric features of the user identification.

S62: If the matching is successful, determine the user ID that is successfully matched as the user ID of the current user (target user ID);

For example, if the user ID that is successfully matched is 000010, then "000010" is determined as the user ID of the current user.

S63: If the matching fails, establish a user ID (target user ID) for the user, and bind the established user ID with the acquired biometric characteristics.

The user ID that is successfully matched in step S62 or the user ID established in step S63 is the target user ID in the foregoing embodiment.

S64: Determine whether the target user ID has been associated with personal calibration coefficients, if yes, go to S65, otherwise go to S66;

S65: Determine the personal calibration coefficient associated with the target user identification as the target calibration coefficient;

S66: Determine the system default calibration coefficient as the target calibration coefficient.

S67: Use the target calibration coefficient to calculate the coordinates of the user's gaze point.

Specifically, the user's eye feature information is acquired first, and the user's gaze point coordinates are calculated according to the acquired eye feature information and the target calibration coefficient.

For specific introduction, please refer to the previous record in this article, so I won't repeat it here.

S68-S610 are similar to the aforementioned S32-S34, and will not be repeated here.

S611: If the target user ID has been associated with a personal calibration coefficient, update the personal calibration coefficient associated with the target user ID to the current personal calibration coefficient;

S612: If the target user ID is not associated with a personal calibration coefficient, associate the current personal calibration coefficient with the target user ID.

S613: Use the calculated personal calibration coefficient to update the target calibration coefficient, and return to S67.

In this embodiment, after extracting the user’s biometric feature (for example, iris), it will be matched with the feature identification feature of the established user identification to identify whether it is a new user. If it matches the established user identification , The user ID that is successfully matched is determined as the user ID of the current user. In this interactive operation, the gaze point coordinates will be calculated using the personal calibration coefficient of the successfully matched user ID.

If it is a new user, a user ID will be created for it, and in the first interactive operation, the system default calibration coefficient will be used to calculate the fixation point coordinates.

The use of biometrics to distinguish users can help provide personalized and more accurate sight positioning services for different users. Because you can imagine this scenario:

Suppose a mobile device is used by multiple family members. Member A first conducted interactive operations and calculated personal calibration coefficients. After that, member B also uses the mobile device. If it is not distinguished, it directly uses the calculated personal calibration coefficient of member A for calibration. The positioning may not be accurate. If the background calibration is triggered, the calculated personal calibration coefficient is actually member B's.

After that, if member A uses it again, without distinction, the personal calibration coefficient of member B will be used to calculate the fixation point coordinates, and the positioning accuracy will still not be achieved.

However, if different users are distinguished by biometrics, the personal calibration coefficients of different users are no longer mixed together, and personalized gaze positioning services can be provided for different users.

Example Six

This embodiment provides an eye tracking calibration device. Referring to FIG. 7, the device includes:

The background calibration activation unit 1 is set to activate the background calibration process of the calibration unit 3 if the interaction mode selected by the user is a non-line-of-sight positioning interaction mode during an interactive operation;

The acquisition unit 2 is configured to acquire the user's eye characteristic information in the background calibration process;

In the background calibration process, the calibration unit 3 is configured to obtain the position coordinates obtained by the non-line-of-sight positioning interactive method as the calibration point coordinates; and according to the obtained eye feature information and the calibration point coordinates, the user’s current Personal calibration factor.

The eye-tracking calibration device according to the embodiment of the present invention can be deployed in terminal devices related to gaze positioning (gaze positioning equipment for short), such as VR systems, AR devices, and eye-controlled tablet computers, to perform the eye-tracking calibration method.

For specific content and beneficial effects, please refer to the aforementioned record, which will not be repeated here.

In other embodiments, referring to FIG. 8, the eye tracking calibration device in all the above embodiments may further include:

The calculation unit 4 is configured to obtain the user's eye feature information before starting the background calibration process, and calculate the user's gaze point coordinates according to the obtained eye feature information and the target calibration coefficient;

Wherein, the target calibration coefficient includes: a system default calibration coefficient, or a personal calibration coefficient associated with the user ID of the user.

The display unit 5 is set to display the interactive effect of the functional area to which the coordinates of the gaze point belong;

The monitoring unit 6 is configured to determine that the interactive mode used in this interactive operation is the non-line-of-sight positioning mode when it is detected that the non-line-of-sight positioning interactive device is used and is positioned to other functional areas according to the non-line-of-sight positioning interactive device.

The user ID of the above-mentioned user may be referred to as the target user ID.

In other embodiments of the present invention, after the current personal calibration coefficient is calculated, the calibration unit 3 in all the above embodiments may also be configured to update the target using the current personal calibration coefficient if the target user ID has been associated with a personal calibration coefficient. The personal calibration coefficient associated with the user ID; and if the target user ID is not associated with a personal calibration coefficient, associate the current personal calibration coefficient with the target user ID.

In other embodiments of the present invention, after the current personal calibration coefficient is calculated, the calculation unit 4 in all the above embodiments may also be configured to associate the target user identification with the personal calibration coefficient if the target user identification has already been associated with the personal calibration coefficient. The personal calibration coefficient is determined as the target calibration coefficient; and if the user ID of the user is not associated with the personal calibration coefficient, the system default calibration coefficient is determined as the target calibration coefficient.

User identification can be bound to biometric features. In other embodiments of the present invention, referring to FIG. 9, the foregoing apparatus may further include:

The biometric identification unit 7 is configured to obtain the user's biometric characteristics before the user's gaze point coordinates are calculated by using the target calibration coefficient; if the obtained biometric characteristics are matched with the established biometric characteristics of the user identification; if If the matching is successful, the user identification that is successfully matched is determined as the user identification of the user; and if the matching fails, the user identification is established for the user, and the established user identification is bound with the acquired biometric characteristics.

Example Seven

An embodiment of the present invention provides a storage medium that stores computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, the eye tracking calibration method described in any one of the first to sixth embodiments is implemented. step.

Example eight

An embodiment of the present invention provides a processor configured to run a program, wherein the steps of the eye tracking calibration method described in any one of the first to the sixth embodiments are executed when the program is running.

Example 9

An embodiment of the present invention provides a calibration device for an eye tracking calibration method. The device includes a processor, a memory, and a program stored on the memory and running on the processor. When the processor executes the program, any of the first to sixth embodiments can be executed. One of the steps of the eye tracking calibration method.

Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

This application is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of this application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of the processes and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment can be used to generate It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

In a typical configuration, the computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

It should also be noted that the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or equipment including a series of elements not only includes those elements, but also includes Other elements that are not explicitly listed, or also include elements inherent to such processes, methods, commodities, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, commodity or equipment that includes the element.

Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

The above are only examples of the application, and are not used to limit the application. For those skilled in the art, this application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the scope of the claims of this application.

Claims

An eye tracking calibration method, including:

In an interactive operation, if the interactive mode selected by the user is a non-line-of-sight interactive mode, start the background calibration process;

In the background calibration process, obtaining the user's eye characteristic information;

Acquiring the position coordinates obtained by positioning in the non-line-of-sight positioning interactive manner as the calibration point coordinates;

According to the acquired eye feature information and calibration point coordinates, the current personal calibration coefficient of the user is calculated.
The method according to claim 1, wherein before starting the background calibration procedure, the method further comprises:

Obtain the user's eye feature information;

The gaze point coordinates of the user are calculated according to the acquired eye feature information and target calibration coefficients; the target calibration coefficients include: system default calibration coefficients, or those associated with the user ID of the user Personal calibration factor;

Display the interactive effect of the functional area to which the coordinates of the gaze point belong;

If it is detected that the non-line-of-sight positioning interactive device is used and the non-line-of-sight positioning interactive device is positioned to other functional areas, it is determined that the interactive mode used in this interactive operation is the non-line-of-sight positioning mode.
The method according to claim 2, wherein the user identification of the user is the target user identification; before the use of the target calibration coefficient to calculate the coordinates of the gaze point of the user, the method further comprises:

If the target user ID has been associated with a personal calibration coefficient, determining the personal calibration coefficient associated with the target user ID as the target calibration coefficient;

If the target user ID is not associated with a personal calibration coefficient, the system default calibration coefficient is determined as the target calibration coefficient.
The method according to claim 3, wherein, after the current personal calibration coefficient is obtained by calculation, the method further comprises:

If the target user ID has been associated with a personal calibration coefficient, updating the personal calibration coefficient associated with the target user ID to the current personal calibration coefficient;

If the target user identification is not associated with a personal calibration coefficient, the current personal calibration coefficient is associated with the target user identification.
The method according to any one of claims 2-4, wherein the user identification is bound to the biometric feature; before the use of the target calibration coefficient to calculate the coordinates of the user's gaze point, the method further comprises:

Acquiring the biometric characteristics of the user;

Match the acquired biometric features with the biometric features of the established user ID;

If the matching is successful, determine the user ID of the user who is successfully matched as the user ID of the user;

If the matching fails, a user identity is established for the user, and the established user identity is bound to the acquired biometric characteristics.
A calibration device for an eye tracking calibration method, including:

The background calibration start unit is set to start the background calibration process if the interactive mode selected by the user is a non-line-of-sight interactive mode during an interactive operation;

An acquisition unit, configured to acquire the user's eye characteristic information in the background calibration process;

The calibration unit is configured to obtain the position coordinates obtained by the non-line-of-sight positioning interactive method as the calibration point coordinates; and calculate the current personal calibration coefficient of the user according to the obtained eye feature information and the calibration point coordinates.
The device of claim 6, further comprising:

The calculation unit is configured to obtain the user's eye feature information before starting the background calibration process; and calculate the user's gaze point coordinates according to the obtained eye feature information and the target calibration coefficient; The target calibration coefficient includes: a system default calibration coefficient, or a personal calibration coefficient associated with the user ID of the user;

The display unit is set to display the interactive effect of the functional area to which the coordinates of the gaze point belong;

The monitoring unit is configured to determine that the interactive mode used in this interactive operation is the non-line-of-sight positioning mode if it is detected that the non-line-of-sight positioning interactive device is used and the non-line-of-sight positioning interactive device is positioned to other functional areas.
7. The device of claim 7, wherein the user identification of the user is a target user identification; before the use of the target calibration coefficient to calculate the user's gaze point coordinates, the calculation unit is further configured to The target user ID has been associated with a personal calibration coefficient, and the personal calibration coefficient associated with the target user ID is determined as the target calibration coefficient; and if the user ID of the user is not associated with a personal calibration coefficient, the system is calibrated by default The coefficient is determined as the target calibration coefficient.
The device according to claim 8, wherein, after the current personal calibration coefficient is calculated, the calibration unit is further configured to associate the target user identification with the personal calibration coefficient if the target user identification has already been associated with the personal calibration coefficient The personal calibration coefficient of is updated to the current personal calibration coefficient; and if the target user identification is not associated with a personal calibration coefficient, the current personal calibration coefficient is associated with the target user identification.
8. The device according to any one of claims 7-9, wherein the user identification is bound to a biometric feature; the device further comprises: a biometric unit configured to calculate the user’s calibration coefficient using the target calibration coefficient. Before the coordinates of the gaze point, obtain the biometric features of the user; match the obtained biometric features with the biometric features of the established user ID; if the matching is successful, determine the user ID that is successfully matched as the user And if the matching fails, establish a user identity for the user, and bind the established user identity with the acquired biometric characteristics.