CN113253829B

CN113253829B - Eyeball tracking calibration method and related product

Info

Publication number: CN113253829B
Application number: CN202010085178.6A
Authority: CN
Inventors: 吴集; 陈岩; 方攀; 王文东
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2020-02-10
Filing date: 2020-02-10
Publication date: 2022-05-06
Anticipated expiration: 2040-02-10
Also published as: CN113253829A

Abstract

The embodiment of the application discloses an eyeball tracking calibration method and a related product, which are applied to electronic equipment, wherein the electronic equipment comprises an eyeball tracking application, an eyeball tracking service, an eyeball tracking algorithm module and an eyeball tracking calibration module, and the method comprises the following steps: the eyeball tracking application displays the target calibration points in the display page and controls the target calibration points to move according to a preset track; the eyeball tracking service transmits request data to the camera and receives image data transmitted by the camera; the eyeball tracking algorithm module determines an attention position in a display page corresponding to an eyeball and a calibration point position of a target calibration point according to image data to obtain a calibration data set, wherein the calibration data set comprises a plurality of position relation pairs, and each position relation pair corresponds to one attention position and one calibration point position; the eyeball tracking calibration module calibrates a preset eyeball tracking algorithm according to the multiple groups of position relations to obtain a target calibration parameter set. By adopting the embodiment of the application, the eyeball tracking precision can be improved.

Description

Eyeball tracking calibration method and related product

Technical Field

The application relates to the technical field of electronics, in particular to an eyeball tracking calibration method and a related product.

Background

With the widespread use of electronic devices (such as mobile phones, tablet computers, and the like), the electronic devices have more and more applications and more powerful functions, and the electronic devices are developed towards diversification and personalization, and become indispensable electronic products in the life of users.

In an eyeball tracking system, in order to ensure the accuracy of eyeball tracking in the use process, calibration needs to be performed first, and a general calibration method is to preset a plurality of points in a screen, so that a user can watch the points, and then, an image is acquired for calibration. Because the user only watches a plurality of preset points in the calibration process, the continuity is lacked, the calibration precision is greatly influenced, and a single independent point cannot better attract the attention of the user, the user can be distracted in the calibration process, thereby influencing the calibration precision.

Disclosure of Invention

The embodiment of the application provides an eyeball tracking calibration method and a related product, which can improve the calibration precision of an eyeball tracking technology and are beneficial to improving the eyeball tracking precision.

In a first aspect, an embodiment of the present application provides an eyeball tracking calibration method, which is applied to an electronic device, where the electronic device includes an eyeball tracking application, an eyeball tracking service, an eyeball tracking algorithm module, and an eyeball tracking calibration module, and the method includes:

the eyeball tracking application displays a target calibration point in a display page, wherein the target calibration point is used for guiding eyeballs of a target object to pay attention to the target calibration point;

the eyeball tracking application controls the target calibration point to move according to a preset track and requests the attention position of the eyeball to the eyeball tracking service;

the eyeball tracking service sends request data to a camera and receives image data sent by the camera;

the eyeball tracking algorithm module determines the attention position of the eyeball corresponding to the display page and the calibration point position of the target calibration point according to the image data to obtain a calibration data set, wherein the calibration data set comprises a plurality of groups of position relation pairs, and each position relation pair corresponds to one attention position and one calibration point position;

and the eyeball tracking calibration module calibrates a preset eyeball tracking algorithm according to the plurality of groups of position relations to obtain a target calibration parameter set, wherein the target calibration parameter set is used for realizing the next eyeball tracking operation of the preset eyeball tracking algorithm.

In a second aspect, an embodiment of the present application provides an eyeball tracking calibration apparatus, which is applied to an electronic device, where the electronic device includes an eyeball tracking application, an eyeball tracking service, an eyeball tracking algorithm module, and an eyeball tracking calibration module, and the eyeball tracking calibration apparatus includes:

the eyeball tracking application is used for displaying a target calibration point in a display page, and the target calibration point is used for guiding eyeballs of a target object to pay attention to the target calibration point;

the eyeball tracking application is used for controlling the target calibration point to move according to a preset track and requesting the attention position of the eyeball to the eyeball tracking service;

the eyeball tracking service is used for sending request data to the camera and receiving image data sent by the camera;

the eyeball tracking algorithm module is used for determining an attention position of the eyeball corresponding to the display page and a calibration point position of the target calibration point according to the image data to obtain a calibration data set, wherein the calibration data set comprises a plurality of groups of position relation pairs, and each position relation pair corresponds to one attention position and one calibration point position;

the eyeball tracking calibration module is used for calibrating a preset eyeball tracking calibration algorithm according to the plurality of groups of position relations to obtain a target calibration parameter set, and the target calibration parameter set is used for realizing the next calibration operation of the preset eyeball tracking calibration algorithm.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the first aspect of the embodiment of the present application.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program enables a computer to perform some or all of the steps described in the first aspect of the embodiment of the present application.

In a fifth aspect, embodiments of the present application provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, where the computer program is operable to cause a computer to perform some or all of the steps as described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

The embodiment of the application has the following beneficial effects:

it can be seen that the eyeball tracking calibration method and the related product described in the embodiments of the present application are applied to an electronic device, the electronic device includes an eyeball tracking application, an eyeball tracking service, an eyeball tracking algorithm module, and an eyeball tracking calibration module, the eyeball tracking application displays target calibration points in a display page, the target calibration points are used for guiding eyeballs of a target object to focus on the target calibration points, the eyeball tracking application controls the target calibration points to move according to a preset track and requests the eyeball tracking service for eyeball focus positions, the eyeball tracking service sends request data to a camera and receives image data sent by the camera, the eyeball tracking algorithm module determines focus positions in the display page corresponding to the eyeballs and calibration point positions of the target calibration points according to the image data to obtain a calibration data set, the calibration data set includes a plurality of sets of position relationship pairs, each position relation pair corresponds to one attention position and one calibration point position, the eyeball tracking calibration module calibrates the preset eyeball tracking algorithm according to the multiple groups of position relations to obtain a target calibration parameter set, and the target calibration parameter set is used for realizing the next eyeball tracking operation of the preset eyeball tracking algorithm.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;

fig. 1B is a schematic structural diagram of an eye tracking system of the electronic device shown in fig. 1A according to an embodiment of the present disclosure;

fig. 1C is a schematic flowchart of an eye tracking calibration method according to an embodiment of the present application;

fig. 1D is a schematic diagram illustrating a preset trajectory provided in an embodiment of the present application;

fig. 1E is a schematic diagram illustrating another preset trajectory provided in the embodiment of the present application;

FIG. 1F is a schematic flowchart of another method for calibrating eye tracking according to an embodiment of the present disclosure;

FIG. 2 is a schematic flowchart of another method for calibrating eye tracking provided by an embodiment of the present application;

fig. 3 is a schematic structural diagram of another electronic device provided in an embodiment of the present application;

fig. 4 is a block diagram illustrating functional units of an eye tracking calibration apparatus according to an embodiment of the present disclosure.

Detailed Description

In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein may be combined with other embodiments.

The electronic device related to the embodiment of the present application may include various handheld devices (smart phones, tablet computers, etc.) having wireless communication functions, vehicle-mounted devices (navigators, vehicle-mounted refrigerators, vehicle-mounted dust collectors, etc.), wearable devices (smart watches, smart bracelets, wireless headsets, augmented reality/virtual reality devices, smart glasses), computing devices or other processing devices connected to wireless modems, and various forms of User Equipment (UE), Mobile Stations (MS), terminal devices (terminal device), and the like. For convenience of description, the above-mentioned devices are collectively referred to as electronic devices.

The following describes embodiments of the present application in detail.

As shown in fig. 1A, fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device includes a processor, a Memory, a signal processor, a communication interface, a display screen, a speaker, a microphone, a Random Access Memory (RAM), a Touch screen (TP), a camera module, a sensor, and the like. The storage, the signal processor, the display screen, the loudspeaker, the microphone, the RAM, the camera module, the sensor and the TP are connected with the processor, and the communication interface is connected with the signal processor.

The Display screen may be a Liquid Crystal Display (LCD), an Organic or inorganic Light-Emitting Diode (OLED), an Active Matrix/Organic Light-Emitting Diode (AMOLED), or the like.

The camera module can include a common camera and an infrared camera, and is not limited herein. The camera may be a front camera or a rear camera, and is not limited herein.

Wherein the sensor comprises at least one of: light sensors, gyroscopes, Infrared light (IR) sensors, fingerprint sensors, pressure sensors, and the like. Among them, the light sensor, also called an ambient light sensor, is used to detect the ambient light brightness. The light sensor may include a photosensor and an analog-to-digital converter. The photosensitive element is used for converting collected optical signals into electric signals, and the analog-to-digital converter is used for converting the electric signals into digital signals. Optionally, the light sensor may further include a signal amplifier, and the signal amplifier may amplify the electrical signal converted by the photosensitive element and output the amplified electrical signal to the analog-to-digital converter. The photosensitive element may include at least one of a photodiode, a phototransistor, a photoresistor, and a silicon photocell.

The processor is a control center of the electronic equipment, various interfaces and lines are used for connecting all parts of the whole electronic equipment, and various functions and processing data of the electronic equipment are executed by operating or executing software programs and/or modules stored in the memory and calling data stored in the memory, so that the electronic equipment is monitored integrally.

The processor may integrate an Application Processor (AP) and a modem processor, wherein the AP mainly processes an operating system, a user interface, an application program, and the like, and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor.

The processor includes a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU). The CPU is one of the main devices of an electronic computer, and is a core accessory in the computer. Its functions are mainly to interpret computer instructions and to process data in computer software. The CPU is the core component of the computer responsible for reading, decoding and executing instructions. The central processor mainly comprises two parts, namely a controller and an arithmetic unit, and also comprises a cache memory and a bus for realizing data and control of the connection between the cache memory and the arithmetic unit. The three major core components of the computer are the CPU, internal memory, and input/output devices. The central processing unit mainly has the functions of processing instructions, executing operations, controlling time and processing data. The GPU is also called a display core, a visual processor, and a display chip, and is a microprocessor that is dedicated to image and graphic related operations on personal computers, workstations, game machines, and some mobile devices (e.g., tablet computers, smart phones, etc.). The GPU reduces the dependence of the graphics card on the CPU, and performs part of the original CPU work, and particularly, the core technologies adopted by the GPU in 3D graphics processing include hardware T & L (geometric transformation and illumination processing), cubic environment texture mapping and vertex mixing, texture compression and bump mapping, a dual-texture four-pixel 256-bit rendering engine, and the like, and the hardware T & L technology can be said to be a mark of the GPU.

The memory is used for storing software programs and/or modules, and the processor executes various functional applications and data processing of the electronic equipment by operating the software programs and/or modules stored in the memory. The memory mainly comprises a program storage area and a data storage area, wherein the program storage area can store an operating system, a software program required by at least one function and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

Further, as shown in fig. 1B, the electronic device depicted in fig. 1A may further include: the system comprises an eyeball tracking application, an eyeball tracking service, an eyeball tracking algorithm module and an eyeball tracking calibration module.

The eyeball tracking application can be arranged on an application layer of the operating system, and the eyeball tracking service, the eyeball tracking algorithm module and the eyeball tracking calibration module can be arranged on a framework layer of the operating system.

The eye-tracking application may include an OEyeTracker SDK, which is an SDK interface provided for the application. And the api which is responsible for providing the acquisition point of regard and input for the common application is in the form of jar/aar package.

The eye tracking service (oeyetracker service) is mainly responsible for managing the gazing point algorithm, gazing point post-processing, input processing, and authentication and parameter settings.

The eyeball tracking algorithm module (eyetracker algo) is a core algorithm for eyeball tracking, and the core algorithm can be provided by a third party or can be configured in advance. The core algorithm comprises two parts: firstly, a calibration algorithm and secondly, a point-of-regard estimation algorithm;

an eyeball tracking calibration module (OEyeCali calibration module) collects images for the first time to generate calibration generation, and subsequently, the calibration parameters are strengthened through returned coordinate information in the using process.

Referring to fig. 1C, fig. 1C is a schematic flowchart of an eye tracking calibration method according to an embodiment of the present disclosure, and as shown in the drawing, the method is applied to an electronic device shown in fig. 1A or fig. 1B, where the electronic device includes an eye tracking application, an eye tracking service, an eye tracking algorithm module, and an eye tracking calibration module, and the method includes:

101. the eyeball tracking application displays a target calibration point in a display page, and the target calibration point is used for guiding eyeballs of a target object to pay attention to the target calibration point.

The eye tracking application may be any application having an eye tracking function, and the target object may be a user or another person watching the electronic device. The target object may be one or more persons.

In a specific implementation, the eyeball tracking application may display a target calibration point in a display page, where the target calibration point may be one point or one area, the target calibration point may be any shape, any color, hollow or solid point or area, preset content may be displayed within an internal range of the target calibration point, the preset content may be set by a user or default by a system, the preset content may be a text, a character, a pattern, or the like, and the target calibration point is not limited herein, and the target calibration point is used for guiding an eyeball of a target object to focus on the target calibration point.

In one possible example, the step 101 of displaying the target calibration point in the display page may include the following steps:

11. when the eyeball tracking application is started, displaying the target calibration point in a first preset size at a preset position of the display page of the eyeball tracking application, and displaying the internal area of the target calibration point in a countdown mode;

12. and at the end of the countdown, scaling the size of the target calibration point from the first preset size to a second preset size.

The first preset size and the second preset size can be set by a user or defaulted by a system, and the first preset size is not equal to the second preset size.

In specific implementation, when the eyeball tracking application is started, the target calibration point can be displayed at a first preset size at a preset position of a display page of the eyeball tracking application, an internal area of the target calibration point is displayed in a countdown form, for example, 5s countdown is performed, and when the countdown is finished, the size of the target calibration point is scaled from the first preset size to a second preset size, so that enough time is provided for a user to pay attention to a screen, and accurate calibration is facilitated.

102. The eyeball tracking application controls the target calibration point to move according to a preset track, and requests the attention position of the eyeball to the eyeball tracking service.

Wherein, the preset track can be set by the user or the default of the system. In a specific implementation, during the moving process of the target calibration point, the preset track may be reserved, that is, the user may see one line on the screen, and of course, the preset track may not be reserved, that is, only one point on the screen is seen to move.

In one possible example, the step 102 of controlling the target calibration point to move according to the preset track may include the following steps:

21. the eyeball tracking application acquires the brightness of the target environment;

22. determining the preset track corresponding to the target ambient light according to a mapping relation between preset ambient light brightness and the track;

23. and controlling the target calibration point to move according to the preset track.

Wherein, electronic equipment can include ambient light sensor, can realize the ambient light through ambient light sensor and gather, and then, eyeball tracking uses and can interact with ambient light sensor, in order to obtain target environment luminance, can also prestore the mapping relation between predetermined ambient light luminance and the orbit in the electronic equipment, and then, can confirm the predetermined orbit that target environment luminance corresponds according to this mapping relation, and control target calibration point moves according to this predetermined orbit, so, set up different orbits to different ambient light, like fig. 1D and fig. 1E, fig. 1D and fig. 1E provide different orbits, help promoting the degree of attention of user's concern calibration point.

103. The eye tracking service transmits request data to a camera and receives image data transmitted by the camera.

In a specific implementation, the eyeball tracking service may send a data request to the camera, where the request data is used to request the camera to shoot the eyeballs of the target object, and receive image data sent by the camera, where the image data includes the eyeball position of the target object.

104. The eyeball tracking algorithm module determines the attention position of the eyeball corresponding to the display page and the calibration point position of the target calibration point according to the image data to obtain a calibration data set, wherein the calibration data set comprises a plurality of groups of position relation pairs, and each position relation pair corresponds to one attention position and one calibration point position.

In a specific implementation, because the position where the eyeball gazes is a certain position of the display page, the target calibration point moves all the time, and the eyeball of the user also keeps gazing at the target calibration point and moves in the moving process, the eyeball tracking algorithm module can determine the attention position of the eyeball in the display page and the calibration point position of the target calibration point according to the image data to obtain a calibration data set, wherein the calibration data set comprises a plurality of position relationship pairs, and each position relationship pair corresponds to one attention position and one calibration point position.

In one possible example, the step 104 of determining, according to the image data, a position of the eyeball corresponding to the attention position in the display page and a calibration point position of the target calibration point to obtain a calibration data set may include the following steps:

41. the eyeball tracking algorithm module acquires shooting moments in the image data to obtain a plurality of shooting moments, and each shooting moment corresponds to the position of a calibration point of the target calibration point;

42. determining, by the preset eyeball tracking algorithm, that the eyeball corresponding to each of the plurality of shooting moments corresponds to an attention position in the display page to obtain a plurality of attention positions, each shooting moment corresponding to one attention position;

43. and generating the plurality of sets of position relation pairs according to the plurality of attention positions and the calibration point positions of the target calibration point in each shooting moment of the plurality of shooting moments, and taking the plurality of sets of position relation pairs as the calibration data set.

The preset eyeball tracking algorithm can be understood as an algorithm for realizing an eyeball tracking function, the shooting time in the image data can be obtained, a plurality of shooting times are obtained, each shooting time corresponds to the position of a calibration point of a target calibration point, the attention position in a display page corresponding to the eyeball corresponding to each shooting time in the plurality of shooting times, namely the pupil position in the eyeball, can be determined through the preset eyeball tracking algorithm, a plurality of attention positions are obtained, each shooting time corresponds to one attention position, a plurality of position relation pairs are generated according to the plurality of attention positions and the positions of the calibration points of the target calibration point in each shooting time in the plurality of shooting times, and the plurality of position relation pairs are used as a calibration data set.

105. And the eyeball tracking calibration module calibrates a preset eyeball tracking algorithm according to the plurality of groups of position relations to obtain a target calibration parameter set, wherein the target calibration parameter set is used for realizing the next eyeball tracking operation of the preset eyeball tracking algorithm.

In the embodiment of the present application, a preset eye tracking algorithm may be pre-stored in the electronic device, and the concerned position may be inconsistent with the position of the corresponding target calibration point, so that the preset eye tracking algorithm needs to be calibrated to improve the eye tracking accuracy of the preset eye tracking algorithm.

Specifically, the eyeball tracking calibration module may calibrate the preset eyeball tracking algorithm according to the plurality of groups of position relationships, and specifically, may adjust control parameters of the preset eyeball tracking algorithm to obtain a target calibration parameter set, where the target calibration parameter set may be used to implement a next eyeball tracking operation of the preset eyeball tracking algorithm.

In a possible example, in the step 105, calibrating the preset eye tracking algorithm according to the plurality of sets of position relationships to obtain the target calibration parameter set, the method may include the following steps:

51. the eyeball tracking calibration module determines the distance between the attention position in each position relationship pair in the plurality of position relationship pairs and the calibration point position to obtain a plurality of distances;

52. determining the mean value of the plurality of distances to obtain a target mean value;

53. determining the mean square deviations of the distances to obtain a target mean square deviation;

54. determining a target adjusting coefficient corresponding to the target mean value according to a mapping relation between a preset mean value and the adjusting coefficient;

55. determining a target fine-tuning coefficient corresponding to the target mean square error according to a mapping relation between a preset mean square error and the fine-tuning coefficient;

56. acquiring an initial calibration parameter set of the preset eyeball tracking algorithm;

57. and adjusting the initial calibration parameter set according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set.

Specifically, the eyeball tracking calibration module may determine a distance between an attention position in each of the plurality of sets of position relationship pairs and a calibration point position, specifically, may perform distance operation on coordinates corresponding to the attention position in each of the plurality of sets of position relationship pairs and the calibration point position to obtain a plurality of distances, may determine a mean value of the plurality of distances to obtain a target mean value, and determines a mean square error of the plurality of distances to obtain a target mean square error, may also pre-store a mapping relationship between a preset mean value and an adjustment coefficient, and a mapping relationship between a preset mean square error and a fine adjustment coefficient in the electronic device, may have a value range of the adjustment coefficient between-1 and 1, may have a value range of the fine adjustment coefficient between-0.1 and 0.1, and may further determine a target adjustment coefficient corresponding to the target mean value according to the mapping relationship between the preset mean value and the adjustment coefficient, and determining a target fine tuning coefficient corresponding to the target mean square error according to a mapping relation between the preset mean square error and the fine tuning coefficient.

Further, the electronic device may obtain an initial calibration parameter set of a preset eye tracking algorithm, which corresponds to different initial calibration parameter sets for different eye tracking algorithms, where the initial calibration parameter set may be understood as a control parameter of the eye tracking algorithm, and the initial calibration parameter set of each eye tracking algorithm may include at least one calibration parameter. Furthermore, the initial calibration parameter set can be adjusted according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set, so that calibration of the preset eyeball tracking algorithm can be realized, and eyeball tracking accuracy of the preset eyeball tracking algorithm is improved.

In a possible example, in the step 56, adjusting the initial calibration parameter set according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set, the step may include the following steps:

a561, the eyeball tracking calibration module determines a first calibration parameter set in the initial calibration parameter set according to a preset mapping relation between an adjusting coefficient and a calibration parameter;

and A562, adjusting the first calibration parameter set according to the target fine tuning coefficient to obtain the target calibration parameter set.

The calibration parameter set may be a calibration parameter set, and the calibration parameter set may be obtained by adjusting a calibration parameter set according to a target fine tuning coefficient, where, for any calibration parameter i in the calibration parameter set, a calibrated calibration parameter i is (1+ target fine tuning coefficient) — calibration parameter i, and thus, calibration operation of the eye tracking algorithm may be achieved.

b561, the eyeball tracking calibration module adjusts the target adjustment coefficient according to the target fine adjustment coefficient to obtain a final adjustment coefficient;

and B562, adjusting each calibration parameter in the initial calibration parameter set according to the final adjustment coefficient to obtain the target calibration parameter set.

In specific implementation, the eyeball tracking calibration module may adjust the target adjustment coefficient according to the target fine adjustment coefficient to obtain a final adjustment coefficient, specifically, the final adjustment coefficient is a target adjustment coefficient (1+ target fine adjustment coefficient), and then, each calibration parameter in the initial calibration parameter set may be adjusted according to the final adjustment coefficient to obtain a target calibration parameter set, and for any calibration parameter i in the target calibration parameter set, a calibrated calibration parameter j is a final adjustment coefficient, so that the calibration operation of the eyeball tracking algorithm may be implemented.

For example, as shown in fig. 1F, when the user starts the eyeball tracking application, calibration points may be displayed, the eyeball tracking application displays target calibration points in a display page, the target calibration points may be used to guide an eyeball of a target object to focus on the target calibration points, the eyeball tracking application controls the target calibration points to move according to a preset track and request an eyeball focus position from the eyeball tracking service, the eyeball tracking service sends request data to a camera and receives image data sent by the camera, the eyeball tracking algorithm module determines, according to the image data, a focus position in the display page corresponding to the eyeball and a calibration point position of the target calibration point to obtain a calibration data set, the calibration data set includes a plurality of sets of position relationship pairs, each position relationship pair corresponds to a focus position and a calibration point position, the eyeball tracking calibration module calibrates a preset eyeball tracking algorithm according to the plurality of position relationships, the target calibration parameter set is obtained and used for achieving next eyeball tracking operation of the preset eyeball tracking algorithm, accordingly, the eyeball of the user can be guided to move along with the target calibration point through the calibration point, in the moving process, the calibration point position and the attention data are collected and used as calibration data, the eyeball tracking algorithm is calibrated through the calibration data, calibration accuracy of an eyeball tracking technology can be improved, and the eyeball tracking accuracy is improved.

It can be seen that the eyeball tracking calibration method described in the embodiment of the present application is applied to an electronic device, the electronic device includes an eyeball tracking application, an eyeball tracking service, an eyeball tracking algorithm module, and an eyeball tracking calibration module, the eyeball tracking application displays a target calibration point in a display page, the target calibration point is used for guiding an eyeball of a target object to focus on the target calibration point, the eyeball tracking application controls the target calibration point to move according to a preset track and requests the eyeball tracking service for an eyeball focus position, the eyeball tracking service sends request data to a camera and receives image data sent by the camera, the eyeball tracking algorithm module determines a focus position in the display page corresponding to the eyeball and a calibration point position of the calibration target calibration point according to the image data to obtain a calibration data set, the calibration data set includes a plurality of position relationship pairs, each position relationship pair corresponds to a focus position and a calibration point position, the eyeball tracking calibration module calibrates the preset eyeball tracking algorithm according to the multi-group position relation to obtain a target calibration parameter set, wherein the target calibration parameter set is used for realizing the next eyeball tracking operation of the preset eyeball tracking algorithm, so that the eyeball of a user can be guided to move along with the target calibration point through the calibration point, in the moving process, the calibration point position and the attention data are collected and used as calibration data, the eyeball tracking algorithm is calibrated through the calibration data, the calibration precision of an eyeball tracking technology can be improved, and the eyeball tracking precision is favorably improved.

Referring to fig. 2 in a manner consistent with the embodiment shown in fig. 1C, fig. 2 is a schematic flowchart of an eye tracking calibration method provided in an embodiment of the present application, and as shown in the figure, the method is applied to an electronic device shown in fig. 1A or fig. 1B, the electronic device includes an eye tracking application, an eye tracking service, an eye tracking algorithm module, and an eye tracking calibration module, and the method includes:

201. when the eyeball tracking application is started, the target calibration point is displayed in a first preset size at a preset position of the display page of the eyeball tracking application, and the internal area of the target calibration point is displayed in a countdown mode.

202. And at the end of the countdown, scaling the size of the target calibration point from the first preset size to a second preset size, wherein the target calibration point is used for guiding an eyeball of a target object to focus on the target calibration point.

203. The eyeball tracking application controls the target calibration point to move according to a preset track, and requests the attention position of the eyeball to the eyeball tracking service.

204. The eye tracking service transmits request data to a camera and receives image data transmitted by the camera.

205. The eyeball tracking algorithm module determines the attention position of the eyeball corresponding to the display page and the calibration point position of the target calibration point according to the image data to obtain a calibration data set, wherein the calibration data set comprises a plurality of groups of position relation pairs, and each position relation pair corresponds to one attention position and one calibration point position.

206. And the eyeball tracking calibration module calibrates a preset eyeball tracking algorithm according to the plurality of groups of position relations to obtain a target calibration parameter set, wherein the target calibration parameter set is used for realizing the next eyeball tracking operation of the preset eyeball tracking algorithm.

For the detailed description of the steps 201 to 206, reference may be made to corresponding steps of the eyeball tracking calibration method described in fig. 1C, which are not described herein again.

It can be seen that the eyeball tracking calibration method described in the embodiment of the present application can guide the eyeball of the user to move along with the target calibration point through the calibration point, and draw the attention of the user in a countdown manner.

In keeping with the above embodiments, please refer to fig. 3, fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where as shown, the electronic device includes a processor, a memory, a communication interface, and one or more programs, the electronic device includes an eye tracking application, an eye tracking service, an eye tracking algorithm module, and an eye tracking calibration module, the one or more programs are stored in the memory and configured to be executed by the processor, and in an embodiment of the present application, the programs include instructions for performing the following steps:

In a possible example, the calibrating the preset eye tracking algorithm according to the plurality of sets of position relationships to obtain the target calibration parameter set includes:

the eyeball tracking calibration module determines the distance between the attention position in each position relationship pair in the plurality of position relationship pairs and the calibration point position to obtain a plurality of distances;

determining the mean value of the plurality of distances to obtain a target mean value;

determining the mean square deviations of the distances to obtain a target mean square deviation;

determining a target adjusting coefficient corresponding to the target mean value according to a mapping relation between a preset mean value and the adjusting coefficient;

determining a target fine-tuning coefficient corresponding to the target mean square error according to a mapping relation between a preset mean square error and the fine-tuning coefficient;

acquiring an initial calibration parameter set of the preset eyeball tracking algorithm;

and adjusting the initial calibration parameter set according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set.

In one possible example, the adjusting the initial calibration parameter set according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set includes:

the eyeball tracking calibration module determines a first calibration parameter set in the initial calibration parameter set according to a preset mapping relation between an adjusting coefficient and a calibration parameter;

and adjusting the first calibration parameter set according to the target fine tuning coefficient to obtain the target calibration parameter set.

the eyeball tracking calibration module adjusts the target adjustment coefficient according to the target fine adjustment coefficient to obtain a final adjustment coefficient;

and adjusting each calibration parameter in the initial calibration parameter set according to the final adjustment coefficient to obtain the target calibration parameter set.

In one possible example, the presenting the target calibration point in the display page includes:

when the eyeball tracking application is started, displaying the target calibration point in a first preset size at a preset position of the display page of the eyeball tracking application, and displaying the internal area of the target calibration point in a countdown mode;

at the end of the countdown, scaling the size of the target calibration point from the first preset size to a second preset size.

In one possible example, the controlling the target calibration point to move according to a preset trajectory includes:

the eyeball tracking application acquires the brightness of the target environment;

determining the preset track corresponding to the target ambient light brightness according to a mapping relation between the preset ambient light brightness and the track;

and controlling the target calibration point to move according to the preset track.

In one possible example, the determining, according to the image data, a position of the eyeball corresponding to the attention position in the display page and a position of a calibration point of the target calibration point to obtain a calibration data set includes:

the eyeball tracking algorithm module acquires shooting moments in the image data to obtain a plurality of shooting moments, and each shooting moment corresponds to a calibration point position of the target calibration point;

determining, by the preset eyeball tracking algorithm, that the eyeball corresponding to each of the plurality of shooting moments corresponds to an attention position in the display page to obtain a plurality of attention positions, each shooting moment corresponding to one attention position;

and generating the plurality of sets of position relation pairs according to the plurality of attention positions and the calibration point positions of the target calibration point in each shooting moment of the plurality of shooting moments, and taking the plurality of sets of position relation pairs as the calibration data set.

The above description has introduced the solution of the embodiment of the present application mainly from the perspective of the method-side implementation process. It is understood that the electronic device comprises corresponding hardware structures and/or software modules for performing the respective functions in order to realize the above-mentioned functions. Those of skill in the art will readily appreciate that the present application is capable of hardware or a combination of hardware and computer software implementing the various illustrative elements and algorithm steps described in connection with the embodiments provided herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiment of the present application, the electronic device may be divided into the functional units according to the method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. It should be noted that, in the embodiment of the present application, the division of the unit is schematic, and is only one logic function division, and when the actual implementation is realized, another division manner may be provided.

Fig. 4 is a block diagram of functional units of an eye tracking calibration apparatus 400 according to an embodiment of the present application. The apparatus 400 for calibrating eye tracking is applied to an electronic device, the electronic device includes an eye tracking application 401, an eye tracking service 402, an eye tracking algorithm module 403, and an eye tracking calibration module 404, the apparatus 400 includes:

the eyeball tracking application 401 is configured to display a target calibration point in a display page, where the target calibration point is used to guide an eyeball of a target object to focus on the target calibration point;

the eyeball tracking application 401 is configured to control the target calibration point to move according to a preset track, and request the eyeball tracking service 402 for a position of interest of the eyeball;

the eyeball tracking service 402 is used for sending request data to a camera and receiving image data sent by the camera;

the eyeball tracking algorithm module 403 is configured to determine, according to the image data, positions of the eyeballs corresponding to the attention positions in the display page and calibration points of the target calibration points, to obtain a calibration data set, where the calibration data set includes a plurality of sets of position relationship pairs, and each position relationship pair corresponds to one attention position and one calibration point position;

the eyeball tracking calibration module 404 is configured to calibrate a preset eyeball tracking calibration algorithm according to the plurality of groups of position relationships to obtain a target calibration parameter set, where the target calibration parameter set is used to implement a next calibration operation of the preset eyeball tracking calibration algorithm.

In a possible example, in terms of calibrating the preset eye tracking algorithm according to the multiple groups of position relationships to obtain the target calibration parameter set, the eye tracking calibration module 404 specifically includes:

determining the distance between the concerned position in each group of position relation pairs in the plurality of groups of position relation pairs and the position of the calibration point to obtain a plurality of distances;

In a possible example, in terms of the adjusting the initial calibration parameter set according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set, the eye tracking calibration module 404 specifically includes:

determining a first calibration parameter set in the initial calibration parameter set according to a preset mapping relation between an adjusting coefficient and a calibration parameter;

adjusting the target adjustment coefficient according to the target fine adjustment coefficient to obtain a final adjustment coefficient;

In one possible example, in terms of the presentation of the target calibration points in the display page, the eye tracking application 401 is embodied as:

In one possible example, in terms of controlling the target calibration point to move according to a preset trajectory, the eye tracking application 401 specifically is:

obtaining the brightness of the target environment;

In a possible example, in the aspect of determining, according to the image data, the position of the eyeball corresponding to the attention position in the display page and the position of the calibration point of the target calibration point to obtain the calibration data set, the eyeball tracking algorithm module 403 is specifically:

acquiring shooting moments in the image data to obtain a plurality of shooting moments, wherein each shooting moment corresponds to a calibration point position of the target calibration point;

It can be understood that the functions of the program modules of the eyeball tracking calibration apparatus according to this embodiment may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description of the foregoing method embodiment, which is not described herein again.

Embodiments of the present application also provide a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, the computer program enabling a computer to execute part or all of the steps of any one of the methods described in the above method embodiments, and the computer includes an electronic device.

Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods as described in the above method embodiments. The computer program product may be a software installation package, the computer comprising an electronic device.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units is only one type of division of logical functions, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit may be stored in a computer readable memory if it is implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above-mentioned method of the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. An eye tracking calibration method, applied to an electronic device including an eye tracking application, an eye tracking service, an eye tracking algorithm module and an eye tracking calibration module, the method comprising:

the eyeball tracking calibration module calibrates a preset eyeball tracking algorithm according to the plurality of groups of position relations to obtain a target calibration parameter set, wherein the target calibration parameter set is used for realizing the next eyeball tracking operation of the preset eyeball tracking algorithm;

wherein, calibrating the preset eyeball tracking algorithm according to the plurality of groups of position relations to obtain a target calibration parameter set, and the method comprises the following steps:

determining the average value of the distances to obtain a target average value;

2. The method of claim 1, wherein the adjusting the initial calibration parameter set according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set comprises:

3. The method of claim 1, wherein the adjusting the initial calibration parameter set according to the target adjustment coefficient and the target fine adjustment coefficient to obtain the target calibration parameter set comprises:

4. The method according to any one of claims 1-3, wherein the presenting the target calibration point in a display page comprises:

when the eyeball tracking application is started, displaying the target calibration point at a first preset size at a preset position of the display page of the eyeball tracking application, and displaying an internal area of the target calibration point in a countdown mode;

5. A method according to any of claims 1-3, wherein said controlling the target calibration point to move according to a preset trajectory comprises:

6. The method according to any one of claims 1 to 3, wherein determining the calibration point positions of the eyeball corresponding to the focus position in the display page and the target calibration point according to the image data to obtain a calibration data set comprises:

the eyeball tracking algorithm module acquires shooting moments in the image data to obtain a plurality of shooting moments, and each shooting moment corresponds to the position of a calibration point of the target calibration point;

7. An eye tracking calibration device, applied to an electronic device, the electronic device including an eye tracking application, an eye tracking service, an eye tracking algorithm module and an eye tracking calibration module, the eye tracking calibration device comprising:

the eyeball tracking calibration module is used for calibrating a preset eyeball tracking calibration algorithm according to the plurality of groups of position relations to obtain a target calibration parameter set, and the target calibration parameter set is used for realizing the next calibration operation of the preset eyeball tracking calibration algorithm;

8. An electronic device comprising a processor, a memory for storing one or more programs and configured for execution by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-6.

9. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-6.