RU219079U1 - Device for determining the direction of gaze - Google Patents

Abstract

The utility model relates to devices for determining the direction of gaze and can be used in various fields of technology, including robotics. The device comprises a body, a left eye camera and a right eye camera for obtaining an image of the eyes, a scene camera for obtaining an image of the surrounding scene, a plurality of light sources for the left eye and the right eye for forming glare on the eyes, a computing module for determining the gaze direction vector, performing calibration procedures, and transmitting view direction vector information to an external device, a control module, and at the same time, the device is configured to connect the components located on the device case to the power supply via a cable. The technical result is an increase in the accuracy of determining the direction of sight.

Description

This utility model relates to devices for determining the direction of gaze and can be used in various fields of technology, including robotics.

RF patent RU 2696042 describes an invention related to technologies used to determine gaze fixation areas during eye movement, and can be used for an objective assessment of visual attention processes, computer interface control by means of gaze direction, in operator activities, marketing, etc. Area detection system gaze fixation during eye movement includes an infrared emitter that covers the area of the respondent's eye, a reflective filter that transmits visible light and reflects infrared radiation, installed at the level of the respondent's eyes, a block for presenting a stimulating image located behind a reflective filter, a block for recording infrared radiation reflected from pupil, in the form of an image of the pupil of the eye, a video camera, and an image processing unit from a video camera.

The disadvantage of this technical solution is that fixation of the gaze during eye movement is carried out with a fixed position of the respondent's head relative to the presented image. In addition, the accuracy of determining the direction of view is not high enough, which is especially important when it is necessary to determine the direction to a specific object located among many other objects.

US Pat. No. 1,0234,940 describes a gaze tracking device and a gaze tracking method comprising the steps of: recording video images of a human eye such that the pupil of the eye and the glare on the eyeball caused by a light source are recorded; processing the video images to calculate an offset between a position of a predetermined spatial element and a predetermined position relative to a flare; and by means of a light source such as a display, emitting light from the light pattern at a location selected from the plurality of preconfigured light pattern locations to the human eye. The location is controlled by a feedback signal; controlling the location of the light pattern among the predetermined locations of the light patterns in response to the offset, so that the predetermined position relative to the glare caused by the light source tracks the spatial feature of human eyes. Meanwhile, the above-described steps are repeated to establish a control loop with the location of the lighting pattern controlled by the feedback signal. The gaze tracker is configured to filter the video images in order to identify one or more glare that may originate from the light pattern, wherein a predetermined position relative to the glare is calculated relative to the identified one or more points.

This device makes it possible to increase the accuracy of eye position recognition and the formation of the resulting image, however, it imposes increased requirements on the hardware.

US Patent US9830512 describes a method and apparatus for eye tracking. The method comprises the following steps: determining the position of the central point of the cornea using at least two points of light reflection detected in the region of the eyeball of the first image of the user's face; calculating a first vector with respect to at least two fixed feature points detected from the first face image and the position of the center point of the cornea; calculating the position of the central point of the cornea relative to the eyeball region of the second face image using the position of the feature point detected from the second face image and the first vector when at least two light reflection points are not detected from the eyeball of the user's second face image region; calculating a second vector using the calculated position of the center point of the cornea and the position of the center point of the pupil detected from the eyeball region of the second face image; and tracking the gaze of the user using the second vector.

These well-known method and system for eye tracking can improve the accuracy of eye position recognition and the formation of the resulting image, but are quite complex and demanding on computing resources.

As the closest analogue of the present utility model, a system for eye tracking based on adaptive tomographic matching according to US patent US9684827 can be chosen. The system comprises at least four light sources for generating corneal glare reflections from the subject's eye; a camera for capturing a current image containing glare; a gaze detector for receiving a current image containing glare and estimating the gaze of the subject's eye. The system also contains a head position offset corrector to correct the offset by matching features corresponding to glare and data related to the subject's pupil to obtain corrected gaze information, the gaze offset corrector using one or more variables representing head positions relative to the calibration position.

This well-known system for eye tracking based on adaptive tomographic matching also improves the accuracy of gaze position recognition and formation of the resulting image, but is quite complex and demanding on computing resources, and in addition, requires additional actions from the user.

Thus, there is a problem to create such a device for determining the direction of gaze, for which one-time calibration for one user is sufficient, which simplifies the repeated use of the device, which increases the accuracy of determining the direction of gaze even with small changes in the position of the system on the user.

The technical result of the claimed device is to increase the accuracy of determining the direction of view.

The problem is solved, and the technical result is achieved in the claimed device for determining the user's gaze direction, which contains a housing, a left eye camera and a right eye camera, a scene camera, a plurality of left eye light sources and right eye light sources, a computing module, a control module, while the device is configured to connect the components located on the device body with the power supply unit via a cable.

The body is essentially made in the form of glasses and has a left rim and a right rim, a nose and two side clamps. The left eye cameras and the right eye cameras are installed in the lower parts of the respective rims below the respective eye and are designed to take images of the left eye and the right eye, i.e. unlike analogues in the claimed device receive images from both eyes, not one. The scene camera is mounted on the nose and is designed to obtain an image of the surrounding scene. A plurality of light sources are installed on the left rim and right rim to form highlights on the eyes. The computing module is configured to determine the gaze vector in the coordinate system of the cameras of the left and right eyes (hereinafter also referred to as local coordinates for brevity); transforming the view direction vector in local coordinates into the view direction vector in the scene camera coordinate system (hereinafter also referred to as global coordinates for brevity); performing calibration procedures and transmitting information about the coordinates of the gaze direction vector in the scene camera coordinate system to an external device, for example, a robotic arm that can be controlled by means of the claimed gaze direction determination device. The control module provides control of the claimed device and interaction between the components of the device. The power supply provides power to the components of the device.

The claimed design of the device for determining the direction of gaze is simple and light enough to be constantly used, and at the same time provides high accuracy in determining the direction of the user's gaze direction vector in global coordinates (in the coordinate system of the scene camera), which is essentially a coordinate system of placement in space object of observation.

According to the utility model, the device uses six left eye light sources and six right eye light sources. At the same time, it is preferable that the sources for each eye be set essentially so as to ensure uniform illumination of the eye, which means increased accuracy in determining the direction of gaze and the possibility of determining it at large angles of rotation of the eyeball.

In the preferred embodiment of the claimed device for determining the local coordinates of the gaze direction vector, the computing module is configured to determine the local coordinates of the highlights and the center of the pupil. In addition, or in addition to this, to convert the local coordinates of the gaze direction vector to the global coordinates of the gaze direction vector, the computing module can be configured to match the local coordinates of the gaze vector and the surrounding scene obtained by the scene camera.

The control module can control the brightness of the left eye light sources and the right eye light sources to provide optimal brightness depending on the surrounding conditions, hence the required accuracy in determining the gaze direction vector.

The device may additionally contain an information storage unit, in particular, for storing the results of calibration procedures for a particular user, necessary, in particular, for converting the local coordinates of the gaze direction vector into the global coordinates of the gaze direction vector.

Next, the device, as well as some possible options for its implementation are explained in detail with reference to the figures, which show:

in fig. 1 shows a general view of the claimed device;

in fig. 2 is a schematic front view of the claimed device;

in fig. 3 is a schematic rear view of the claimed device;

in fig. 4 schematically shows the relative position of the left or right eye, the camera of the left or right eye and one light source of the left or right eye, as well as the ray path.

Reference numerals in the figures indicate:

1 - device for determining the direction of view;

2 - body;

3.1 - left bezel;

3.2 - right bezel;

4 - nose;

5.1, 5.2 - side clamps;

6.1 - camera of the left eye;

6.2 - camera of the right eye;

7 - stage camera;

8 - light source;

8i - i-th light source 8;

9 - computing module;

10 - control module;

11 - eye;

12 - iris of the eye;

C - nodal point of the eye;

P - the center of the pupil;

R - refraction point;

Qi - reflection point of the i-th light source 8;

O - nodal point of the camera;

V - image of the center of the pupil;

Ui - image of the glare of the i-th light source 8.

The device 1 for determining the direction of the user's gaze includes a body 2, which is essentially made in the form of glasses, having a left rim 3.1 and a right rim 3.2, nose 4 and two side pressure bars 5.1, 5.2.

The body 2 is preferably made of a durable and lightweight material suitable for permanent or long-term wearing of the device 1, taking into account the anthropometric data of the user's head. For greater user convenience and to provide the possibility of adjusting the position of the device 1 in order to ensure the correct operation of all systems, replaceable nosepieces 4 can be used, which adjust the position of the body 2 in height and are selected under the bridge of the nose to prevent it from being pinched. The correct selection of the distance from the eye to the rims 3.1, 3.2 contributes to the correct operation of the eye image recognition system.

In the lower parts of the rims 3.1, 3.2, preferably in the areas of the light openings in planes passing through the vertical axes of the light openings, the camera 6.1 of the left eye and the camera 6.2 of the right eye are installed, respectively, for brevity also referred to as the cameras 6.1, 6.2 of the eye.

Eye cameras 6.1, 6.2 are designed to register and record a video sequence showing the process of eye movement and to obtain images of the corresponding eye. At the same time, the movement of both the left and right eyes of the user is recorded, which is necessary to accurately determine the direction of gaze and eliminate false positives. Eye cameras 6.1,6.2 contain photosensitive matrices and corresponding optical systems (objectives) for forming an enlarged focused image of the eyes on photosensitive matrices.

To correctly determine the direction of gaze, the images of the pupil and highlights are registered and the coordinates of the center of the pupil and each of the highlights are determined in the coordinate system of the cameras 6.1, 6.2 of the eye (local coordinates).

When working in real conditions due to obstacles created by the eyelids and eyelashes, correct registration of the pupil image in any direction of gaze is possible only when the constituent components of the cameras 6.1, 6.2 of the eye - the photosensitive matrix and the lens - are placed in a plane parallel to the main planes of the optical system of the eye, i.e. directly in front of the eye. However, such an arrangement obscures the user's view and creates discomfort when using the product. To improve the ergonomic characteristics of the product, the photosensitive matrix of the cameras 6.1, 6.2 of the eye is located in a plane at an angle to the main planes of the optical system of the eye 11, as schematically shown in Fig. 4. Such an arrangement will not create obstacles and close the user's view.

To determine the direction of gaze at any position of the pupil, one eye camera 6.1, 6.2 is used to register the image of each eye 11 from one angle - from below (Fig. 4). The chosen angle is due to the fact that the lower eyelashes are significantly shorter than the upper ones and do not create obstacles for image registration. The angle between the optical axis of the lens and the line of sight is chosen based on considerations of a compromise between the best image registration angle (corresponding to the 0° angle) and the angle at which the camera is guaranteed not to fall into the user's field of view (corresponding to the 90° angle). Most preferred is the value of the specified angle from 45° to 60°.

The advantage of using two eye cameras 6.1, 6.2 is to increase the reliability of the device 1 due to the possibility of maintaining its operability in case of failure of one of the recording channels, including, among other things, the eye cameras 6.1, 6.2. In this case, the determination of the gaze direction will be determined according to the data from the camera 6.1, 6.2 of the eye, related to the working recording channel. In this mode, the device 1 will remain operational with an acceptable loss of performance in a number of ways.

As photosensitive matrices, it is preferable to use a matrix made on the basis of CMOS technology. The choice of a CMOS matrix is due to the fact that this type of matrix is characterized by a small pixel size, low power consumption, the ability to implement a number of processing functions on a chip, low blurring and blurring of the image, and the ability to read information both from the entire matrix and from a separate area. The use of this matrix makes it possible to reliably highlight the detected objects due to their achromatic colors (the pupil is sharply black, the highlights are bright white) at the stage of eye image processing. This approach simplifies the detection algorithm for detected objects by eliminating additional particle filtering procedures that lead to the loss of important image details. When testing the prototype sample, the optimal distance from the front focal plane of the lens of cameras 6.1, 6.2 of the eye to the corresponding eye 11 was determined experimentally, corresponding to approximately 20 ± 1 mm.

The format of the photosensitive matrix is chosen based on the requirements for photosensitivity. Thus, a larger sensor size with the same number of pixels has a greater sensitivity due to the larger pixel area, thereby introducing a lower noise level, providing a better image quality, simplifying and speeding up the processing of the received eye images. In addition, the larger size of the matrix provides the maximum viewing angle of the cameras 6.1, 6.2 eyes at small focal lengths of the lens. The optimal solution is to choose a 1/3" photosensitive matrix with a side size of 4.8x3.6 mm, which allows for sufficient light sensitivity. Such a matrix can be, for example, OmniVision OV4688.

The optical system of cameras 6.1, 6.2 of the eye (not shown in the figures) is designed to form an image in the plane of the photosensitive matrix. The optical system contains several lenses and a light filter. The distance from the entrance pupil of the lens to the eye does not change significantly, and the distance determined on the current frame is used as an initial approximation in the calculations on the next frame. To form an undistorted image in the plane of the matrix, the optical scheme of a compact four-component lens with an output aspherical lens to compensate for distortion can be used. There are no special requirements for the depth of field of the image. To ensure the operation of the camera module 6.1, 6.2 of the eye, including in the near infrared (IR) range, it may be necessary to install a light filter in front of the matrix to select the spectral interval corresponding to the near infrared radiation of 750-1400 nm.

The nose 4 has a scene camera 7 (FIG. 2) which is used to obtain an image of the surrounding scene. The scene camera 7 is a front-facing high-resolution digital video camera to provide fixation of the environment, to which the gaze direction vector is subsequently linked. The scene camera 7 contains a photosensitive matrix and is preferably located on the top of the body 2 of the glasses. Since there are no special requirements for the stage camera 7, any standard compact camera based on a CMOS color sensor and without autofocus can be used.

On each of the rims 3.1, 3.2, light sources 8 are installed around the light openings, which form glare on the corresponding eye (Fig. 3). Light sources 8 are designed to create the minimum required level of illumination and the formation of point highlights due to reflections from the cornea of the eye.

The light source 8 can be an IR source, in particular an IR LED.

Off-axis IR illumination of the eye creates a dark pupil effect and forms images of light sources 8 by reflecting radiation from the cornea of the eye. Images of light sources 8 formed by reflection from the cornea are called first Purkinje images, or highlights. The dark pupil and glare are then visualized by the optical system of cameras 6.1, 6.2 of the eye and are captured by photosensitive matrices with sufficient sensitivity in the near infrared region. The pupil and glare images move in proportion to the rotation of the eyeball, but along different trajectories. The difference vector between these two features is used to determine the gaze direction vector.

The central emission wavelength of the light sources 8 is preferably selected from the wavelength range from 860 to 990 nm, which in the best embodiment of the claimed device 1 corresponds to the operating range of the eye cameras 6.1, 6.2 (near infrared range). The choice of near infrared radiation for light sources 8 is due to several reasons, in particular:

- near infrared radiation is invisible to the human eye, does not distract the user's attention and does not cause pupil dilation;

- near infrared radiation does not cause degradation and destruction of the receptor apparatus of the human eye;

- near infrared radiation is recorded by the same means as visible radiation;

- when using IR radiation, the image of the pupil is characterized by high contrast due to reflection from the retina (illumination scheme with a dark pupil);

- the use of IR radiation makes it possible to separate useful information from external flare falling on the visible part of the spectrum.

The number of light sources 8 determines the number of highlights on the cornea, against which the distance to the center of the pupil will be measured. To improve the accuracy of determining the direction of sight, two or more highlights are usually used. In the claimed device 1 for determining the direction of view, the method of measuring six glare for each eye is used. This approach improves not only the accuracy, but also the reliability of the device. So, when the image of one or more glare disappears due to the failure of one or more light sources 8, an obstacle arises in the path of the beam incident (reflected) from the cornea, and also in the absence of information about the coordinates due to a malfunction of the device, the possibility remains determining the gaze direction by the coordinates of three or two highlights with an acceptable loss of accuracy in determining the gaze direction vector. In addition, the use of six flares makes it possible to increase the operating range of angles of the device 1 due to the reliable recovery of information about the direction of the gaze vector at large angles of rotation of the eyeball. At such angles, part of the glare falls on the sclera of the eye and is not detected by the cameras 6.1, 6.2 of the eye, but due to the use of six glare, at least four of them fall on the pupil or iris 12 (Fig. 4), and their coordinates are determined with a sufficiently high reliability.

Light sources 8 are installed in rims 3.1, 3.2 in the region of the left and right light openings and are positioned relative to the light opening in such a way that six glare on the cornea of each eye forms a control pattern in the form of a hexagon. Light sources 8 can be installed on platforms located in special grooves on the rims 3.1, 3.2. The angles of the sites are calculated so as to provide uniform illumination of the eye area. The installation depth of the grooves in the rims 3.1, 3.2 is chosen in such a way as to ensure the unhindered passage of radiation.

Preferably, the irradiation is performed with non-collimated divergent beams to uniformly illuminate the entire area under analysis. Since an off-axis (relative to the optical axis of the lenses of cameras 6.1, 6.2 eyes) lighting scheme is selected, the angle between the optical axis of the lens of cameras 6.1, 6.2 of the eye and the normal to the light emitting area of the light sources 8 lies in the range from 0 to 90 ° and much more than zero (see Fig. 4). Such a scheme allows the use of six separate light sources 8 for each light opening and simplifies the layout process of the device 1. At the same time, it is preferable to place the light sources 8 symmetrically about the horizontal and vertical axes of the light opening, at equal distances from the optical axes of the lenses of the cameras 6.1, 6.2 of the eye.

In addition, it is also preferable if the light sources 8 operate in a periodic modulation mode to increase the efficiency of the process of extracting useful information against the background of external illumination, extend the service life, reduce energy consumption and reduce exposure to the cornea and retina of a person. Control of the intensity of the glow of the light sources 8 is carried out by modulating the supply current.

As a light source 8, a single IR LED with an appropriate standard lens, in particular, SFH-4053, can be used.

The device 1 also contains a computing module 9, a control module 10 and a power supply unit (not shown in the figures). The placement of the computing module 9 and the control module 10 can be any, for example, on the clamping bars 5.1, 5.2, as shown in Fig. 1.3.

Computing module 9 is designed to implement the algorithm for determining the direction of gaze and performs the following functions:

- determination of the coordinates of the gaze direction vector in the coordinate system of the cameras 6.1, 6.2 of the eye;

- transformation of the coordinates of the gaze direction vector from the coordinate system of the cameras 6.1, 6.2 of the eye into the coordinate system of the camera 7 of the scene,

- implementation of gaze direction calibration;

- if necessary, transfer of information about the global coordinates of the view direction vector to an external device.

In order to perform its functions, the computing module 9 contains a pupil position and size detector (not shown in the figures; hereinafter also PRSD) or is connected to it as an external device. RPPD is designed to process a high-resolution image of the eye in real time with a frame rate of at least 50 Hz in order to construct an ellipse that coincides with the contour of the pupil, and then determine the coordinates of its center. The DPRZ also calculates the positions of the glare. The small size, weight and low power consumption of the DPRP unit should make it possible to place the computing module 9 on the housing 2, in particular, on the clamping bar 5.1 or 5.2 (as shown in Fig. 2 for an example).

The RPPP is implemented in the form of a field-programmable gate array (FPGA), which allows not only to reduce power consumption, but also to reduce the overall weight of the device and makes it possible to set the shape of the computing module 9 to the dimensions and shape of the case 2 instead of creating a heavier and bulkier one, and a less convenient shape around case 2, whose form factor often cannot be changed.

The control module 10 (not shown in the figures) provides two functions:

- coordination of the interface of the cameras 6.1, 6.2 of the eye with the interface of the computing module 9;

- control of the brightness of the light sources 8, as mentioned above, for example, using pulse-width regulation.

The data from the cameras 6.1, 6.2 of the eye is fed through the control module 10 to the RRP.

The exchange of data between the DPR and the computing module 9 is carried out via any interface suitable for these purposes, for example, USB 3.2 Gen 2. The ellipse and glare parameters determined by the DPR from the pupil image are transmitted to the computing module 9, where they are used to calculate the direction of view.

Any suitable power source, rechargeable (battery) or non-rechargeable, can be used as the power supply.

Preferably, if the power supply is remote, connected to the components of the device 1, located on the housing 2, via a cable.

In addition, the device 1 may further comprise an information storage unit (not shown in the figures), in particular for storing the results of gaze direction calibration, as discussed below.

The device 1 continuously monitors the direction of the user's gaze and transmits the coordinates of the intersection of the gaze direction vector with the image plane received from the scene camera 7, provided that the gaze direction vector is in the field of view of the scene camera 7. Fixation of a point (area) of attention can be carried out by volitional blinking, keeping attention at this point (area) for a fixed time, or by another method.

To determine the gaze direction, or the gaze direction vector, by means of the claimed device 1, the method of digital infrared video oculography can be used, followed by linking the found gaze direction vector to the image of the environment. In this method, the eye is illuminated with light, in particular infrared light, which is reflected from the cornea and lens of the eye 11 and then recorded by the eye cameras 6.1, 6.2. The position of the pupil is calculated as the center of an area of sharp contrast within the iris, which is observed when illuminated by light sources 8 . The glare on the cornea of the eye, caused by the corneal reflection, is used as a reference point for measuring the direction of gaze. The vector of the difference between the coordinates of the center of the pupil and the glare changes with a change in the direction of gaze and is connected by geometric relationships with the vector of the gaze direction. Binding of the view direction vector to the environment is carried out by superimposing the found coordinates of the vector on the image of the surrounding scene, which can be obtained using the scene camera 7. Improving the accuracy of determining the gaze direction vector is achieved, among other things, by calibrating the device, which can be performed both during the implementation of the method and in advance, once for a given user.

Image processing is performed in real time.

Device 1 can operate in two modes: calibration mode and operating mode.

An example of a calibration option includes the following steps:

- form a calibration label, for example, displaying it on the screen of a computer monitor;

receiving images from the scene camera 7, thereby also determining the coordinate system of the scene camera 7 (hereinafter also referred to as the global coordinate system);

- determine the coordinates of the center of the calibration marks in the coordinate system of the camera 7 of the scene;

- determine the vector of the direction of view in the coordinate system of the cameras 6.1, 6.2 of the eye (hereinafter also referred to as the local coordinate system);

- calculate the conversion coefficients from the local coordinate system to the global coordinate system.

It is enough to carry out the calibration procedure once for a particular user.

After the calibration procedure, you can use the device 1 in the operating mode, in which by means of the cameras 6.1, 6.2 the eyes determine the gaze direction vector in the local coordinate system (i.e., in the coordinate system of the cameras 6.1, 6.2 of the eye), recalculate the local coordinates of the gaze direction vector into global coordinates of the gaze direction vector (ie, to the coordinate system of the scene camera 7) and obtain the corresponding gaze fixation point on the scene image.

The following describes a variant of the main stages of determining the position of the pupil of the eye by means of the claimed device 1.

First stage. Obtaining images of the left and right eyes from cameras 6.1, 6.2 eyes. The image of the eye comes from the cameras 6.1, 6.2 of the eye with a frequency of at least 50 Hz. The field of view of the cameras 6.1, 6.2 of the eye is chosen so that the eye always remains in the frame, regardless of variations in the position of the device 1 on the user's head. The position of the eye in the eye image frame can change, it is not always located in the center of the frame.

Second phase. Preliminary pupil search. The pupil is the largest coherent dark area in the image of the eye. By means of the computing module 9, the preliminary position of the center of the pupil is determined, as well as the number of pixels in the pupil area, which previously characterizes its size. Due to the not quite uniform illumination of the dark area of the pupil, part of the pupil may not be included in the pupil area, so that the center and size of the pupil are not quite accurate at this stage.

Third stage. By analyzing the histogram of the brightness values of the pixels in the area selected in the second stage, by means of the computing module 9, the binarization threshold is found to accurately build the pupil ellipse. Next, binarization is used to determine the pupil boundary, and based on the pupil boundary, a preliminary pupil ellipse is constructed.

Fourth stage. Accurate construction of the ellipse of the pupil. The exact construction of the pupil ellipse is performed by the nodal points of the pupil boundary determined in the third stage, filtered so that the nodal points form a convex figure. The construction of the ellipse of the icon is carried out by the least squares method using the computing module 9.

The modeling of these methods carried out by the authors according to the third and fourth stages shows a high probability of a visual coincidence of a precisely constructed pupil ellipse with the pupil border visible on the images of the eye obtained from the cameras 6.1, 6.2 of the eye at the first stage.

Fifth stage. Determination of the position of highlights and numbering of highlights using the computing module 9.

Glare is numbered to determine the direction vector of the gaze in the global coordinate system determined by the scene camera 7, for example, in the following order: from one of the upper pair closest to the bridge of the nose, in a circle, away from the bridge of the nose (clockwise for the right eye and counterclockwise for the left).

Sixth stage. Determination of the direction of the optical axis of the eye, which is performed by means of a computing module 9.

Seventh stage. Determining the gaze direction, which is also performed by the computing module 9. First, the direction angles of the gaze direction vector in the local coordinate system (determined by the eye cameras 6.1, 6.2) are determined. Then they are recalculated into the global coordinate system determined by the camera 7 of the scene. At the same time, the individual characteristics of the user are taken into account (the angles of deviation of the best vision area from the eye direction vector) and the design features of the device for determining the direction of the user's gaze (in particular, the relative position of the cameras 6.1, 6.2 and the camera 7 of the scene), which are determined at the calibration stage.

Calibration step. Calibrating the user's gaze direction consists of two steps. At the first step, the actual calibration is performed, at the second step, the calibration is checked. Calibration is carried out using, for example, a monitor screen or a tablet, along which the ArUco label moves. During the development of the calibration procedure, it was found that a moving mark gives approximately three times more accurate results than a stationary mark, and also holds the user's attention better.

Thus, the present device can achieve high accuracy in determining the direction of view even with small changes in the position of the system on the user, while remaining compact and consuming low power.

Claims (4)

1. A device for determining the direction of the user's gaze, containing a body made essentially in the form of glasses and having a left rim and a right rim, a nose and two side clamps, a left eye camera and a right eye camera installed in the lower part of the left rim and the right rim, respectively below the corresponding eye, configured to obtain an image of the left eye and the right eye, respectively, in the eye camera coordinate system, a scene camera for obtaining an image of the surrounding scene in the scene camera coordinate system, mounted on the nose, six light sources of the left eye and six light sources of the right eye, installed respectively on the left rim and right rim to form glare on the left eye and right eye, respectively, in such a way as to ensure the uniformity of illumination, a computing module with the ability to determine the coordinates of the gaze direction vector in the coordinate system of the eye cameras by determining the coordinates of the glare and the center of the pupil in the system coordinates of the eye cameras, converting the coordinates of the gaze direction vector from the eye camera coordinate system to the scene camera coordinate system, calibrating the gaze direction and transmitting information about the coordinates of the gaze direction vector in the scene camera coordinate system to an external device, a control module, while the device is configured to connecting the components located on the device case with the power supply via a cable. 2. The device according to claim 1, in which, in order to transform the coordinates of the gaze direction vector, the computing module is configured to compare the coordinates of the gaze direction vector in the coordinate system of the eye cameras and the surrounding scene obtained by the scene camera, in the coordinate system of the scene camera. 3. The apparatus of claim 1, wherein the control module is configured to control the brightness of the left eye light sources and the right eye light sources. 4. The device according to claim. 1, additionally containing an information storage unit.

Images