WO2020042589A1

WO2020042589A1 - User distance estimation method and apparatus, device, and storage medium

Info

Publication number: WO2020042589A1
Application number: PCT/CN2019/078025
Authority: WO
Inventors: 付阳; 王云飞; 黄通兵
Original assignee: 北京七鑫易维信息技术有限公司
Priority date: 2018-08-29
Filing date: 2019-03-13
Publication date: 2020-03-05
Also published as: CN109284002B; CN109284002A

Abstract

Disclosed in the embodiments of the present invention are a user distance estimation method and apparatus, a device, and a storage medium. The method comprises: acquiring a user eye image; and, by means of the user eye image, determining the distance between the user and a line of sight tracking apparatus. The embodiments of the present application, by means of acquiring a user eye image and determining the distance between the user and a line of sight tracking apparatus, accurately estimate the distance between the user and the line of sight tracking apparatus.

Description

User distance estimation method, device, equipment and storage medium

Technical field

The embodiments of the present application relate to the technical field of gaze tracking, and in particular, to a user distance estimation method, device, device, and storage medium.

Background technique

Some new types of operating electronic devices with display platforms such as desktop monitors, notebook monitors and flat-panel monitors have gaze tracking devices. FIG. 1 is a schematic structural diagram of a line-of-sight tracking device provided in the prior art. Generally, the line-of-sight tracking device is formed by connecting data lines of the image acquisition device 10, the infrared light source 20, and the body 30 side of the line-of-sight tracking device. Infrared light has no light sense, so the infrared light source can replace the general light source to illuminate the eyes at a short distance without affecting the normal visual effect of a person.

In some application scenarios, it is necessary to know the distance between the user and the gaze tracking device. Without additional distance measuring equipment, it is difficult to measure or calculate the distance based on the images acquired by the image acquisition device. This patent proposes a method for estimating the distance from a user to a gaze tracking device.

Summary of the Invention

The embodiments of the present application provide a user distance estimation method, device, device, and storage medium, so as to accurately estimate the distance between the user and the gaze tracking device.

In a first aspect, an embodiment of the present application provides a user distance estimation method. The user distance estimation method includes:

Obtaining an image of a user's eye;

A distance between the user and the gaze tracking device is determined by an image of the eye of the user.

Optionally, the acquiring an eye image of a user includes:

The user's eye image obtained by the user is received through a wired and / or wireless communication module.

Optionally, determining the distance between the user and the gaze tracking device through the eye image of the user includes:

Determining the spot distance or binocular distance of the eyes in the eye image through the eye image;

A distance between the user and the gaze tracking device is determined according to the spot distance and / or the binocular distance.

Optionally, determining the distance between the user and the gaze tracking device according to the light spot distance and / or the binocular distance includes:

The distance between the user and the gaze tracking device is determined by a function relationship or a mapping table of the spot distance and / or binocular distance and the distance.

In a second aspect, an embodiment of the present application further provides a user distance estimation device including a target screen, a gaze tracking device main body, an image acquisition device, and an infrared light source, and further includes:

A wireless communication module configured to receive a user's eye image collected by a current user terminal;

The image processing module is connected to the wireless communication module and is configured to determine a distance between the user and the gaze tracking device through an image of the eye of the user.

Optionally, the wireless communication module is configured as:

Optionally, the image processing module includes:

A spot distance or binocular distance determination unit, configured to determine, based on the eye image, a spot distance or binocular distance of an eye in the eye image;

The distance determining unit is configured to determine a distance between the user and the gaze tracking device according to the spot distance or the binocular distance.

Optionally, the distance determining unit is set as:

In a third aspect, an embodiment of the present application further provides a device, where the device includes:

One or more processors;

A storage device configured to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the user distance estimation method according to any one of the embodiments of the present application.

According to a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the user distance estimation method according to any one of the embodiments of the present application is implemented.

In the embodiment of the present application, the distance between the user and the gaze tracking device is determined by acquiring an eye image of the user, and the position of the main body of the gaze tracking device is adjusted according to the distance, so as to accurately estimate the user's line of sight. Track the distance of the device.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a schematic structural diagram of a gaze tracking device provided in the prior art;

2 is a flowchart of a user distance estimation method provided in Embodiment 1 of the present application;

3 is a flowchart of a user distance estimation method provided in Embodiment 2 of the present application;

4 is a structural diagram of a user distance estimation device provided in Embodiment 3 of the present application;

FIG. 5 is a schematic structural diagram of a device provided in Embodiment 4 of the present application.

detailed description

In order to make the purpose, technical solution, and advantages of the present application clearer, specific embodiments of the present application are described in further detail below with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application.

It should also be noted that, for the convenience of description, only some parts related to the present application are shown in the drawings instead of the entire content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although flowcharts describe operations (or steps) as sequential processing, many of these operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of operations can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the drawings. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and so on.

Example one

FIG. 2 is a flowchart of a user distance estimation method provided in Embodiment 1 of the present application. This embodiment is applicable to a case where a distance between a user and a gaze tracking device is estimated. The method may be performed by the user distance estimation device. carried out. The user distance can be an abstract / general description, that is, the distance from the user's feature point to the sight-tracking device, or the average distance from multiple user feature points to the sight-tracking device. The user feature point can be the user's eyebrow, For the left eye, right eye, nose tip, or forehead, the position corresponding to the gaze tracking device may be the optical center of the image acquisition device at the gaze tracking device, the sensor, or the plane of the display at the gaze tracking device. The user distance estimation method provided in the embodiment of the present application specifically includes the following execution steps:

S110. Obtain an eye image of the user.

Specifically, the eye image may be directed to the eye through a light source, and the eye is captured by the eye image acquisition device. The eye image acquisition device may include at least one image acquisition device, and the image acquisition device may be an infrared camera or an infrared camera. Wait. Correspondingly, the light spot on the cornea, which is the light spot (also referred to as a Purchin spot), is captured, and the user's eye image obtained from this is the user's eye image with light spot.

S120. Determine a distance between the user and the gaze tracking device through an eye image of the user.

As the eyeball rotates, the relative positional relationship between the pupil center and the light spot changes, and the corresponding collected eye image of the user with the light spot will reflect the corresponding positional change relationship, and thus according to the position change relationship Gaze / gaze point estimation for gaze tracking. That is, when the distance between the user and the gaze tracking device is too close, the relative position of the light spot formed on the cornea and the center of the pupil changes when the light source shines on the eye, and the corresponding eye image with the light spot cannot be detected by the eye. The image acquisition device captures, and at this time, it is necessary to realize the estimation of the distance between the user and the gaze tracking device through the obtained eye image of the user with light spots.

The embodiment of the present application determines the distance between the user and the gaze tracking device by acquiring an eye image of the user, so as to accurately estimate the distance between the user and the gaze tracking device.

Example two

FIG. 3 is a flowchart of a user distance estimation method provided in Embodiment 2 of the present application. This embodiment is optimized based on the foregoing embodiment. In this embodiment, the step of obtaining the eye image of the user is further optimized to receive the eye image of the user obtained by the user through a wired and / or wireless communication module. Based on this, the step of determining the distance between the user and the gaze tracking device through the eye image of the user is further optimized as follows: determining the light spot distance of the eyes in the eye image through the eye image Or binocular distance; determining the distance between the user and the gaze tracking device according to the spot distance and / or the binocular distance.

Accordingly, the method in this embodiment specifically includes the following execution steps:

S210. Receive a user's eye image obtained by a user terminal through a wired and / or wireless communication module.

The user terminal may be various users of operating electronic devices with a display platform, and the user terminal may be provided with an image acquisition device for acquiring a screen image of the electronic device display platform. According to an example, a user may wear a smart head-mounted device, such as smart glasses, which has an image acquisition device that can capture screen images of a display platform of an electronic device, and the image acquisition device may be a camera or a camera, etc. The device for the target screen image is only explained in the embodiment of the present application, and is not limited thereto.

Specifically, after acquiring the user's eye image, the collected eye image is sent to the wireless communication module of the gaze tracking device through the wireless communication module on the user side, that is, the electronic device on the user side and the display platform can be built in The wireless communication module performs wireless data transmission, so that the electronic device end of the display platform obtains the user's eye image collected by the user-side image acquisition device.

S220. Determine the spot distance or binocular distance of the eyes in the eye image through the eye image.

S230. Determine a distance between the user and the gaze tracking device according to the spot distance and / or the binocular distance.

A geographic coordinate system, also called a real-world coordinate system, is a coordinate system used to determine the location of features on the earth. A specific geographic coordinate system is composed of a specific ellipsoid and a specific map projection. The ellipsoid is a mathematical description of the shape of the earth, and the map projection is a mathematical method for converting spherical coordinates into plane coordinates. Most maps display coordinate data according to a known geographic coordinate system. For example, the 1:25 million topographic map of the country uses the Gauss-Kruger projection on the Krasovsky ellipsoid. The most commonly used geographic coordinate system is the latitude and longitude coordinate system. This coordinate system can determine the position of any point on the earth. It should be noted that the latitude and longitude coordinate system is not a flat coordinate system, because degrees are not standard length units and cannot be measured. Area length.

In the technical solution of the present application, based on the real world coordinate system, it is assumed that the distance from the user to the gaze tracking device is d, and the spot distance on the image plane of the spot in the same eye is defined as igd (inter glint distance) and binocular distance It is defined as ipd (interpupil distance) on the image plane.

Optionally, determining the distance between the user and the gaze tracking device according to the spot distance and / or the binocular distance includes:

Specifically, a mapping relationship between d and igd, d and ipd, or d and ipd and ipd can be established to estimate d. This mapping relationship may be a function (for example, the following function) or a mapping table.

d = f (igd)

d = f (ipd)

d = f (igd, ipd)

Among them, for most scenes and the distance d from the user to the gaze tracking device, there can be various functions, which are referenced according to the following formulas, which can be specifically:

Specifically, C is a constant. When the distance between the eyes of the user is smaller (that is, the distance between the eyes in the user's eye image is greater), the greater the distance between the user and the gaze tracking device, the user can decide whether to reduce or increase the user according to the actual needs. Distance to gaze tracking device.

or

d = k _n ipd ⁿ + ... + k ₁ ipd ¹ + C

Among them, n is the degree of the estimation equation, which can be 2, 3, 4, and so on.

Similarly, you can also refer to it by the following formula, specifically:

Specifically, C is a constant. When the spot distance of the spot in the same eye of the user is smaller, the distance between the user and the gaze tracking device is larger, and the user can decide whether to reduce or increase the distance from the user to the gaze tracking device according to the actual needs.

or

d = k _n igd ⁿ + ... + k ₁ igd ¹ + C

For the above k and c, it can be determined through empirical statistics on a large number of experimental objects, or can be derived through the positional relationship between the gaze tracking device and the spatial model of the user's eyeball.

It should be noted that the interval of ipd or igd, that is, the domain of the function, can be partitioned, and different functions are used for multiple intervals to obtain more accurate results.

It should be noted that when determining the distance between the user and the gaze tracking device, the method of determining the spot distance or the binocular distance can be specifically selected by the user according to the actual situation. This embodiment of the present application only explains this and does not This makes no restrictions.

In the embodiment of the present application, an eye image of a user is obtained, a distance between the user and the gaze tracking device is determined, and a position of the gaze tracking device body is adjusted according to the distance, where the gaze tracking device body includes an image Acquisition device and light source.

Example three

FIG. 4 is a structural diagram of a user distance estimation device provided in Embodiment 3 of the present application. This embodiment is applicable to a case where a distance between a user and a gaze tracking device is estimated.

As shown in FIG. 4, the user distance estimation device includes a wireless communication module 410 and an image processing module 420, where:

The wireless communication module 410 is configured to receive a user's eye image collected by a current user terminal;

The image processing module 420 is connected to the wireless communication module and is configured to determine a distance between the user and the gaze tracking device through an image of the eye of the user.

A user distance estimation device of this embodiment determines the distance between the user and the gaze tracking device by acquiring an eye image of the user, so as to accurately estimate the distance between the user and the gaze tracking device.

Based on the above embodiments, the wireless communication module 410 is configured to:

Based on the above embodiments, the image processing module 420 includes:

The distance determining unit is configured to determine a distance between the user and the gaze tracking device according to the spot distance and / or the binocular distance.

Based on the above embodiments, the distance determining unit is configured as:

The user distance estimation device provided by each of the foregoing embodiments can execute the user distance estimation method provided by any embodiment of the present application, and has the corresponding functional modules and beneficial effects of executing the user distance estimation method.

Embodiment 4

FIG. 5 is a schematic structural diagram of a device provided in Embodiment 4 of the present application. FIG. 5 shows a block diagram of an exemplary device 512 suitable for use in implementing embodiments of the present application. The device 512 shown in FIG. 5 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 5, the device 512 is represented in the form of a general-purpose computing device. The components of the device 512 may include, but are not limited to, one or more processors 516, a system memory 528, and a bus 518 connecting different system components (including the system memory 528 and the processor 516).

The bus 518 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor 516, or a local area bus using any of a variety of bus structures. By way of example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the enhanced ISA bus, the Video Electronics Standards Association (VESA) local area bus, and peripheral component interconnects ( PCI) bus.

The device 512 typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the device 512, including volatile and non-volatile media, removable and non-removable media.

System memory 528 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 and / or cache memory 532. The device 512 may further include other removable / non-removable, volatile / nonvolatile computer system storage media. For example only, the storage device 534 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 5 and is commonly referred to as a “hard drive”). Although not shown in FIG. 5, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, etc.) may be provided. Or other optical media). In these cases, each drive may be connected to the bus 518 through one or more data medium interfaces. The memory 528 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of the embodiments of the present application.

A program / utility tool 540 having a set (at least one) of program modules 542 may be stored in, for example, the memory 528. Such program modules 542 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data Each of these examples, or some combination, may include an implementation of a network environment. Program module 542 generally performs functions and / or methods in the embodiments described herein.

The device 512 may also communicate with one or more external devices 514 (eg, keyboard, pointing device, display 524, etc.), may also communicate with one or more devices that enable a user to interact with the device 512, and / or with the device that enables The device 512 can communicate with any device (eg, network card, modem, etc.) that is in communication with one or more other computing devices. This communication can be performed through an input / output (I / O) interface 522. Moreover, the device 512 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 520. As shown, the network adapter 520 communicates with other modules of the device 512 through the bus 518. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the device 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and Data backup storage system.

The processor 516 executes various functional applications and data processing by running a program stored in the system memory 528, for example, implementing a user distance estimation method provided in the embodiment of the present application. The method includes:

Obtaining an image of a user's eye;

Of course, those skilled in the art can understand that the processor may also implement the technical solution of the user distance estimation method provided by any embodiment of the present application.

Example 5

Embodiment 5 of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the user distance estimation method provided by the embodiment of the present application is implemented. The method includes:

Obtaining an image of a user's eye;

Certainly, the computer-readable storage medium provided in the embodiment of the present application is not limited to the method operations described above, and the computer program stored on the computer program can also perform the correlation in the user distance estimation method provided by any embodiment of the present application. operating.

The computer storage medium in the embodiments of the present application may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

The computer-readable signal medium may include a data signal in baseband or propagated as part of a carrier wave, which carries a computer-readable program code. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .

Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for performing the operations of this application may be written in one or more programming languages, or a combination thereof, including programming languages such as Java, Smalltalk, C ++, and also conventional Procedural programming language—such as "C" or similar programming language. The program code can be executed entirely on the user's computer, partly on the user's computer, as an independent software package, partly on the user's computer, partly on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider) Internet connection).

Note that the above are only the preferred embodiments of this application and the technical principles applied. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and that those skilled in the art can make various obvious changes, readjustments, and substitutions without departing from the scope of protection of this application. Therefore, although the present application has been described in more detail through the above embodiments, this application is not limited to the above embodiments, and without departing from the concept of this application, it may also include more other equivalent embodiments, and this application The scope is determined by the scope of the appended claims.

Claims

A user distance estimation method includes:

Obtaining an image of a user's eye;

A distance between the user and the gaze tracking device is determined by an image of the eye of the user.
The method according to claim 1, wherein the acquiring an eye image of a user comprises:

The user's eye image obtained by the user is received through a wired and / or wireless communication module.
The method according to claim 1, wherein the determining the distance between the user and the gaze tracking device through the eye image of the user comprises:

Determining the spot distance or binocular distance of the eyes in the eye image through the eye image;

A distance between the user and the gaze tracking device is determined according to the spot distance and / or the binocular distance.
The method according to claim 3, wherein determining the distance between the user and the gaze tracking device according to the spot distance and / or the binocular distance comprises:

The distance between the user and the gaze tracking device is determined by a function relationship or a mapping table of the spot distance and / or binocular distance and the distance.
A user distance estimation device includes a target screen, a gaze tracking device main body, an image acquisition device, and an infrared light source, and further includes:

A wireless communication module configured to receive a user's eye image collected by a current user terminal;

The image processing module is connected to the wireless communication module and is configured to determine a distance between the user and the gaze tracking device through an image of the eye of the user.
The apparatus according to claim 5, wherein the wireless communication module is configured to:

The user's eye image obtained by the user is received through a wired and / or wireless communication module.
The apparatus according to claim 5, wherein the image processing module comprises:

A spot distance or binocular distance determination unit, configured to determine, based on the eye image, a spot distance or binocular distance of an eye in the eye image;

The distance determining unit is configured to determine a distance between the user and the gaze tracking device according to the spot distance and / or the binocular distance.
The apparatus according to claim 7, wherein the distance determination unit is configured to:

The distance between the user and the gaze tracking device is determined by a function relationship or a mapping table of the spot distance and / or binocular distance and the distance.
A device includes:

One or more processors;

A storage device configured to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors enable the user distance estimation method according to any one of claims 1-4.
A computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the user distance estimation method according to any one of claims 1-4.