ES2809648A1 - SYSTEM AND PROCEDURE FOR OPTIMIZING IMAGE RENDERING BASED ON PUPIL MONITORING (Machine-translation by Google Translate, not legally binding) - Google Patents

Abstract

Image rendering optimization system and procedure based on pupil tracking. Constituted from a device that captures the image and the movement of the user's eyes, such as a camera, processes said information, and sends it to the image synthesis engine to optimize the rendering process, dedicating more resources (resolution, GPU and CPU power, video memory, screen resolution, etc.) in representing the area of the image that will affect the fovea of the retina, which is where the center of attention of the gaze is focused. (Machine-translation by Google Translate, not legally binding)

Description

DESCRIPTION

SYSTEM AND PROCEDURE FOR THE OPTIMIZATION OF THE RENDERING OF

IMAGES FROM PUPIL MONITORING

OBJECT OF THE INVENTION

The present invention refers, taking advantage of the fact that the density of receptors in the retina is concentrated in certain points of the retina of the eye, to a system capable of improving the vision of the image area where the focus of attention is located. , using the information of the orientation of the pupils to optimize the computing power of the rendering system of the image to be displayed in virtual reality glasses, monitors, screens, etc. at these points of maximum receptor density.

It solves the hitherto unsolved problem of improving the quality and optimizing the resources and performance of rendering systems and representation of images generated by numerical algorithms.

In most animals, the distribution of light receptors in the retina is such that in the areas where the center of attention of the image to be processed by the brain is focused, the density of these receptors increases, thus obtaining more detail. the observing subject in the area where his interest is centered. In humans this area is called the fovea and is about 2mm square in area; in fact, half of the information that the optic nerve transfers to the brain corresponds to that generated in the fovea.

The procedure itself uses a device to monitor the position of the pupil to determine which part of the synthesized field of view in the image to be represented is going to affect the fovea so that the image synthesis engine uses an appropriately greater amount of resources. (resolution, GPU and CPU power, video memory, screen resolution and / or VR glasses, etc.) in rendering and representing the area of the image that will affect the fovea of the retina. The rest of the image can be synthesized with less detail since the density of light receptors in the rest of the retina surface is lower.

The industrial application of this invention is therefore in the design and development of systems and devices for capturing movement and ocular position. combined with image rendering systems in training and / or training simulators, in games and streaming videos, and their optimization.

BACKGROUND OF THE INVENTION

Although no invention identical to that described has been found, we present below the documents found that reflect the state of the art related thereto.

Thus document ES2327633T3 refers to an installation for the detection of eyes that comprises: one or more light sources to emit light in directions towards the head of a user, a detector to receive light from the head of a user and to repeatedly capture images thereof, and an evaluation unit connected to the detector to determine the position and / or gaze direction of an eye, and arranged to determine, in an image captured by the detector, an area in which an image of an eye or images of eyes, the evaluation unit is arranged to, after having determined the area, control the detector to send to the evaluation unit the information about the successive images that only corresponds to the determined area of the image captured by the detector, and the detector is arranged to read information only from the portion of the detector surface that corresponds to the determined area, and therefore the data that it lu ego have to be sent to the evaluation unit. The facility referred to in said document focuses on eye detection, while the main invention seeks the orientation of the pupil to optimize image display systems and resources.

US6152563A1 proposes a gaze direction detection system that uses an infrared light emitting diode mounted coaxially with the optical axis and in front of the image lens of an infrared sensitive video camera to remotely record images from the computer operator's eye. Infrared light enters the eye and is absorbed and then re-emitted by the retina, causing a "bright eye effect" that makes the pupil brighter than the rest of the eye. It also results in an even brighter little glow shiny that forms on the surface of the cornea.The computer includes software and hardware that acquires a video image, digitizes it into a pixel array, and then analyzes the array to identify the location of the student's center in relation to the center of the flare. With this information, the software calibrates the system to provide a high degree of precision in determining the user's point of view. When combined with a computer screen and graphical user interface, the system can place the cursor at the user's point of view and then perform the various actions of clicking the mouse at the location on the screen where the user is located. fixed. In this case, it is an interface and cursor control system in a visualization medium through the orientation of the eye in which an infrared light beam is projected that registers a camera, however, it does not include any optimization module. of resources in renderings, as offered by the main invention.

DE19731301A1 describes a method for controlling microscopes and apparatus connected to them by viewing direction analysis. It also refers to an apparatus for measuring the direction of vision and an eyepiece accessory for carrying out said method. The aim of the invention is to provide a method, an apparatus for measuring the direction of vision and a fixture of the type described above that allows the microscope and any apparatus operating in the working area of said microscope to be controlled exclusively by means of the optical data from the observer's eyes without disturbing reflections or improper handling of the microscope. For this purpose, highly diffuse light with a wavelength between 800 and 1000 nm is used as infrared light that is generated coaxially around the observer's eye at the level of the eyepiece tube. Furthermore, in the beam path of the optical system at least one other infrared signal is generated which is independent of the place of production of the diffuse light and is directed to the eye of the observer, whose infrared signal has an intensity different from that of diffuse light and a wavelength between 800 and 1000 nm and is subjected to stray light to produce at least one additional corneal reflex. As in the previous case, the control of the orientation of the eye is used for a different task than the optimization of image rendering, specifically for the control of microscopes and devices associated with them.

US5638176A1 refers to an inexpensive eye tracking system that does not require head gear. The eye tracking system uses the interference fringes between corneal brightness and "red eye" retinal reflection to obtain an immune background spot signal and angular resolution for use as a pointing device for personal computers. Spectroscopy Tunable diode laser (eye safe) is used to measure the period and amplitude of Fabry-Perot fringes caused by interference between corneal brightness and the "red" reflection of the retina. Again, the system associated with the analysis of the orientation of the eye refers to a control device other than the one proposed by the main invention, being in this case the use as a pointing device for computers and computers.

Conclusions: As can be seen from the research carried out, none of the documents found solves the problems raised as the proposed invention does.

DESCRIPTION OF THE INVENTION

The system for optimizing the rendering of images from the tracking of the pupils object of the present invention is constituted from virtual reality glasses that have a device or means for monitoring the position of the pupil installed, and a processor that analyzes this information to, together with the rendering engine, use an appropriately greater amount of resources (resolution, GPU and CPU power, video memory, screen resolution and / or VR glasses, etc.) in rendering and rendering the area of the image that will affect the fovea of the retina, being able to dedicate less power to the rest of the image.

To carry out the synthesis of the images, the 3D rendering engine must synthesize two images per frame, one that occupies the entire field of view in low quality (detail) and one with the highest possible quality (detail) with the field of view reduced to part of the image that will hit the fovea and overlap the low-quality image in the appropriate area to hit the fovea. From this way the user will not perceive difference with an image of the total field of vision rendered at high quality (detail).

In a different embodiment, where the final objective is the optimization of an image signal of a streaming video or game, the information on the position of the pupils is sent to the streaming server, so that it is the one that optimizes the algorithms of video compression that it uses, so that the detail of the generated image is adapted to the area of the retina that will affect when observed.

In the case where the image reproduction medium such as a laptop, a smartphone or another type of device that integrates a screen and a camera, you can use said camera to track the pupil.

To improve the result of the observer's perception, an algorithm for extrapolation of the trajectory followed by the pupils can be used to predict the position of the image of greater detail and sufficiently extend the area of said image towards the position it will occupy in the / los next frame / s.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present description, some drawings are attached that represent a preferred embodiment of the present invention:

Figure 1: Conventional perspective view of a VR glasses with a camera for capturing the movements, position and orientation of the pupil.

Figure 2: Schematic view of a rendered image composed of two superimposed images from the optimization system object of the present invention.

The numerical references that appear in said figures correspond to the following constitutive elements of the invention:

1. VR glasses

2. Camera

3. Processor

4. Final image

5. Low-quality image that occupies the entire field of view

6. High quality image with the field of vision reduced to the part of the image that will affect the fovea.

DESCRIPTION OF A PREFERRED EMBODIMENT

A preferred embodiment of the system and procedure for optimizing the rendering of images from the tracking of the pupils object of the present invention, implanted in VR glasses by way of example, with reference to the numerical references, can be based on the development of the following technical stages:

i. - Gaze orientation capture.

A camera (2) installed in the virtual reality glasses (1) captures the image of the user's eyes, and a software associated with said camera (2) calculates through a processor (3) the movements, position and orientation of the pupil, and sends this information to the image synthesis engine.

ii. - Optimization of rendering resources.

The image synthesis engine takes the information from the previous stage and implements it in the rendering to be projected as a final image (4), dedicating more resources (resolution, GPU and CPU power, video memory, screen resolution, etc.) in representing the area of the image that will affect the fovea of the retina, in such a way that it performs the synthesis of two images (5, 6) per frame, one (5) that occupies the entire field of view in low quality and one (6) with the highest possible quality with the field of view reduced to the part of the image that will hit the fovea and overlap the low quality image in the appropriate area to hit the fovea.

The image represented by the number (5) of figure 2 can be rendered ignoring the area that will occupy in the image represented by the number 4 the image represented by the number 6 if ignoring said area implies a more efficient use of the resources available for rendering.

In the case of screens, it will be necessary to use a system that captures the distance to the observer.

Claims (3)

1.

- Procedure for optimizing the rendering of images from the tracking of the pupils, characterized by being developed in the following technical stages: i.- Gaze orientation capture. A camera (2) or other similar means of monitoring the position of the pupil installed in the virtual reality glasses (1) captures the image of the user's eyes, and a software associated with said camera (2) calculates through a processor (3) the movements, position and orientation of the pupil, and sends said information to the image synthesis engine. ii.- Optimization of rendering resources. The image synthesis engine takes the information from the previous stage and implements it in the rendering to be projected as a final image (4), dedicating more resources (resolution, GPU and CPU power, video memory, screen resolution, etc.) in representing the area of the image that will affect the fovea of the retina, in such a way that it performs the synthesis of two images (5, 6) per frame, one (5) that occupies the entire field of view in low quality and one (6) with the highest possible quality with the field of view reduced to the part of the image that will hit the fovea and overlap the low quality image in the appropriate area to hit the fovea. two.

- Procedure for optimizing the rendering of images from the tracking of the pupils, according to claim 1, where in the first stage an algorithm for extrapolation of the trajectory followed by the pupils predicts the position of the image with greater detail and sufficiently extends the area from said image to the position it will occupy in the next frame (s). 3.

- Procedure for optimizing the rendering of images from the tracking of the pupils, according to claim 1, where, when the final objective is the optimization of an image signal of a video or streaming game, the information of the position of the pupils is sent to the streaming server, so that be it the one that optimizes the video compression algorithms that it uses, so that the detail of the generated image is adapted to the area of the retina that will affect when observed.