RU2799119C1 - Device for displaying floating image and methods of its operation, system of interactive display of floating image, method of operation of interactive display system of floating image - Google Patents

Abstract

FIELD: optical engineering.

SUBSTANCE: invention is intended to create augmented reality devices that form three-dimensional floating images in free space. The floating image display device comprises an image source, an electronic control unit, a system with tuneable optical power, a projection module, and a waveguide system. The device is configured to form a flat floating image, a flat floating video, a three-dimensional floating image, a three-dimensional floating video.

EFFECT: possibility of forming a volumetric or non-volumetric floating image focused in free space, with a wide field of view, which can be viewed from several viewpoints.

20 cl, 2 dwg

Description

Technical field

The present invention relates to the field of optotechnics and is intended to create integrated optical devices, namely augmented reality devices that form three-dimensional "floating" images in free space.

Description of the Prior Art

The actively developing field of mobile technologies requires more and more original solutions with high information content and comfort. One idea that requires technical implementation is a floating image display with an increased field of view, which displays without an additional scattering medium. This display has the following requirements:

enlarged, color, high quality three-dimensional image;

wide field of view so that the image can be viewed from multiple viewpoints or by multiple users;

the image must be placed in front of the display plane, i.e. there must be a positive offset of the image;

no moving parts of the system;

secure and contactless user interface.

From the prior art, augmented reality glasses based on a waveguide, an input and output diffractive optical element (DOE) are known, in such systems the image field of view is rather small, and the image brightness is highly dependent on the viewing angle. Augmented reality glasses are also used based on an architecture containing many input, output and multiplying DOEs; in such systems, the field of view of the image increases. Also known are systems for displaying a "floating" image for mobile devices, in such systems the field of view of the image is increased compared to the field of view obtained in augmented reality glasses, and, in addition, the image can be viewed by several users at the same time. However, in such systems, the size of the hovering image itself is small, and when scaling it, it is difficult to achieve good brightness, image uniformity, and image quality.

From the prior art (document US 2019/0222828 A1, publication date 07/18/2019) a display system is known that contains a radially symmetrical mirror, a display screen and a radial lens array. Light from the screen can pass through the radial grille and then bounce off the mirror, creating a 360-degree auto-multiscopic display. The automultiscope display can display multiple rendered views of a 3D scene, each showing the scene from a different virtual camera angle. This well-known solution has large dimensions, moving parts, reduced resolution of the projected image due to the use of a raster system.

From the prior art (document US 20190285904 A1, publication date 09/19/2019) a three-dimensional visualization device based on an array of corner reflectors is known, which can transfer the primary display plane to free space. To visualize the image, the display of the mobile device is placed at an angle to the array of corner reflectors, the image is formed in free space. The disadvantage of the known device is that the array of corner reflectors operates with a small field of view, which does not exceed 20 degrees. Also, this well-known device forms an image with a low resolution, has large dimensions. In addition, the known solution does not provide an increase in the volumetric "floating" image.

From the prior art (document US 20170199496 A1, publication date 07/13/2017) a system is known for generating a sequence of images with different depths, containing a modulation matrix. The modulation matrix contains a plurality of light modulators that can shift the beam of light incident on the modulators by several degrees. The plurality of light modulators may shift the light beam in concert according to the modulation shift pattern. The modulation shift pattern can be configured to focus incident light on a voxel or to form a 3D image. One or more modulation shift patterns may be modified to rasterize one or more image objects in one or more image depth planes. The disadvantages of the known system is that a "floating" image with a positive offset is not formed, and different parameters of the input diffraction gratings are required to increase the field of view.

From the prior art (document US20190041634A1, publication date 02/07/2019) a holographic waveguide optical tracker is known. The tracker includes: light source; at least one waveguide optically coupled to said source; at least one detector waveguide comprising a diffractive optical element for introducing and deflecting a beam with a first polarization of light reflected from said object into a first direction of the waveguide and deflecting a beam with a second polarization of light reflected from said object into a second direction of the waveguide; at least one detector optically coupled to said detector waveguide, operable to receive light propagating in said first direction of the waveguide; and at least one detector optically coupled to said detector waveguide, operable to receive light propagating in said second direction of the waveguide. The disadvantages of the known device are a lot of additional elements, the lack of the possibility of forming an image in the air, that is, a three-dimensional "floating" image.

The problem to be solved by the present invention is to obtain a three-dimensional "floating" image with an enlarged field of view, and the three-dimensional "floating" image should be displayed in space without an additional scattering medium. It is necessary to obtain a high quality enlarged volumetric image, with a wide field of view, so that the image can be viewed from several points of view and / or several users. Moreover, the device for reproducing a three-dimensional "floating" image should not have moving parts and should have a safe and non-contact user interface.

The essence of the invention

A device for displaying a "floating" image is proposed, comprising:

image source;

electronic control unit;

a system with tunable optical power;

projection module;

waveguide system;

and

the image source is connected to the electronic control unit and is configured to store the digitized image in the memory and output the digitized image to the electronic control unit in the form of a signal containing the original image data and information about the distance from the "floating" image device on which the image should be formed, corresponding to the original;

an electronic control unit is connected to the variable power system and to the projection module, the electronic control unit is configured to separate said signal into a signal containing source image data and a signal containing voltage data, the value of which corresponds to information about said distance;

the projection module is optically matched to the waveguide system and is configured to convert the signal containing said source image data into a light field corresponding to the source image;

the waveguide system is optically matched with the tunable optical power system and is configured to multiply the light beams constituting said light field;

moreover, the system with a tunable optical power consists of a polarizer, an element with a first optical power, an element with a second optical power and a tunable optical element located between the said elements, and

the polarizer is configured to polarize the multiplied light beams emerging from the waveguide system in such a way that the direction of polarization of said light beams coincides with the direction of polarization of the tunable optical element,

wherein the element with the first optical power is configured to direct the polarized light beams that have passed through the polarizer to the tunable optical element;

the tunable optical element is configured to introduce a phase delay into the wave front of the transmitted light field, thereby changing the distance at which a "floating" image will be formed in space, under the influence of voltage supplied by the electronic control unit;

the element with the second optical power is configured to focus the mentioned light beams, constituting the light field corresponding to the original image, and emerging from the tunable optical element, in space, forming a "floating" image at a distance corresponding to the voltage applied to the tunable optical element.

Moreover, the element with the first optical power may be an element with a positive optical power, the element with the second optical power may be an element with a negative optical power. The optical power D _pos of the optical element with positive optical power can be related to the optical power D _Neg of the optical element with negative optical power as follows:

D _Pos ≈ -1.1 × D _Neg .

The first power element may be a negative power element, the second power element may be a positive power element. The first power element may be a positive power element and the second power element may be a positive power element. There may be no air gap between the element with the first optical power, the tunable optical element and the element with the second optical power. The source of the images may be the memory of the electronic device. The tunable optical element can be made from a layer of liquid crystals. The tunable optical element can be made of an optically active material that changes its optical properties as a result of stress. The image source may include a memory storing data about each slice of an image, including a digitized image of the slice and data about the depth of that slice.

Also proposed is a method for operating a floating image display device for displaying a flat floating image, comprising the steps of:

A) output, by means of an image source, a digitized original flat image, which enters the electronic control unit, and the digitized original flat image is a signal containing data of the original flat image and information about the distance at which the flat "floating" image should be formed, corresponding to the original flat image;

B) processing, by means of an electronic control unit, the said signal, dividing it into a signal containing the said data of the original flat image, and a voltage signal, the value of which corresponds to information about the distance of the display device of the "floating" image, on which a flat "floating" image should be formed image;

C) applying to the tunable optical element, by means of an electronic control unit, a voltage corresponding to the voltage signal;

D) submitting to the projection module, by means of an electronic control unit, a signal containing the said original flat image; and

steps C) and D) are carried out simultaneously;

E) converting, by means of a projection module, the data of the original flat image into a light field corresponding to the original flat image, projecting, by means of the projection module, said light field into the waveguide system;

E) multiplying, by means of a waveguide system, a set of light beams constituting said light field;

G) polarize, by means of a polarizer of the system with a tunable optical power, the multiplied light field emerging from the waveguide system;

H) a polarized light field is supplied to the element with the first optical power, then to the tunable optical element, and under the action of the mentioned voltage, the tunable optical element is tunable in such a way that the transmitted light field, which has passed the tunable optical element and the element with the second optical power, is formed in the form a flat "floating" image, corresponding to the original image, in space at a distance corresponding to the applied voltage.

A method is proposed for the operation of a device for displaying a "floating" three-dimensional image, which contains the steps in which:

A) rendering, by means of a CAD system, a digitized initial volumetric image into a sequence of digitized flat slices of a volumetric image,

wherein each digitized flat slice of the volume image is a signal containing image data of the flat slice of the volume image and information about the distance at which the "floating" image of the flat slice of the volume image should be formed,

transmitting a sequence of digitized flat slices as a sequence of signals to an image source;

B) transmitting said sequence of signals from the image source to the electronic control unit,

processing each signal from the sequence, by means of an electronic control unit, dividing into a signal containing image data of a flat slice of a volume image and a voltage signal, the value of which corresponds to information about the distance from the floating image display device on which the floating image of a flat cut volumetric image;

C) is applied to a tunable optical element, by means of an electronic control unit, sequentially with a time shift, with a frequency exceeding the resolution of recognizing images as separate for the observer:

voltages corresponding to voltage signals for the "floating" image of a flat slice of a volume image for each image of a flat slice of a volume image from the sequence;

D) are fed to the projection module, by means of an electronic control unit, sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for the observer:

signals containing 3D planar slice image data for each 3D planar slice image of the sequence;

and

steps C) and D) are carried out simultaneously;

sequentially with a time shift:

E) converting, by means of the projection module, the image data of the flat slice of the volume image for each image of the flat slice of the volume image from the sequence to the light field,

projecting through the projection module, each mentioned light field in the waveguide system;

E) multiplying, by means of a waveguide system, a set of light beams constituting each said light field;

G) polarize, by means of a polarizer of the system with a tunable optical power, each multiplied light field emerging from the waveguide system;

3) the polarized light field hits the element with the first optical power, hits the tunable optical element, and under the action of the said voltage, the tunable optical element is tunable in such a way that the light field that has passed the tunable optical element and the element with the second optical power is formed in the form "floating" image of a flat section of a three-dimensional image, in space at a distance corresponding to the applied voltage;

moreover, a sequence of "floating" images of flat sections of a three-dimensional image, in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a three-dimensional "floating" image for the observer.

The original volumetric image may be the original volumetric color image;

wherein each digitized flat slice of the volumetric image consists of a red (R) component, a green (G) component, and a blue (B) component;

wherein said volume planar slice image data is volume planar slice red image channel data (R), volume plane slice green image channel data (G), and volume planar slice blue image channel data (B),

wherein said signal containing image data of a flat slice of a volume image and information about a distance at which a "floating" image of a flat slice of a volumetric color image is to be formed includes:

a signal containing the red image channel data (R) of a flat slice of a three-dimensional color image,

a signal containing green image channel data (G) of a flat slice of a volumetric color image,

a signal containing blue image channel data (B) of a flat slice of a volumetric color image,

and

the distance from the floating image display device at which the floating image of the red image channel (R) of the flat slice of the volumetric color image should be formed,

the distance from the floating image display device at which the floating image of the green image channel (G) of the flat slice of the three-dimensional color image should be formed,

the distance from the floating image display device at which the floating image of the blue image channel (B) of the flat slice of the volumetric color image should be formed,

equal to the distance at which a “floating” color image of a flat slice of a volumetric color image should be formed, corresponding to the original volumetric color image;

wherein said voltage signal includes:

a voltage signal for the red image channel (R) of a flat slice of a volumetric color image, the value of which corresponds to information about the distance from the display device of the "floating" image, on which a "floating" image of the red image channel (R) of a flat slice of a volumetric color image should be formed,

a voltage signal for the green image channel (G) of a flat slice of a color volume image, the value of which corresponds to information about the distance from the floating image display device, on which a floating image of the green image channel (G) of a flat slice of a volumetric color image should be formed,

a voltage signal for the blue image channel (B) of a flat slice of a volumetric color image, the value of which corresponds to information about the distance from the floating image display device, on which a floating image of the blue image channel (B) of a flat slice of a volumetric color image is to be formed,

moreover, for each flat slice of a volumetric color image, steps (B)-(C) are repeated, and the sequence of "floating" images R, G, B of the components of flat slices of a volumetric color image in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms for the observer a three-dimensional "floating" color image.

A method is proposed for operating a floating image display device for displaying a floating 3D video, comprising the steps of:

A) rendering, by means of a CAD system, each digitized source volumetric image from a sequence of digitized source volumetric images constituting a video image onto a sequence of digitized flat image slices,

wherein each digitized flat image slice is a signal containing image data of the flat image slice and distance information at which the "floating" image of the flat image slice is to be formed,

store a sequence of digitized flat image slices in the image source,

B) transmitting a sequence of digitized flat image slices from the image source to the electronic control unit;

for each digitized original volumetric image:

C) processing each signal from a sequence of digitized flat image slices by means of an electronic control unit, dividing it into a signal containing image data of a flat image slice and a voltage signal, the value of which corresponds to information about the distance at which a “floating” image of a flat image slice should be formed ,

D) is applied to a tunable optical element, by means of an electronic control unit, sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for the observer:

voltages corresponding to voltage signals for a "floating" image of a flat image slice for each image of a flat image slice from the sequence of digitized flat image slices;

E) are fed to the projection module, by means of an electronic control unit, sequentially with a time shift, with a frequency exceeding the resolution of recognizing images as separate for the observer:

signals containing image data of a planar image slice for each image of a planar image slice from the sequence of digitized planar image slices;

and

steps D) and E) are carried out simultaneously;

sequentially with a time shift:

E) converting, by means of a projection module, image data of a flat image slice for each image of a flat image slice from a sequence of digitized flat image slices into a light field,

projecting through the projection module, each mentioned light field in the waveguide system;

G) multiplying, by means of a waveguide system, a set of light beams constituting each said light field;

3) polarizing, by means of a polarizer of the system with a tunable optical power, each multiplied light field emerging from the waveguide system;

I) the polarized light field hits the element with the first optical power, hits the tunable optical element, and under the action of the said voltage, the tunable optical element is tunable in such a way that the light field that has passed the tunable optical element and the element with the second optical power is formed in the form floating" image of a flat slice of the image in space at a distance corresponding to the applied voltage;

moreover, the sequence of "floating" images of flat slices of images from the sequence of digitized original images that make up the video in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a three-dimensional "floating" video for the observer.

The source volumetric image from the sequence of digitized source volumetric images constituting the video image may be the source volumetric color image from the sequence of digitized source volumetric color images constituting the color video image,

wherein each digitized flat slice of a color image consists of a red (R) component, a green (G) component, and a blue (B) component;

wherein said flat slice image data is red image channel data (R), green image channel data (G), blue image channel data (B);

wherein said signal containing image data of the planar slice of the image and information about the distance at which the "floating" image of the planar slice of the image is to be formed includes:

a signal containing the red image channel data (R) of a flat slice of a color image,

a signal containing green image channel data (G) of a flat slice of a color image,

a signal containing blue image channel data (B) of a flat slice of a color image;

and

the distance from the floating image display device at which the floating image of the red image channel (R) of the flat slice of the color image should be formed,

the distance from the floating image display device at which the floating image of the green image channel (G) of the flat slice of the color image is to be formed,

the distance from the floating image display device at which the floating image of the blue image channel (B) of the flat slice of the color image is to be formed,

equal to the distance at which a “floating” color image of a flat slice of a color image corresponding to the original color image should be formed;

wherein said voltage signal includes:

a voltage signal for the red image channel (R) of a flat slice of a color image, the value of which corresponds to information about the distance from the floating image display device, on which a floating image of the red image channel (R) of a flat slice of a color image is to be formed,

a voltage signal for the green image channel (G) of the flat slice of the color image, the value of which corresponds to the information about the distance from the floating image display device on which the floating image of the green image channel (G) of the flat slice of the color image is to be formed,

a voltage signal for the blue image channel (B) of the flat slice of the color image, the value of which corresponds to the information about the distance from the floating image display device on which the floating image of the blue image channel (B) of the flat slice of the color image is to be formed,

moreover, for each flat slice of the image from the sequence of digitized source color images that make up the video, steps (B) - (I) are repeated, and the sequence of "floating" images R, G, B of the components of flat slices of color images from the sequence of digitized source color images, constituting video, in space, transmitted with a frequency exceeding the resolution of recognizing images as separate for the observer, forms for the observer a three-dimensional "floating" color video.

A floating image interactive display system is also provided, comprising:

the proposed device for displaying a "floating" image;

beam splitter;

infrared detector;

an infrared waveguide located between the beam splitter and the waveguide system;

infrared illumination module;

a control module connected to an infrared detector and an electronic control unit;

and

the electronic control unit is connected to the infrared illumination module and is additionally configured to supply a control signal to the infrared illumination module;

the system with tunable optical power is additionally configured to collimate infrared radiation scattered by the user;

the waveguide system is transparent to infrared radiation;

the infrared illumination module is configured to illuminate the entire area of the "floating" image;

the beam splitter is configured to transmit scattered infrared radiation to an infrared detector;

the infrared detector is configured to receive scattered infrared radiation that has passed through the beam splitter and transmit it to the control module;

the control module is configured to recognize the fact of user interaction with the area of the "floating" image, as well as the place of interaction in the area of the "floating" image, and generate a command corresponding to the position of the place of interaction with the area of the "floating" image.

An infrared waveguide may be combined with a waveguide system. An infrared illumination module can be built into the projection module. The proposed system may further comprise an array of ultrasonic transmitters.

A method of operation of the proposed interactive display system of a "floating" image is proposed, containing the steps in which:

submitting, by means of the electronic control module, a control signal to the infrared illumination module;

illuminate, by means of the infrared illumination module, the area of the "floating" image with infrared radiation;

the user interacts with the plane of the "floating" image, due to the mentioned interaction, infrared radiation is scattered;

collimating the scattered infrared radiation, by means of a system with a tunable optical power;

directing collimated scattered infrared radiation through a waveguide system that is transparent to infrared radiation to an infrared waveguide;

output infrared radiation from the infrared waveguide and through the beam splitter is sent to the infrared detector;

detecting, by means of an infrared detector, scattered radiation and transmitting signals to the control module;

recognize, by means of the control module, the fact of user interaction with the area of the "floating" image, as well as the place of interaction in the area of the "floating" image;

form a command, by means of a control module, corresponding to the position of the place of interaction in the area of the "floating" image.

Brief description of the drawings

The above and other features and advantages of the present invention are explained in the following description, illustrated by drawings, in which the following is presented:

Fig. 1 schematically illustrates the construction of a device for displaying a three-dimensional floating image.

Fig. 2 illustrates a floating image interactive display system.

Detailed description of the invention

A device is proposed for forming a volumetric or non-volumetric "floating" image, focused in free space, which can be seen with the naked eye in the field of view (Field of View, FoV) at some distance from the display. The proposed technical solution combines the use of lenses with opposite optical powers and a tunable optical element between them, in addition, a two-channel user interaction system is provided that allows you to form an image in the visible range of the spectrum and interact with the user in the infrared (IR) range of the spectrum.

When using the proposed invention, the user can observe a real three-dimensional or non-volumetric image in space in a large field of view, the convenience of viewing the image by the user at a distance and the convenience of user interaction with the image are also increased. The proposed device for displaying a "floating" image displays a "floating" image without an additional scattering medium, while forming an enlarged high-quality image with a wide field of view, the image can be viewed from several viewpoints by one or more users. At the same time, the proposed device has no moving parts and has a safe and contactless user interface.

The present invention provides increased efficiency in the use of radiation directed from the projector, improved image uniformity regardless of the angle at which the user observes the image, excellent image quality, and the presence of a system for non-contact user interaction with the image.

The floating image display device is compact and thin, while the floating image is three-dimensional and large, for this, a system is used whose optical power can be adjusted, while displaying various image slices, that is, image frames formed in several planes, located at different distances from the display, while the observer has a feeling of three-dimensionality of the image. In addition, using the proposed system with tunable optical power, it is possible to form a high-quality color image by compensating for chromatic aberrations.

The following terms are used in describing the present invention:

The field of view of an optical system (angular field) is a cone of rays that have left the optical system and form an image at infinity (an optical term). The center of the field of view corresponds to the center of the hovering image, and the edge of the field of view corresponds to the edge of this image.

The exit pupil (or the pupil of the optical system) is a paraxial image of the aperture stop in image space, formed by the next part of the optical system in the forward path of the rays. This term is well-established in optics. The main property of the exit pupil is that all fields of the image exist at any point in it. By multiplying the exit pupil, one thereby increases its size without resorting to an increase in the longitudinal dimensions of the optical system. Classical optics makes it possible to increase the size of the exit pupil, but at the same time, the longitudinal dimensions of the optical system increase, while waveguide optics, due to the multiple reflection of beams of rays inside the waveguide, allows this to be done without increasing the dimensions of the system.

In FIG. 1 illustrates the construction of a device for displaying a three-dimensional "floating" image. The proposed device consists of an image source 1, an electronic control unit 2, a system 3 with a tunable optical power, a projection module 4, a waveguide system 5. The system 3 with a tunable optical power consists of a polarizer 6, an element with a first optical power 3a, an element with a second optical power force 3c and a tunable optical element 3b located between said elements 3a and 3c. The image source 1 is connected to the electronic control unit 2. The electronic control unit 2 is connected to the variable power system 3 and to the projection module 4. The projection module 4 is optically matched to the waveguide system 5. The waveguide system 5 is optically matched to the variable power system 3.

A device for displaying a "floating" image can be placed in the body of an electronic device (smartphone, computer, laptop, etc.), then the device for displaying a "floating" image can serve as a display of an electronic device, or simultaneously work with other types of displays, with this image source is the memory of the electronic device.

A device for displaying a three-dimensional "floating" image may be located outside the housing of the electronic device, then the memory of the electronic device acts as a source 1 of images. Connection to an electronic device can be carried out both wired and wirelessly. When a device for displaying a three-dimensional "floating" image is made outside the housing of an electronic device, all elements of the device for displaying a three-dimensional "floating" image are enclosed in a separate housing.

The initial three-dimensional image of a scene or object is modeled by the artist in an accessible CAD system (Computer-aided design). The result file from the CAD system is then loaded/transferred to the image source memory of the floating image display. In the CAD system, rendering is performed, that is, drawing (displaying) a 3D volumetric image of a scene or object onto flat parts of this volumetric image, which are called slices.

The CAD system is not included in the device for displaying a "floating" image and is a suitable means known from the prior art, the result of which is a file that contains a sequence of frames, audio tracks and other information necessary for playing the file, including the depth of the image or slice volumetric image. The data about each slice of the image includes a digitized image of the slice and data about the depth of this slice, that is, the distance from the floating image display device on which this slice is to be formed (projected). Slices of the volumetric "floating" image are flat. Creation of a three-dimensional image (3D model) can be carried out in any available development environment. The artist only needs to know the maximum restructuring of the element with tunable optical power. The result of the above-mentioned development environments (CAD systems) in the form of a file is stored in the memory of the electronic device and processed by the electronic control unit (ECU).

A file with data on a 3D model of a three-dimensional image of a scene or object, obtained as a result of rendering a 3D model in a CAD system, is loaded into the memory of the source 1 of the images, where it is stored, and when this three-dimensional image of the scene or object is reproduced, it enters the electronic control unit 2. In other words, a 3D image processed in the CAD system is a set of signals, each signal carrying information about one of the 3D image slices. Such information contains data about the slice, as a flat image of a part of the entire volumetric image, and about the depth, that is, about the distance from the display device of the "floating" image, on which the said flat image (slice) is to be formed.

CAD systems that convert volumetric images into a set of signals containing data about each slice as a flat image and about depth are known to those skilled in the art (for more details, see, for example, Stroud, Ian, and Hildegarde Nagy. Solid modeling and CAD systems: how to survive a CAD system, Springer Science & Business Media, 2011).

In the electronic control unit 2, the depth data is converted into voltage values that must be applied to the electrodes of the tunable optical element 3b with a tunable phase delay, so that the image that has passed through the system 3 with a tunable optical power is formed at the required distance from the display device "floating " Images. The voltage values applied to the electrodes are determined from the phase dependence of the voltage, which is characteristic of any optically active material, i.e. one that is capable of introducing a phase delay when the applied voltage changes as light propagates through it. When choosing an optically active material for the tunable optical element 3b, it is necessary to know the dependence of the phase delay of the light passing through the material on the voltage at the electrodes of the electrode structure. This entire process can be automated by standard algorithms well known in the art (see, for example, document US 20150277151 A1, publication date 10/01/2015). The range of possible cut depths, and hence the total depth of the three-dimensional image of a scene or object, is limited by the tuning range of the tunable optical element. The same restrictions (extreme depth positions) are also introduced into the CAD model of a three-dimensional image of a scene or object.

If the original image of a scene or object is a single flat image contained in the memory of the image source 1, then the signal transmitted from the image source 1 to the electronic control unit 2 contains flat image data with depth information, that is, the distance from the display device " "floating" image on which said flat image is to be formed. For a single image, any available distance from the tuning range of the tunable optical element 3b can be used. In addition, if the image is flat, then there is no need to go through the render stage to draw slices, since a flat image does not have slices. The depth of a single flat image from the possible range of the tunable optical element 3b is determined and set by the user who creates this image in the CAD system. Depth information is entered when a single flat image is created in the file.

It should be explained that the device for displaying a "floating" image is capable of reproducing both a single flat image and a three-dimensional image (that is, a sequence of slices thereof), or a sequence of such images for video playback. As soon as the playback file of a single image or a sequence of images (volumetric image slices or video) ends, the device finishes working with this file, and, if there is a request, opens the next file from the image source 1 memory.

The proposed floating image device can reproduce a single flat floating image, a flat floating video, a three-dimensional floating image, a three-dimensional floating video. The resulting "floating" image can be either monochrome or color.

The floating image display device operates as follows.

To reproduce a flat "floating" image in space, the following steps are carried out.

A) The image source 1 generates and outputs a digitized original image or outputs an image stored, for example, in the memory of an electronic device. The original image can be either color or monochrome. The digitized original image is fed to the electronic control unit 2. The digitized original image is a signal containing the original image data and information about the distance from the floating image device on which the image corresponding to the original is to be formed.

B) The electronic control unit 2 processes said signal, dividing it into a signal containing data of the original image, and a signal containing data on voltage, the value of which corresponds to information about the distance from the device on which the “floating” image corresponding to the original should be formed.

B) The electronic control unit 2 supplies the tunable optical element 3b with a voltage corresponding to the voltage signal. Under the action of said voltage, the tunable optical element 3b is retuned in such a way that the light field that has passed through the tunable optical power system 3 forms a "floating" image corresponding to the original image at a distance from the device corresponding to the applied voltage.

D) The electronic control unit 2 supplies the projection module 4 with a signal containing said image data.

Steps C) and D) are carried out simultaneously.

The electronic control unit 2 is a CPU (central processing unit). The electronic control unit 2 processes the received signal and separates it into a direct image for the projection module 4 and data for the variable power system 3, which is a voltage signal, the value of which corresponds to the depth information. Such signal processing and separation is known in the art. Examples of such signal processing and separation are known in the theory of data transmission in the concept of the Internet of Things (Internet of Things, IoT) (see source: Shinde G. R. et al. Internet of things augmented reality. - Springer, 2021). The processed depth information is a voltage value to be applied to the tunable optical element 3b of the tunable optical power system 3 at the moment when the projection unit 4 projects the corresponding image. When a voltage is applied to the tunable optical element 3b, the refractive index of the tunable optical element changes, and, as a result, the optical power of the system 3 with tunable optical power, thus changing the depth (distance) from the display device of the “floating” image, on which the “floating” image will be formed. " image.

At the same time, the electronic control unit 2 generates and transmits a signal to the projection module 4. The signal is a single image or a sequence of images without information about the depth of the image.

E) The projection module 4 converts the signal containing said original image data into a light field corresponding to the original image. The light field is a set of light beams that make up the original image, which propagate at different angles, and in each beam the rays propagate parallel to each other. Such a set of light beams emanating from the projection module 4 corresponds to the original image, which is projected into the waveguide system 5.

E) In the waveguide system 5, the said set of light beams is multiplied, i.e. the exit pupil aperture of the projection system 4 expands. Such waveguide systems, in which the exit pupil aperture of the projection system expands, are widely known in the prior art (see, for example, document US 10203762 B2, publication date 02/12/2019). In this way, a large field of view of the "floating" flat/volumetric image is achieved. After multiplication, the light beams that make up the light field are output from the waveguide system in an aperture significantly larger than the aperture of the exit pupil of the projection system. At the same time, the angular size of the original image, formed at infinity by the projection module, is preserved.

G) Next, the multiplied light field is directed from the waveguide to system 3 with a tunable optical power and enters polarizer 6. Polarizer 6 polarizes the multiplied light field that has emerged from waveguide system 5.

Polarizer 6 is positioned and oriented in such a way that a set of parallel beams passing through it acquires a polarization direction consistent (coinciding) with the direction of polarization of the tunable optical element 3b. It should be clarified that the tunable optical element 3b is made of a material that works only for light with a certain polarization, i.e. light with a different polarization cannot interact with the tunable optical element 3b. The tunable optical element 3b may be made of a layer of liquid crystals (liquid crystal cell), in which case the polarization is determined by the initial arrangement of the liquid crystals in the cell. Also, as an optically active material in tunable optical elements, polymer gels or other optically active materials that change their optical properties as a result of voltage can be used. Specific examples of optically active materials suitable for use in accordance with the invention will be apparent to those skilled in the art based on the information provided in the present description.

In the case of the proposed invention, the polarization can be arbitrary, the main thing is that the light that leaves the waveguide system 5 and passes through the polarizer 6 is such that the material of the tunable optical element 3b is able to process it in the way necessary for this invention. That is, the polarizer 6 matches the radiation coming out of the waveguide system 5 with the parameters of the tunable optical element 3b. Such polarizers are widely known in the art.

H) After polarization in the required plane, the light field hits the element 3a with the first optical power.

The first power element 3a and the second power element 3c may be lenses or lens systems.

The element 3a with the first optical power is located between the polarizer 6 and the tunable optical element 3b. The element 3c with the second optical power is located between the tunable optical element 3b and the user (observer).

Preferably, the first power element is a positive power element, then the radiation that has passed through the first power element will be focused. In this case, it is preferable that the element with the second optical power is the element with a negative optical power, then the radiation transmitted through the tunable optical element 3b will be scattered. The end result of the system 3 with tunable optical power will be the focusing of the radiation and the formation of a real image. Thanks to just such an arrangement, when changing the optical power of the system 3 with tunable optical power (when applying an appropriate voltage to the tunable optical element 3b), the maximum difference is achieved between the extreme positions of the focal plane of the system 3 with tunable optical power, that is, the most distant from the "floating" display device image and the position of the focal plane of the system 3 with tunable optical power closest to the device. Thus, the greatest depth scanning range of the volumetric "floating" image is achieved.

To achieve the maximum difference between the extreme positions of the focal plane, it is also necessary that the optical power D _pos of the optical element with positive optical power is related to the optical power D _Neg of the optical element with negative optical power as follows:

D _Pos ≈ -1.1 × D _Neg .

It is with this ratio that the maximum distance between the far and near focus of the entire system with tunable optical power is achieved, thus it is possible to create deep volumetric images, that is, with a large distance between the extreme reproducible slices in a “floating” volumetric image.

Mentioned optical elements with negative optical power and with positive optical power can be made of any suitable materials, for example, glass, plastic, and can also be diffraction gratings and holographic diffraction gratings, can also be metal lenses, diffractive lenses, liquid crystal lenses, geometric phase lenses, etc.

If a negative lens is placed in front of the liquid crystal layer, and a positive lens after the liquid crystal layer, then the average optical power of the system decreases, and the image is focused at a longer distance, then the user can be located far enough from the display to see the "floating" image.

It is possible to use two positive lenses, but the restructuring of the focal length in this case will be less than in the case described above, so the effect of the three-dimensional image will not show up well.

In one embodiment, in a tunable power system, there may be no air gap between the tunable optical element 3b and said optical elements 3a and 3c. In another embodiment, there may be an air gap between the tunable optical element 3b and the mentioned optical elements 3a and 3c, however, in this case, the focal length tuning depth of the entire system will be less than without a gap, and, consequently, a smaller depth of the volumetric image will be achieved.

The electronic control unit 2 applies a voltage to the tunable optical element 3b of the tunable optical power system 3 in accordance with the voltage signal (step B). Under the action of the applied voltage, the refractive index of the tunable optical element 3b changes, due to which, due to the properties of the material of the tunable optical element, the optical power of the system 3 with tunable optical power changes, that is, the distance to the “floating” flat image changes, i.e. there is a change in the depth of the "floating" image. Thus, the image is focused by the tunable optical power system 3 in a certain focal plane, i.e. at a certain distance from the floating image display device, which corresponds to the voltage applied to the tunable optical element 3b. In turn, the said voltage corresponds to the image supplied by the electronic control unit 2 to the projection module 4 and projected by the projection module, thus forming a flat image or one slice of a three-dimensional image in the form of a “floating” image in space at the mentioned distance from the “floating” display device. Images. That is, the voltage applied to the tunable optical element 3b determines the depth of the individual currently projected image or slice.

Using the method described above, it is possible to reproduce in space both a monochrome and a color flat "floating" image or slice. However, a color "floating" image will break up into red (R), green (G) and blue (G) components due to chromatic aberration, which will be formed at slightly different depths. The present invention can be carried out without correcting chromatic aberration, but to improve the quality of the color "floating" image, it is possible to correct chromatic aberration.

To reproduce in space a flat color "floating" image with corrected chromatic aberration, the following steps are carried out.

A) The image source 1 generates or outputs a digitized original color image stored, for example, in the memory of an electronic device. Moreover, the digitized color image is a signal containing red image channel data (R), green image channel data (G), blue image channel data (B), and information about the distance at which a "floating" color image should be formed, corresponding to original color image;

B) The electronic control unit 2 processes the mentioned signal, dividing it into the following signals:

signal containing red channel image data (R),

a signal containing green channel image data (G),

a signal containing blue channel image data (B),

a voltage signal for the red channel of the image (R), the value of which corresponds to information about the distance from the device on which the “floating” image of the red channel (R) of the digitized original color image should be formed,

a voltage signal for the green channel of the image (G), the value of which corresponds to information about the distance from the device on which the “floating” image of the green channel (G) of the digitized original color image should be formed,

a voltage signal for the blue channel of the image (B), the value of which corresponds to information about the distance from the device on which the "floating" image of the blue channel (B) of the digitized original color image should be formed.

And

the distance from the device at which the "floating" image of the red channel (R) of the digitized original color image should be formed,

the distance from the device at which the "floating" image of the green channel (G) of the digitized original color image should be formed,

the distance from the device at which the "floating" image of the blue channel (B) of the digitized source color image should be formed,

are equal to the distance at which a “floating” color image corresponding to the original color image should be formed.

C) The electronic control unit 2 supplies the tunable optical element sequentially with a time shift, with a frequency exceeding the resolution of recognizing images as separate for a person:

a voltage for the red image channel (R) corresponding to a voltage signal for the red image channel (R);

a voltage for a green image channel (G) corresponding to a voltage signal for a green image channel (G);

a voltage for the blue image channel (B) corresponding to a voltage signal for the blue image channel (B);

D) The electronic control unit 2 delivers to the projection module 4 sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for a person:

signal containing red channel image data (R),

a signal containing green channel image data (G),

a signal containing blue channel image data (B).

Steps C) and D) are carried out simultaneously.

E) The projection module 4 sequentially with a shift in time converts

a signal containing red image channel data (R) into the red image channel (R) light field

a signal containing green image channel data (G) into a green image channel (G) light field;

a signal containing blue image channel data (B) into the blue image channel (B) light field.

The light field of each of the image channels is a set of light beams that propagate at different angles, and in each beam the rays propagate parallel to each other. Such a set of light beams coming from the projection module is the original color R, G, B image.

E) The projection module 4 sequentially with a time shift projects

light field of the red image channel (R),

light field of the green image channel (G),

light field blue channel image(B)

into the waveguide system 5.

G) The waveguide system 5 multiplies the set of light beams constituting said light fields.

H) Polarizer 6 of system 3 with tunable optical power polarizes the R, G, B multiplexed light fields emerging from the waveguide system.

I) The system 3 with tunable optical power forms a "floating" image in space at a distance corresponding to the voltage applied to the tunable optical element 3b.

In this case, the voltage value for each of the R, G, B image components corresponds to the same distance at which a color "floating" image should be formed;

K) Next, steps (B) - (I) are repeated, while the R, G, B components of the original color image and their corresponding depth values remain constant during repetition.

To reproduce in space a three-dimensional "floating" image, both monochrome and color, the following steps are carried out.

A) The original three-dimensional image (monochrome or color) of a scene or object is modeled in a CAD system. Produce rendering (rendering) of the original three-dimensional image of the scene or object on digitized flat slices of the image. Moreover, the data on each slice includes a digitized image of the slice and data on the depth of this slice, that is, the distance at which this slice should be formed from the display. The result of the CAD system is a file of a digitized initial volumetric image, consisting of a sequence of digitized flat image slices. Each digitized planar image slice is a signal containing image data of the planar image slice and distance information at which the planar image slice should be formed to float. The file of the digitized original volumetric image is transferred to the image source 1 memory.

B) Said file, consisting of a sequence of digitized flat slices of a volumetric image, is transmitted from the image source 1 to the electronic control unit 2.

The electronic control unit 2 processes each signal from the above sequence, dividing into a signal containing image data of a flat slice of the image, and a voltage signal, the value of which corresponds to information about the distance from the device on which the "floating" image of the flat slice of the volumetric image is to be formed.

C) The electronic control unit 2 applies to the tunable optical element 3b, sequentially with a time shift, with a frequency exceeding the resolution of recognizing images as separate for a person:

voltages corresponding to the voltage signals for the "floating" image of the flat slice of the volume image for each image of the flat slice of the volume image from the sequence.

D) The electronic control unit 2 delivers to the projection module 4, sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for a person:

signals containing image data of a planar slice image for each planar slice image of a volumetric image in the sequence.

Stages C) and D) are carried out synchronously.

E) The projection unit 4 sequentially converts the time-shifted flat slice image data for each flat slice image of the volumetric image from the sequence to the light field. The light field is a set of light beams that propagate at different angles, and in each beam the rays propagate parallel to each other. Such a set of light beams emanating from the projection module 4 is an image of a flat slice of a three-dimensional image. The projection module 4 then projects each light field sequentially and time-shifted into the waveguide system 5;

E) The waveguide system 5 multiplies a set of light beams that make up each light field of the image of a flat slice of the image from the sequence.

G) By means of the polarizer 6 of the system 3 with a tunable optical power, the light field of each image from the sequence, which has emerged from the waveguide system 5, is polarized.

H) The polarized light field passes the element with the first optical power and hits the tunable optical element 3b, and under the action of the said voltage, the tunable optical element 3b is tunable in such a way that the light field that has passed the tunable optical element 3b and the element with the second optical power is formed into in the form of a real "floating" image of a flat slice of the image, in space at a distance corresponding to the applied voltage.

A sequence of "floating" images of flat slices of a three-dimensional image in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a three-dimensional "floating" image (monochrome or color) for the observer.

To reproduce in space a flat "floating" video image, both monochrome and color, the following steps are carried out.

A) The image source 1 generates or supplies a complete sequence of digitized source images constituting the video. The sequence enters the electronic control unit 2, each digitized original image from the sequence is a signal containing the data of the original image and information about the distance at which the "flying" video should be formed.

For each digitized source image from the sequence:

B) By means of the electronic control unit 2, said signal is processed, dividing into a signal containing said original image data, and a voltage signal, the value of which corresponds to information about the distance from the device on which the “floating” image is to be formed;

C) The electronic control unit 2 supplies the tunable optical element with a voltage corresponding to the voltage signal;

D) Also, the electronic control unit 2 supplies the projection module 4 with a signal containing said image data.

Steps C) and D) are carried out simultaneously.

E) In the projection module 4, the image data is converted into a light field corresponding to the original image. The projection module 4 projects the light field into the waveguide system 5.

E) In the waveguide system 5, the set of light beams constituting said light field is multiplied.

G) The multiplied light field leaving the waveguide system 5 hits the polarizer 6 of the system 3 with a tunable optical power, which polarizes the light field that has passed through it.

H) The polarized light field hits the element 3a with the first optical power, then hits the tunable optical element 3b. Under the action of the mentioned voltage, the tunable optical element 3b is rebuilt in such a way that the light field that has passed the tunable optical element 3b and the element 3c with the second optical power is formed in the form of a real “floating” image corresponding to the original image in space at a distance corresponding to the applied voltage .

The processed digitized original images from the sequence constituting the video image are fed from the electronic control unit at a frequency exceeding the resolution of recognizing images as separate for the observer, forming a "floating" video for the observer.

To reproduce in space a three-dimensional "floating" video image, both monochrome and color, the following steps are carried out.

A) The CAD system renders each digitized original volumetric image (monochrome or color) from a sequence of digitized source volumetric images constituting a video volumetric image onto a sequence of digitized flat slices of each volumetric image from the sequence. Each digitized planar image slice is a signal containing image data of the planar image slice and distance information at which the planar image slice should be formed to float.

The resulting sequence of digitized flat image slices can be stored in the image source 1.

As necessary, a sequence of digitized flat image slices is transmitted from the image source 1 to the electronic control unit 2.

Further, for each digitized original volumetric image:

B) The electronic control unit 2 processes each signal from the sequence of digitized flat slice images, dividing into a signal containing image data of the flat slice image and a voltage signal, the value of which corresponds to the distance information at which the "floating" image of the flat image slice should be formed.

C) Next, the electronic control unit 2 applies to the tunable optical element 3b of the system 3 with tunable optical power sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for the observer:

voltages corresponding to the voltage signals for the "floating" image flat slice image for each flat slice image of the sequence of digitized flat image slices.

D) The electronic control unit 2 delivers to the projection module 4, sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for the observer:

signals containing image data of a planar image slice for each planar image slice image of the sequence of digitized planar image slices.

Stages C) and D) are carried out synchronously.

Further, sequentially with a shift in time:

E) The projection unit 4 converts the image data of the planar image slice for each image of the planar image slice from the sequence of digitized planar image slices into a light field.

The projection module 4 then projects each said light field into the waveguide system 5.

E) In the waveguide system 5, the set of light beams constituting each said light field is multiplied.

G) By means of the polarizer 6 of the system 3 with a tunable optical power, each multiplied light field emerging from the waveguide system is polarized.

H) The polarized light field hits the element 3a with the first optical power, then hits the tunable optical element 3b. Under the action of the mentioned voltage, the tunable optical element 3b is rebuilt in such a way that the light field that has passed the tunable optical element 3b and the element 3c with the second optical power is formed in the form of a real "floating" image of a flat image slice, in space at a distance corresponding to the applied voltage.

The resulting sequence of "floating" images of flat slices of images from the sequence of digitized initial volumetric images that make up the video in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a "floating" volumetric video for the observer.

To reproduce in space a color volumetric "floating" image, the following steps are carried out.

A) The original color volumetric image of a scene or object is modeled in a CAD system. The initial color volumetric image of the scene or object is rendered onto a sequence of digitized flat slices of the color image, each digitized flat slice of the color image consists of a red (R) component, a green (G) component and a blue (B) component.

Moreover, each digitized flat slice of a color image is a signal containing red image channel data (R), green image channel data (G), blue image channel data (B) and information about the distance at which a “floating” image of a flat cut color image.

A sequence of digitized flat slices of a color image is transmitted as a sequence of signals to the image source 1.

From the image source 1, said sequence of signals is transmitted to the electronic control unit 2.

B) The electronic control unit 2 processes each signal from the sequence through the electronic control unit, dividing into the following signals:

a signal containing red image channel data (R) of a flat slice of a color image,

a signal containing green image channel data (G) of a flat slice of a color image,

a signal containing blue channel image data (B) of a flat slice of a color image,

a voltage signal for the red image channel (R) of a flat slice of a color image, the value of which corresponds to information about the distance from the device on which a "floating" image of the red image channel (R) of a flat slice of a color image is to be formed,

a voltage signal for the green image channel (G) of a flat slice of a color image, the value of which corresponds to information about the distance from the device on which a “floating” image of the green image channel (G) of a flat slice of a color image is to be formed,

a voltage signal for the blue image channel (B) of the flat slice of the color image, the value of which corresponds to the information about the distance from the device on which the "floating" image of the blue image channel (B) of the flat slice of the color image is to be formed.

At the same time, the distance from the device at which the “floating” image of the red image channel (R) of a flat section of a color image should be formed, the distance from the device at which a “floating” image of the green image channel (G) of a flat section of a color image should be formed, the distance from the device at which the "floating" image of the blue image channel (B) of the flat slice of the color image should be formed is equal to the distance at which the "floating" color image of the flat slice of the color image corresponding to the original color image should be formed.

C) The electronic control unit 2 supplies the tunable optical element 3b for each flat slice of the color image sequentially with a time shift, with a frequency exceeding the resolution of recognizing images as separate for the observer:

a voltage for a red image channel (R) corresponding to a voltage signal for a red image channel (R) of a flat slice of a color image;

a voltage for a green image channel (G) corresponding to a voltage signal for a green image channel (G) of a flat slice of a color image;

voltage for the blue image channel (B) corresponding to the voltage signal for the blue image channel (B) of the flat slice of the color image.

D) The electronic control unit 2 delivers to the projection module 4 for each flat slice of a color image, sequentially with a time shift with a frequency exceeding the resolution of recognizing images as separate for the observer:

signal containing red channel image data (R),

a signal containing green channel image data (G),

a signal containing blue channel image data (B).

Steps C) and D) are carried out simultaneously;

E) The projection module 4 for each flat slice of the color image converts sequentially with a shift in time:

a signal containing red image channel (R) data into a red image channel (R) light field of a flat slice of a color image;

a signal containing green image channel data (G) into a green image channel (G) light field of a flat slice of a color image;

a signal containing blue image channel data (B) into a blue image channel (B) light field of a flat slice of a color image.

E) The projection module for each flat slice of the color image projects sequentially with a shift in time:

light field of the red image channel (R) of a flat section of a color image,

light field of the green channel of the image (G) of a flat section of the color image,

light field of the blue image channel (B) of a flat section of a color image

into the waveguide system 5.

G) Waveguide system 5 for each flat slice of a color image multiplies:

light field of the red image channel (R) of a flat section of a color image,

light field of the green channel of the image (G) of a flat section of the color image,

light field of the blue image channel (B) of a flat slice of a color image.

H) Polarizer 6 for each flat slice of a color image polarizes:

multiplied light field of the red image channel (R) of a flat section of a color image,

multiplied light field of the green image channel (G) of a flat slice of the color image,

the multiplexed light field of the blue image channel (B) of a flat slice of a color image.

I) Each said polarized light field sequentially with a shift in time hits the element 3a with the first optical power, from which it hits the tunable optical element 3b.

Under the influence

voltage for the red channel of the image (R) of a flat section of a color image,

the tunable optical element is tunable in such a way that

the light field of the red image channel (R) of the flat slice of the color image, which has passed the tunable optical element 3b and the element 3c with the second optical power, is formed in the form of a real "floating" image of the red (R) component of the flat slice of the color image, in space at a distance corresponding to applied voltage;

the light field of the green channel of the image (G) of the flat slice of the color image, which has passed the tunable optical element and the element with the second optical power, is formed in the form of a real "floating" image of the green (G) component of the flat slice of the color image in space at a distance corresponding to the applied voltage;

the transmitted light field of the blue image channel (B) of the flat slice of the color image, which has passed through the tunable optical element and the element with the second optical power, is formed in the form of a real “floating” image of the blue (B) component of the flat slice of the color image in space at a distance corresponding to the applied voltage .

Moreover, the "floating" image of the red (R) component of the flat section of the color image, the "floating" image of the green (G) component of the flat section of the color image, the "floating" image of the blue (B) component of the flat section of the color image are formed sequentially with a time shift on the same distance.

K) Repeat steps (B)-(I) for each planar slice of the color image. A sequence of "floating" images R, G, B of the components of flat sections of a color image in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a color volumetric "floating" image for the observer.

To reproduce in space a color "floating" volumetric video image, the following steps are carried out.

A) The CAD system renders each digitized source color volume image from a sequence of digitized source color volume images constituting a video image onto a sequence of digitized flat color image slices. Each digitized flat slice of a color image consists of a red (R) component, a green (G) component, and a blue (B) component.

Each digitized flat slice of a color image is a signal containing red image channel data (R), green image channel data (G), blue image channel data (B), and information about the distance at which the flat slice “floating” image should be formed. color image.

Save the sequence of digitized flat slices from the CAD system as a sequence of the mentioned signals in the image source 1. As needed, from the source 1 of images, a sequence of digitized flat slices is fed into the electronic control unit 2.

For each digitized original color volumetric image:

B) In the electronic control unit 2, each signal from the sequence of digitized flat slices is divided into the following signals:

a signal containing red image channel data (R) of a flat slice of a color image,

a signal containing green image channel data (G) of a flat slice of a color image,

a signal containing blue channel image data (B) of a flat slice of a color image,

a voltage signal for the red image channel (R) of a flat slice of a color image, the value of which corresponds to information about the distance from the device on which a "floating" image of the red image channel (R) of a flat slice of a color image is to be formed,

a voltage signal for the green image channel (G) of a flat slice of a color image, the value of which corresponds to information about the distance from the device on which a “floating” image of the green image channel (G) of a flat slice of a color image is to be formed,

a voltage signal for the blue image channel (B) of the flat slice of the color image, the value of which corresponds to the information about the distance from the device on which the "floating" image of the blue image channel (B) of the flat slice of the color image is to be formed.

The distance at which the red image channel (R) of the flat slice of the color image should be formed to float, the distance at which the green channel of the image (G) of the flat slice of the color image should be formed, the distance at which formed a "floating" image of the blue image channel (B) of a flat slice of a color image, are equal to the distance at which a "floating" color image of a flat slice of a color image corresponding to the original color image should be formed.

C) The electronic control unit 2 delivers to the tunable optical element 3b, for each flat slice of the color image, sequentially with a time shift and with a frequency exceeding the resolution of recognizing images as separate for the observer:

a voltage for a red image channel (R) corresponding to a voltage signal for a red image channel (R) of a flat slice of a color image;

a voltage for a green image channel (G) corresponding to a voltage signal for a green image channel (G) of a flat slice of a color image;

voltage for the blue image channel (B) corresponding to the voltage signal for the blue image channel (B) of the flat slice of the color image.

D) Also, the electronic control unit 2 delivers to the projection module 4 for each flat slice of a color image sequentially with a shift in time with a frequency exceeding the resolution of recognizing images as separate for the observer:

signal containing red channel image data (R),

a signal containing green channel image data (G),

a signal containing blue channel image data (B).

Stages C) and D) are carried out synchronously.

E) The projection module 4 for each flat slice of the color image is converted sequentially with a shift in time

a signal containing red image channel (R) data into a red image channel (R) light field of a flat slice of a color image;

a signal containing green image channel data (G) into a green image channel (G) light field of a flat slice of a color image;

a signal containing blue image channel data (B) into a blue image channel (B) light field of a flat slice of a color image.

E) Next, the projection module 4 projects for each flat slice of the color image sequentially with a time shift

light field of the red image channel (R) of a flat section of a color image,

light field of the green channel of the image (G) of a flat section of the color image,

light field of the blue image channel (B) of a flat section of a color image

into the waveguide system 5.

G) In the waveguide system 5 is multiplied

light field of the red image channel (R) of a flat section of a color image,

light field of the green channel of the image (G) of a flat section of the color image,

light field of the blue image channel (B) of a flat slice of a color image.

H) With the help of a polarizer 6, systems 3 with a tunable optical power are polarized:

multiplied light field of the red image channel (R) of a flat section of a color image,

multiplied light field of the green image channel (G) of a flat slice of the color image,

the multiplexed light field of the blue image channel (B) of a flat slice of a color image.

I) Each said polarized light field sequentially with a shift in time hits the element 3a with the first optical power, from which it hits the tunable optical element 3b. Under the influence

voltage for the red channel of the image (R) of a flat section of a color image,

the tunable optical element 3b is tunable in such a way that

the light field of the red image channel (R) of the flat slice of the color image, which has passed the tunable optical element 3b and the element 3c with the second optical power, is formed in the form of a real "floating" image of the red (R) component of the flat slice of the color image in space at a distance corresponding to the given tension;

the light field of the green channel of the image (G) of the flat section of the color image, which has passed the tunable optical element 3b and the element with the second optical power of 3c, is formed in the form of a real “floating” image of the green (G) component of the flat section of the color image in space at a distance corresponding to the given tension;

the transmitted light field of the blue image channel (B) of the flat slice of the color image, which has passed the tunable optical element 3b and the element 3c with the second optical power, is formed as a real "floating" image of the blue (B) component of the flat slice of the color image in space at a distance corresponding to applied voltage.

At the same time, the “floating” image of the red (R) component of the flat section of the color image, the “floating” image of the green (G) component of the flat section of the color image, the “floating” image of the blue (B) component of the flat section of the color image are formed sequentially with a time shift of the same distance.

J) For each flat color image slice from the sequence of digitized source color volume images constituting the volume video, steps (B)-(K) are repeated, with the sequence of floating images R, G, B components of flat color image slices from the sequence of digitized source color volumetric images that make up volumetric video in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a color “floating” volumetric video for the observer.

If the depth of all slices is the same, the three-dimensional effect is lost. If one image is used at one depth, then the user will see a "floating" flat image. If you use a sequence of images formed at the same depth, then the user will see a flat "floating" video. If you use a sequence of images that make up the same scene at different depths, and each depth has its own image, then the user will see a "floating" three-dimensional image. If you use a sequence of images of different scenes at different depths, and such a sequence should contain for each scene a sequence of images of this scene at different depths, then the user will see a "floating" three-dimensional video.

A “floating” three-dimensional image is formed by rapidly changing the focal length, that is, by changing the optical power of the system 3 with a tunable optical power, synchronously with the change of the corresponding images projected from the projection module 4. In other words, if you switch images from the image source simultaneously with the supply of the corresponding voltages on the tunable optical element, which move the images to the appropriate distances from the user, it is possible to form a sense of a three-dimensional image for the user (observer).

It should be noted that when a liquid crystal cell is used as a tunable optical element, with a smooth change in the voltage on the liquid crystal cell, for example, along a sinusoid, the same smooth rotation of the liquid crystals in the cell occurs. When the voltage changes abruptly, for example from minimum to maximum and then back to minimum, the system will have some delay as the crystals take time to turn. That is, the electronic control unit has voltage information from the signal from the image source 1. The signal from the image source 1 enters the electronic control unit in such a way that the voltage on the LCD cell changes smoothly, and not abruptly.

It should be noted that if the system 3 with tunable optical power is not used in the device, then the color image will fall into three planes, that is, each of the RGB colors (red, green, blue) will be focused in its own separate plane due to chromatic aberration.

That is, the image, having passed the waveguide system, breaks up into three images R, G, B, which are located in different planes. Further, in order to restore a single image, these R, G, B images must be supplied with a shift in time, during which the system is rebuilt with a tunable optical power. That is, the electronic control unit, in the case of a color image, receiving a signal from an image source, divides the signal into image data that is supplied to the projection module, and into a signal containing information about the amount of voltage that needs to be applied to the tunable optical element. Additionally, the signal for the tunable optical element and the signal for the projection module are each further divided into 3 signals, since there are 3 colors in the image, namely R, G, B, i.e. these three signals for the tunable optical element are slightly different voltages, and for the projection module, the corresponding image voltages are R, G, B. In this case, the focal length of the variable optical power system is set under each image of each of the RGB colors in such a way that the images are combined. With the same optical power of the tunable optical power system, the components (RGB, that is, the three images R, G, B) of the color image are formed at different distances from the display (at different depths), i.e. they are scattered in space. The frequency of operation of the electronic control unit should be increased by 3 times, that is, in the same unit of time as before, the electronic control unit sends to the projection system R, G and B components of the same image separately with a time shift , and 3 voltages are applied to the tunable optical element with the same time shift, which correspond to the same distances from the display, on which the “floating” R, G and B image components are formed. Thus, previously spatially spaced R, G, B components of one image are combined in one plane.

For example, to achieve the effect of combining the RGB components of one volumetric "floating" image from 10 slices (depth planes) at a frequency of formation of each slice of 24 Hz (24 Hz is a generally accepted frequency that exceeds the ability to recognize images as separate for a person), each of the 10 slices (depth planes ) is derived for the three primary colors, with each color encoded with at least 4 bits to produce 16 gradations of brightness for each color. Thus, it is necessary to have an operating frequency of the entire device of at least 2880 Hz.

For a better understanding of the essence of the invention, we give the following example.

There is one volumetric image (3D model in the CAD computer modeling system), which is divided into slices at 10 depths by rendering. That is, there are 10 slices that correspond to one volumetric "floating" image. If the image is not RGB (the image is not color), then each of the 10 frames is supplied to the projection module, and 10 different voltages corresponding to each of the frames are applied to the tunable optical power system. If the image is RGB (color image), then each frame out of 10 is decomposed into RGB components (i.e. into 3 separate frames, 30 frames in total) and fed to the projection module. For each of 30 frames, the corresponding voltage (30 voltage values) is supplied to the tunable optical element, and the voltage is such that the R, G, B components of one slice of the volume “floating” image obtained after the waveguide system are formed in the same plane. There are 10 such planes, and in each plane R, G, B, the image components are combined.

When projecting a “floating image”, as a result of dispersion, namely, as a result of the difference in the refractive indices of all optical materials of the device for different wavelengths (different colors), longitudinal and transverse chromatic aberrations appear, distorting the “floating image”. Such distortions are corrected as follows. As described above, the projection module projects the color image or video received from the electronic control unit as a sequence of red (R, frame #1) images, green (G, frame #2) images, and blue (B, frame #3) images with a certain frequency, which together constitute the corresponding displayed slice of the volumetric "floating" image. As described above, the frequency should be such that the slice rate of the volumetric "floating" image exceeds the ability of the images to be recognized as distinct by the user. To correct chromatic aberrations, the electronic control unit instructs the tunable power system at the same frequency to change the power from D(R) to D(G) and then to D(B), where D(R)> D(G)> D(B) to combine and focus all the R, G and B components of the image at a certain depth and thereby eliminate the effect of aberrations. To do this, the frame rate of the projection module must be equal to the product

the frame rate of the image/video received from the image source,

the number of displayed depth planes, determined by the resolution of the volumetric image in depth,

the number of primary colors (i.e. red, blue, green, so 3).

For example, for a frequency of 24 Hz, 10 depth planes/volumetric slices and 3 primary colors (R, G, B), and a reasonable choice of color depth (for example, 4 bits), the frame rate of the projector should be 2880 Hz , which is achievable for modern projectors. Color in computer image processing is encoded in bits. 4 bits means that each image pixel can take on any intensity value in the range from 0 to 15 intensity gradations of a given color, where 0 corresponds to the minimum intensity, and 2 ⁴ -1 (i.e. 15) corresponds to the maximum intensity of the given color. The final information capacity of the image in bytes depends on the color depth. Thus, for the types of projectors listed above, the bandwidth of the data transmission channel allows the transmission and reproduction of color images with depth, for example, 12 bits (4 bits x 3 colors) for the entire image: for a frame rate of 24 frames per second, x 10 depth planes, x 3 colors x 4 frames (bit-plane)/color, a frame output rate of 2880 frames per second is required for pulse-width modulation of the intensity of a full-color image, for example, the DMD projection system known from the prior art operates at a frequency of up to ~ 16 kHz, and known from of the state of the art, the FLCoS projection system operates at frequencies up to ~6 kHz).

The size of the hovering image depends on the refractive power of the tunable optical power system and the R, G, B emission wavelength. color volumetric image in all sections. The lower the optical power of the tunable optical power system, the larger the image that must be reduced to obtain a three-dimensional image with a constant slice size, and vice versa. The longer the wavelength of the incident radiation, the larger the image to be reduced, and vice versa. Such scaling of R, G, B images is well known in the art.

The proposed device operates in the optical power ranges of optical elements with positive and negative optical power (in particular, lenses) that are part of a system with tunable optical power. One of the main parameters of a system with tunable optical power is the ratio of the optical powers of the mentioned optical elements (lenses). Calculations show that the greatest depth of the volume "floating" image is obtained when the mentioned ratio is approximately 1.1 (either -1.1 or 1.1 modulo). Based on the mentioned optimal ratio, the thickness of the liquid crystal layer is calculated in the case of using a liquid crystal cell as a tunable optical element.

When calculating the proposed device, the following are determined:

- aperture size,

- f-number (f-number) (a value showing the ratio of the focal length to the maximum aperture size, f-number=f/D),

- the ratio of the optical powers of the two mentioned optical elements (lenses), should preferably belong to the range from - 1.05 to - 1.15,

- used in a tunable optical element optically active material, the choice of which in this invention is determined by the value of Δn optical anisotropy (anisotropy of the refractive indices). In this case, the higher the value of optical anisotropy, the greater the value of the restructuring of the focal length of the optical system can be achieved,

is the thickness of the optically active material layer.

Calculations are made on the basis of relationships known from the prior art based on matrix optics.

The key relationships are as follows.

For the thickness of the optical active material layer:

(1)

(1)

For the amount of restructuring of the optical system (the maximum difference in focal lengths that the optical system can focus on):

(2)

(2)

For the magnitude of the optical power of a lens with a positive optical power:

(3)

(3)

Where

,

,

Wherein:

D ₁ - the value of the optical power of the element (lens, lens system) with a positive optical power,

D ₂ - the value of the optical power of the element (lens, lens system) with a negative optical power,

- толщина слоя оптически активного материала перестраиваемого оптического элемента,

is the thickness of the layer of optically active material of the tunable optical element,

- величина перестройки (изменения) фокусного расстояния оптической системы,

- the amount of restructuring (change) of the focal length of the optical system,

- первоначальное расстояние (до перестройки), на котором фокусируются лучи, проходящие через оптическую систему,

is the initial distance (before tuning) at which the beams passing through the optical system are focused,

- расстояние, на котором фокусируются лучи вследствие перестройки оптической системы,

- the distance at which the beams are focused due to the restructuring of the optical system,

- показатели преломления оптически активного материала, при переключении между которыми вследствие изменения напряжения, приложенного к перестраиваемому оптическому элементу, происходит перестройка фокусного расстояния всей оптической системы. Например, для случая жидких кристаллов в качестве

принимаются показатели преломления обыкновенного и необыкновенного лучей.

- refractive indices of the optically active material, when switching between which, due to a change in the voltage applied to the tunable optical element, the focal length of the entire optical system is rebuilt. For example, for the case of liquid crystals as

the refractive indices of the ordinary and extraordinary rays are taken.

- величины, характеризующие величину изменения показателя преломления оптически активного материала перестраиваемого оптического элемента;

- values characterizing the magnitude of the change in the refractive index of the optically active material of the tunable optical element;

- отношение оптических сил элементов (линз, систем линз) с фиксированной оптической силой.

- the ratio of the optical powers of elements (lenses, lens systems) with a fixed optical power.

It should be noted that an increase in the thickness of the layer of an optically active material (for example, liquid crystals) leads to an increase in the tuning range. The thicker the layer of liquid crystals, the greater will be the degree of restructuring of the optical system, i.e. change in the focal length at which the focusing of the rays passing through the system. Increasing the degree of adjustment will result in a "deeper" or three-dimensional image resulting from such adjustment of the focal length.

To find the thickness of an optically active material according to formula (1), it is necessary to choose a material, preferably with the highest optical anisotropy of the material (liquid crystals); choose the ratio of optical powers of elements (lenses, lens systems) with fixed optical powers, preferably from the range from -1.05 to -1.15; select the amount of necessary restructuring (changes in focal length) of the optical system, which is determined by the necessary sense of depth of the 3D image.

The specific order of choosing the parameters in formulas (1)-(3) does not affect the final result, namely, the image quality and the sense of depth of the 3D object, i.e. its realism. When choosing parameters outside of the preferred ranges, the invention will work, but possibly with worse image quality and depth perception of the 3D object.

To find the value of the optical restructuring of the optical system according to formula (2), it is necessary to choose a material, preferably with the highest optical anisotropy of the material (liquid crystals); choose the ratio of optical powers of elements (lenses, lens systems) with fixed optical powers, preferably from the range - 1.05 to -1.15; choose the required thickness of the layer of optically active material, which is determined by the form factor of the system and the technological capabilities of creating a tunable optical element with a selected thickness.

To find the magnitude of the optical power of elements (lenses, lens systems) with fixed optical powers according to formula (3), it is necessary to choose a material, preferably with the highest optical anisotropy of the material (liquid crystals); choose the value of the ratio of optical powers of lenses with fixed optical powers, preferably from the range from -1.05 to -1.15; choose one of the values of the focal length at which the rays that have passed through the optical system will be focused, which must certainly be at the boundaries of the optical tuning range of the system with a tunable optical power. For example, if liquid crystals with a certain amount of optical anisotropy have been chosen as the optically active material, then it is preferable to choose a focal length value corresponding to the refractive index of the ordinary ray or the refractive index of the extraordinary ray for these liquid crystals.

In liquid-crystal tunable optical cells, the "restructuring" of the focus is carried out with the help of electrodes that make up the electrode structure in each tunable optical element.

The mechanism of “restructuring” of electrodes is based on two principles.

In accordance with the first principle, automatic selection of addressable electrodes is implemented, i.e. those electrodes in the electrode structure of a tunable optical element, to which the voltage corresponding to them is applied. Automatic selection of addressable electrodes is associated with the choice of the required optical power. The optical power depends on the number of Fresnel zones, i.e. addressable electrodes are selected depending on the number and location of the Fresnel zones activated by them. Here it is necessary to clarify that the formation of Fresnel zones is determined by the shape, size and location of the electrodes, as well as the value of the voltage applied to these electrodes. Fresnel zones are regions into which the surface of a light wave can be divided in order to calculate the results of light diffraction. After the passage of light through an optical element having optical power, the surface of the light wave can be divided into Fresnel zones, the number and size of which corresponds to the optical power of this optical element. The method for calculating Fresnel zones and calculating the optical power of a diffractive lens is described in the source RU 2719341 C1 (publication date 04/17/2020). Thus, the optical power and efficiency of an optical element based on liquid crystals is primarily determined by the size, shape, location of the electrodes and the voltage applied to them, and the methods for calculating, arranging, choosing the material of the electrodes are known from the prior art (for more details, see, for example, in source RU 2719341 C1 (publication date 04/17/2020).

In accordance with the second principle, the values of the voltages applied to the electrodes are determined from the dependence of the voltage on the phase characteristic of any optically active material (i.e., one that is capable of introducing a phase delay when the applied voltage changes when light propagates through it). When choosing an optically active material for a tunable optical element, it is necessary to know the dependence of the phase delay of the light passing through the material on the voltage at the electrodes of the electrode structure. Then, in order to simulate the introduction of a certain optical power, it is necessary to apply voltages to the electrodes in such a way that the phase delay profile of the output light corresponds to that of an ideal thin lens with the same optical power. This entire process can be automated by standard algorithms well known in the art (for more details see, for example, US 20150277151 A1, publication date 01.10.2015).

In the present invention, the restructuring of the focal length of a tunable optical element with an optically active substance is implemented on the basis of the second principle. Hereinafter, unless otherwise indicated, by the restructuring of a system with a tunable optical power we mean the restructuring (i.e., a change in a certain range) of the focal length (or optical power, which is equal to the reciprocal of the focal length), at which this system with a tunable optical force focuses the rays of a certain range of wavelengths passing through it.

To create an electric field, which is necessary to change the refractive index of a tunable optical element, and, as a result, change the optical power of the entire system with a tunable optical power, an electrode coating is used. The coating can be applied in the form of a one-dimensional coating, strips, circles, and in the general case, the coating can have any arbitrary shape to change the refractive index of the tunable optical element (for example, in liquid crystals under the electrode, the electric field is stronger than in the space of liquid crystals above which no electrode).

By way of example, and not limitation, the electrodes in the electrode structure of each of the tunable optical cells may be made of indium tin oxide (ITO). In other embodiments, the electrodes may be made from other transparent conductive materials widely known to those skilled in the art (eg, indium oxide, tin oxide, indium zinc oxide (IZO), zinc oxide).

The electrode is applied to a substrate that is transparent in the visible wavelength range and is usually made of glass or plastic. Moreover, the tunable optical element consists of two substrates, where an electrode is deposited on one of the surfaces of each substrate. The optically active layer is located between the surfaces of the substrates on which the electrodes are deposited.

As a layer of liquid crystals, it is possible to use a single cell of liquid crystals, while the layer of liquid crystals is divided into smaller cells, that is, instead of one large cell, a mosaic of small ones is used. This partitioning occurs in production, in ways known in the art, like pixels in a conventional display. Such cells are needed to obtain the necessary properties, for example, ease of management. Managing each cell individually is easier than managing one large cell. At the same time, these cells usually require lower voltage to drive than one large one, and they are also easier to manufacture. When the rays of radiation projected by the projector fall on a layer of liquid crystals (both one large cell and a set of small cells), the optical phase shifts, which leads to an increase in the optical power of the system.

It is possible to combine several tunable optical elements with their arrangement one after another. Also, the layer of liquid crystals may contain not one cell, but a plurality of cells. Through a plurality of liquid crystal cells located one behind the other, the rays propagate with an increasing phase shift. Those. instead of using one thick liquid crystal cell, it is possible to use a set of thin liquid crystal cells, while the operation of the device does not fundamentally change. When combining several tunable optical elements, it is possible to use a combination of a liquid crystal layer with a single cell and with a plurality of cells, as well as a combination of positive and negative optical elements (lenses) in any sequence. The more layers of liquid crystals, the more restructuring. Each layer can be controlled individually, while the tuning range is increased. The thickness of one layer of liquid crystals is not more than 30 microns.

Instead of conventional fixed optical elements (lenses, lens systems), it is possible to use liquid crystal lenses, and to place a layer of liquid crystals between such liquid crystal lenses.

Lenses can have a variety of shapes that meet the technological requirements for the display form factor.

Lenses can be coated with a variety of coatings such as polarizing, anti-reflection, and filters can be applied to allow only certain wavelengths to pass through. Such coatings are necessary to reduce radiation losses in the system (to reduce reflection).

In accordance with the invention, the user (observer) can not only observe/view the three-dimensional "floating" image, but also interact with the three-dimensional "floating" image. That is, the proposed invention can be used as an interactive display of a three-dimensional "floating" image. The hovering interactive display system shown in FIG. 2 is designed in such a way that a user can interact with the system, and the floating image device can respond immediately or after some time to the user's input.

The hovering interactive display system shown in FIG. 2 is the floating image display device described above, further comprising an infrared waveguide, an infrared illumination source, a beam splitter, and an infrared detector. Thus, the floating image interactive display system comprises

source 1 images;

electronic control unit 2;

projection module 4;

beam splitter 7;

infrared detector 8;

infrared waveguide 9;

waveguide system 5;

system 3 with tunable optical power;

infrared illumination module 10;

control module 11;

lens 12.

The image source 1 is optically matched with the beam splitter 7 and the lens 12. The infrared waveguide 9 is located between the beam splitter 7 and the waveguide system 5, namely between the lens 12 and the waveguide system 5. The infrared illumination module 10 is arranged to illuminate the entire area of the "floating" image. The control module 11 is connected to the infrared detector 8 and the electronic control unit 2. The electronic control unit 2 is connected to the infrared illumination module 10 and is additionally configured to supply a control signal to the infrared illumination module 10. The tunable optical power system 3 is additionally configured to collimate infrared radiation scattered by the user. The waveguide system 5 is transparent to infrared radiation. Beam splitter 7 is configured to transmit scattered infrared radiation to infrared detector 8. Infrared detector 8 is configured to receive scattered infrared radiation that has passed through beam splitter 7 and transmit it to control module 11. Control module 11 is configured to recognize the fact of user interaction with the area " floating" image, as well as the place of interaction in the area of the "floating" image, and the formation of a command corresponding to the position of the place of interaction with the area of the "floating" image.

The interactive display system "floating" image works as follows.

The electronic control unit 2 generates a control signal for a tunable optical element (not shown as a separate element in FIG. 2) of a system 3 with a tunable optical power. In accordance with the control signal, the tunable optical element 3b sets the focus to a certain depth corresponding to the depth of the reproduced floating image.

The operating wavelength is preferably near infrared wavelength, eg 860 nm. The floating image interactive display system operates in sequential mode, where the formation of the floating image and the implementation of feedback to the user take place in turn. In the sequential mode of operation, the signals from the projection system 4 and from the infrared illumination module 10 are pulsed and shifted in time. Thus, there is an alternation of the infrared signal (in Fig. 2, the infrared signal is shown by solid arrows coming from the infrared illumination module 10), and signals that form a "floating" image (in figure 2, it is shown as an image of a teapot). In this case, the visible radiation that may hit the infrared detector 8 will not be taken into account, since the infrared signal and the signal that forms the volumetric "floating" image will hit the infrared detector 8 at different times. When the frequency of the device exceeds the resolution of recognizing images as separate for a person, the user has a feeling of synchronous operation of the feedback system with the system for forming a three-dimensional "floating" image.

The electronic control unit 2 generates a control signal transmitted to the infrared illumination module 10. The control signal can cause the infrared illumination module to operate in both pulsed and non-pulsed modes. The infrared illumination module illuminates the area of the "floating" image in space, in Fig. 2 solid arrows coming from the infrared illumination module 10 indicate infrared radiation illuminating the area of the "floating" image. The infrared illumination module 10 provides maximum illumination power density over the entire volume of the "floating" image.

When the user brings a hand or an object to the floating image area, the infrared light illuminating the floating image area is scattered, in FIG. 2 dotted arrows indicate user-scattered radiation. The radiation scattered by the user or the object is collimated by the system 3 with a tunable optical power and is directed through the waveguide system 5. Moreover, the waveguide system 5 is designed in such a way that the scattered infrared radiation passes through it without hindrance, that is, the waveguide system is transparent to the scattered infrared radiation, which is achieved due to the choice of parameters of the diffractive optical elements of the waveguide system, the main parameter in this case is the period of the diffractive optical elements of the waveguide system, such systems are known from the prior art. Next, the scattered infrared radiation enters the infrared waveguide 9, which is configured to input, transmit and output the scattered infrared radiation to the beam splitter 7 through the lens 12, such waveguides are known from the prior art. Lens 12 operates in several spectral ranges and serves as an element of projection optics that operates in the RGB range and an element for receiving scattered IR radiation.

The beam splitter 7 transmits the scattered infrared radiation to the infrared detector 8 with a narrow-band infrared filter. Moreover, the narrow-band infrared filter transmits only the radiation of the infrared illumination module 10 and does not transmit radiation of other ranges.

The scattered infrared radiation that hits the infrared detector 8 is processed (for example, by image processing algorithms) to determine the coordinates of objects that fall into the "floating" image area.

To determine the depth of interaction of the object with the area of the "floating" image, the variable power system 3 scans the available depth range, that is, the variable power system 3 sequentially adjusts from the minimum depth to the maximum depth and back to perceive IR radiation in order to detect a hand user or object.

By processing images from the infrared detector, it is possible to recognize the object that the user is using, or to realize face or fingerprint recognition by methods known from the prior art.

The electronic control unit 2 generates a signal that is fed to the tunable optical element of the system 3 with a tunable optical power. In this case, as described above, the system 3 with adjustable optical power changes the focus for scanning depth, the electronic control unit 2 generates a pulse signal, which is sent to the module 10 infrared illumination. The infrared illumination module 10 illuminates an area in which a three-dimensional "floating" image is formed. When an object, for example, the user's hand, enters the image volume, infrared radiation is scattered by this object, and the rays of scattered infrared radiation fall on a system with a tunable optical power, where they are collimated. Next, the collimated scattered infrared radiation enters the infrared waveguide 9 through the input diffraction grating. Further, the radiation propagates along the infrared waveguide 9 due to total internal reflection from the walls of the infrared waveguide 9 and through the output diffraction grating (not shown in the figure as a separate element) is output from the infrared waveguide 9 and enters the lens 12, which operates in several spectral ranges and serves as an element of projection optics that operates in the RGB range and an element for receiving scattered infrared radiation. Next, the radiation enters the beam splitter 7, which separates the useful infrared radiation from the visible, in this case, the visible radiation is glare and spurious reflections. Next, the separated infrared radiation enters the infrared detector 8. The infrared detector 8 may have a narrow-band infrared filter that passes only the necessary infrared radiation, which will improve the signal-to-noise ratio. The radiation that has passed into the infrared detector 8 is processed by the control module 11, which calculates the coordinates of the point where the user interacted with the "floating" image.

In the parallel operation mode, the signals from the projection module 4 and from the infrared illumination module 10 are sent at the same time. In this mode of operation, the brightness increases, but the signal-to-noise ratio of the feedback system with the user decreases.

In the sequential mode of operation, the signals from the projection system 4 and from the infrared illumination module 10 operate in a pulsed mode and are shifted in time. Thus, there is an alternation of the reverse infrared signal and a signal that forms a three-dimensional “floating” image. In this case, the visible radiation that may fall on the infrared detector 8 will not be taken into account, since the return infrared signal and the signal that forms the three-dimensional "floating" image fall on the infrared detector 8 at different times. In this embodiment, the brightness is slightly lost, but the signal-to-noise ratio of the user feedback system is significantly increased.

The infrared illumination module 10 can be integrated inside the projection module 4. Since the waveguide system 5 is transparent to IR radiation, and the infrared waveguide 9 senses IR radiation, the infrared waveguide 9 can be combined with the waveguide system 5. Devices known from the prior art can also be used. to track the user.

An array of ultrasonic transmitters can be used together with the device for a three-dimensional "floating" image. Due to the modulation of the wave phase of each transmitter, it is possible to focus the signal from ultrasonic transmitters into any area of the “floating” image space. That is, when receiving signals about the interaction of the user with the area of the "floating" image, the electronic control unit 2 instructs the control module 11 to transmit an ultrasonic signal to the area where the object is located. Thus, a reverse tactile response can be implemented, which will signal to the user about "pressing" on any element of the "floating" image, i.e. the user has the feeling that he really touched the image.

In addition, the system can be configured in such a way that when you “press” on a certain part of the “floating” image, that is, when you receive a signal from the detector about the reception of scattered radiation in a certain part of the “floating” image, the system will emit a sound signal corresponding to this part of the "floating" image. The user may also receive feedback from interacting with the floating image in the form of image changes.

Thus, the control module 11 can be connected to any necessary transmitters, which, at the command of the control module, can transmit radiation of the visible range, the invisible range, that is, the radiation of any ranges suitable for user interaction, as well as sound and ultrasound.

Thus, thanks to the proposed invention, a “floating” image is formed projected in the air, the image has a large size, a wide viewing angle, that is, the image can be seen from different angles, the brightness of the “floating” image does not depend on the viewing angle of the “floating” image, the user has the ability to interact with the "floating" image, receiving feedback.

The invention eliminates the physical interaction of the user with any surface to receive information/response or to turn on and work with any device. The user simply puts his finger in the air where the "floating" image of the button is visible, and the device with the "floating" control panel performs the action corresponding to "pressing" this button.

The proposed device for displaying a "floating" image can be used not only as an image display, but also in creating a holographic user interface when the user interacts with, for example, household appliances such as a refrigerator, cooktop, TV, air conditioner, intercom, etc. .p., as well as the proposed device can be used in hazardous industries. That is, controls can be displayed "floating" in space. In this case, using an additional camera, you can detect:

- Explicit interaction, which can be expressed by user gestures. Gestures can be symbolic (eg, raising the thumb), deictic (eg, pointing), iconic (eg, mimicking a specific movement), and pantomime (eg, using an invisible instrument);

- implicit interaction (proxemics). In this case, proxemics is understood as a sign system in which the space and time of the organization of the communication process have a semantic load. For example, if two users who have mobile devices with the proposed display form a “floating” three-dimensional image of the interlocutor (called a hologram in this case and may not be identical to the size of the user’s body) using the proposed display, then since the proposed display allows you to project dynamic images, holograms of interlocutors can change with time and the context of communication. At the same time, such a modification of a three-dimensional image can occur both with the participation of the user (using gestures, pressing buttons, voice control, user eye movements, etc.), and without his participation using a preprogrammed reaction (i.e. visual change 3D images) to the other party's message. In this case, it should be understood that communication between the holograms of the interlocutors can occur without active actions on the part of users, for example, if the proposed display is used with additional sensors for the position and reactions of the user's body.

The use of multiple handheld and portable devices can add additional context-sensitive features for interacting with the generated "floating" images. For example, they can act as a temporary space to transfer information from one hologram to another.

The proposed device can be used to recognize a fingerprint or hand, it is also possible to recognize the user's face. Such devices can be used as a lock that, when opened, recognizes the user's face or hand or any other limb.

Although the invention has been described with some illustrative embodiments, it should be understood that the invention is not limited to these specific embodiments. On the contrary, the summary is intended to include all alternatives, corrections, and equivalents that may be included within the spirit and scope of the claims.

In addition, the invention includes all equivalents of the claimed invention, even if the claims are changed in the course of consideration.

Claims (138)

1. The device for displaying a "floating" image, containing: image source; electronic control unit; a system with tunable optical power; projection module; waveguide system; and the image source is connected to the electronic control unit and is configured to store the digitized image in the memory and output the digitized image to the electronic control unit in the form of a signal containing the original image data and information about the distance from the "floating" image device on which the image should be formed, corresponding to the original; an electronic control unit is connected to the variable power system and to the projection module, the electronic control unit is configured to separate said signal into a signal containing original image data and a signal containing voltage data, the value of which corresponds to information about said distance; the projection module is optically matched to the waveguide system and is configured to convert the signal containing said source image data into a light field corresponding to the source image; the waveguide system is optically matched with the tunable optical power system and is configured to multiply the light beams constituting said light field; moreover, the system with a tunable optical power consists of a polarizer, an element with a first optical power, an element with a second optical power and a tunable optical element located between the said elements, and the polarizer is configured to polarize the multiplied light beams emerging from the waveguide system in such a way that the direction of polarization of said light beams coincides with the direction of polarization of the tunable optical element, wherein the element with the first optical power is configured to direct the polarized light beams that have passed through the polarizer to the tunable optical element; the tunable optical element is configured to introduce a phase delay into the wave front of the transmitted light field, thereby changing the distance at which a "floating" image will be formed in space, under the influence of voltage supplied by the electronic control unit; the element with the second optical power is configured to focus the mentioned light beams, constituting the light field corresponding to the original image, and emerging from the tunable optical element, in space, forming a "floating" image at a distance corresponding to the voltage applied to the tunable optical element. 2. The device according to claim 1, wherein the first power element is a positive power element, the second power element is a negative power element. 3. The apparatus of claim 2, wherein the optical power D _pos of the positive optical element is related to the optical power D _Neg of the negative optical element as follows: D _Pos ≈ -1.1 × D _Neg . 4. The apparatus of claim 1, wherein the first power element is a negative power element, the second power element is a positive power element. 5. The apparatus of claim 1, wherein the first power element is a positive power element and the second power element is a positive power element. 6. The device according to any one of paragraphs. 1-5, in which there is no air gap between the element with the first optical power, the tunable optical element and the element with the second optical power. 7. The device according to any one of paragraphs. 1-6, in which the image source is the memory of the electronic device. 8. The device according to any one of paragraphs. 1-7, in which the tunable optical element is made of a layer of liquid crystals. 9. The device according to any one of paragraphs. 1-7, in which the tunable optical element is made of an optically active material that changes optical properties as a result of stress. 10. The device according to any one of paragraphs. 1-9, wherein the image source comprises a memory storing data about each image slice, including a digitized image of the slice and data about the depth of that slice. 11. A method for operating a floating image display device for displaying a flat floating image, comprising the steps of: A) output, by means of an image source, a digitized original flat image, which enters the electronic control unit, and the digitized original flat image is a signal containing data of the original flat image and information about the distance at which the flat "floating" image should be formed, corresponding to the original flat image; B) processing, by means of an electronic control unit, the said signal, dividing it into a signal containing the said data of the original flat image, and a voltage signal, the value of which corresponds to information about the distance of the display device of the "floating" image, on which a flat "floating" image should be formed image; C) applying to the tunable optical element, by means of an electronic control unit, a voltage corresponding to the voltage signal; D) submitting to the projection module, by means of an electronic control unit, a signal containing the said data of the original flat image; and steps C) and D) are carried out simultaneously; E) converting, by means of a projection module, the data of the original flat image into a light field corresponding to the original flat image, projecting, by means of the projection module, said light field into the waveguide system; E) multiplying, by means of a waveguide system, a set of light beams constituting said light field; G) polarize, by means of a polarizer of the system with a tunable optical power, the multiplied light field emerging from the waveguide system; I) a polarized light field is supplied to the element with the first optical power, then to the tunable optical element, and under the action of the said voltage, the tunable optical element is tunable in such a way that the transmitted light field, which has passed the tunable optical element and the element with the second optical power, is formed in the form a flat "floating" image, corresponding to the original image, in space at a distance corresponding to the applied voltage. 12. The method of operation of the device for displaying a "floating" three-dimensional image, containing the steps in which: A) rendering, by means of a CAD system, a digitized initial volumetric image into a sequence of digitized flat slices of a volumetric image, wherein each digitized flat slice of the volume image is a signal containing image data of the flat slice of the volume image and information about the distance at which the "floating" image of the flat slice of the volume image should be formed, transmitting a sequence of digitized flat slices as a sequence of signals to an image source; B) transmitting said sequence of signals from the image source to the electronic control unit, processing each signal from the sequence, by means of an electronic control unit, dividing into a signal containing image data of a flat slice of a volume image and a voltage signal, the value of which corresponds to information about the distance from the floating image display device on which the floating image of a flat cut volumetric image; C) is applied to a tunable optical element, by means of an electronic control unit, sequentially with a time shift, with a frequency exceeding the resolution of recognizing images as separate for the observer: voltages corresponding to the voltage signals for the "floating" image of a flat slice of a volume image for each image of a flat slice of a volume image from the sequence; D) are fed to the projection module, by means of an electronic control unit, sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for the observer: signals containing 3D planar slice image data for each 3D planar slice image of the sequence; and steps C) and D) are carried out simultaneously; sequentially with a time shift: E) converting, by means of the projection module, the image data of the flat slice of the volume image for each image of the flat slice of the volume image from the sequence to the light field, projecting through the projection module, each mentioned light field in the waveguide system; E) multiplying, by means of a waveguide system, a set of light beams constituting each said light field; G) polarize, by means of a polarizer of the system with a tunable optical power, each multiplied light field emerging from the waveguide system; 3) the polarized light field hits the element with the first optical power, hits the tunable optical element, and under the action of the said voltage, the tunable optical element is tunable in such a way that the light field that has passed the tunable optical element and the element with the second optical power is formed in the form "floating" image of a flat section of a three-dimensional image, in space at a distance corresponding to the applied voltage; moreover, a sequence of "floating" images of flat slices of a three-dimensional image in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a three-dimensional "floating" image for the observer. 13. The method according to p. 12, and the original volumetric image is the original volumetric color image; wherein each digitized flat slice of the volumetric image consists of a red (R) component, a green (G) component, and a blue (B) component; wherein said volume planar slice image data is volume planar slice red image channel (R), volume plane slice green image channel (G), and volume planar slice blue image channel (B), wherein said signal containing image data of a flat slice of a volume image and information about a distance at which a "floating" image of a flat slice of a volumetric color image is to be formed includes: a signal containing the red image channel data (R) of a flat slice of a three-dimensional color image, a signal containing green image channel data (G) of a flat slice of a volumetric color image, a signal containing blue image channel data (B) of a flat slice of a volumetric color image, and the distance from the floating image display device at which the floating image of the red image channel (R) of the flat slice of the volumetric color image should be formed, the distance from the floating image display device at which the floating image of the green image channel (G) of the flat slice of the three-dimensional color image should be formed, the distance from the floating image display device at which the floating image of the blue image channel (B) of the flat slice of the volumetric color image should be formed, equal to the distance at which a “floating” color image of a flat slice of a volumetric color image should be formed, corresponding to the original volumetric color image; wherein said voltage signal includes: a voltage signal for the red image channel (R) of a flat slice of a volumetric color image, the value of which corresponds to information about the distance from the display device of the "floating" image, on which a "floating" image of the red image channel (R) of a flat slice of a volumetric color image should be formed, a voltage signal for the green image channel (G) of a flat slice of a color volume image, the value of which corresponds to information about the distance from the floating image display device, on which a floating image of the green image channel (G) of a flat slice of a volumetric color image should be formed, a voltage signal for the blue image channel (B) of a flat slice of a volumetric color image, the value of which corresponds to information about the distance from the floating image display device, on which a floating image of the blue image channel (B) of a flat slice of a volumetric color image is to be formed, moreover, for each flat slice of a volumetric color image, steps (B)-(C) are repeated, and the sequence of "floating" images R, G, B of the components of flat slices of a volumetric color image in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms for the observer a three-dimensional "floating" color image. 14. A method for operating a floating image display device for displaying a floating surround video, comprising the steps of: A) rendering, by means of a CAD system, each digitized source volumetric image from a sequence of digitized source volumetric images constituting a video image onto a sequence of digitized flat image slices, wherein each digitized flat image slice is a signal containing image data of the flat image slice and distance information at which the "floating" image of the flat image slice is to be formed, store a sequence of digitized flat image slices in the image source, B) transmitting a sequence of digitized flat image slices from the image source to the electronic control unit; for each digitized original volumetric image: C) processing each signal from a sequence of digitized flat image slices by means of an electronic control unit, dividing it into a signal containing image data of a flat image slice and a voltage signal, the value of which corresponds to information about the distance at which a “floating” image of a flat image slice should be formed , D) is applied to a tunable optical element, by means of an electronic control unit, sequentially with a shift in time, with a frequency exceeding the resolution of recognizing images as separate for the observer: voltages corresponding to voltage signals for the "floating" planar slice image for each planar slice image of the sequence of digitized planar image slices; E) are fed to the projection module, by means of an electronic control unit, sequentially with a time shift, with a frequency exceeding the resolution of recognizing images as separate for the observer: signals containing image data of a planar image slice for each image of a planar image slice from the sequence of digitized planar image slices; and steps D) and E) are carried out simultaneously; sequentially with a time shift: E) converting, by means of a projection module, image data of a flat image slice for each image of a flat image slice from a sequence of digitized flat image slices into a light field, projecting through the projection module, each mentioned light field in the waveguide system; G) multiplying, by means of a waveguide system, a set of light beams constituting each said light field; 3) polarizing, by means of a polarizer of the system with a tunable optical power, each multiplied light field emerging from the waveguide system; I) the polarized light field hits the element with the first optical power, hits the tunable optical element, and under the action of the said voltage, the tunable optical element is tunable in such a way that the light field that has passed the tunable optical element and the element with the second optical power is formed in the form floating" image of a flat slice of the image in space at a distance corresponding to the applied voltage; moreover, the sequence of "floating" images of flat slices of images from the sequence of digitized original images that make up the video in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms a three-dimensional "floating" video for the observer. 15. The method according to claim 14, and the original volumetric image from the sequence of digitized source volumetric images constituting the video image is the source volumetric color image from the sequence of digitized source volumetric color images constituting the color video image, wherein each digitized flat slice of a color image consists of a red (R) component, a green (G) component, and a blue (B) component; wherein said flat slice image data is red image channel data (R), green image channel data (G), blue image channel data (B); wherein said signal containing image data of the planar slice of the image and information about the distance at which the "floating" image of the planar slice of the image is to be formed includes: a signal containing the red image channel data (R) of a flat slice of a color image, a signal containing green image channel data (G) of a flat slice of a color image, a signal containing blue image channel data (B) of a flat slice of a color image; and the distance from the floating image display device at which the floating image of the red image channel (R) of the flat slice of the color image should be formed, the distance from the floating image display device at which the floating image of the green image channel (G) of the flat slice of the color image is to be formed, the distance from the floating image display device at which the floating image of the blue image channel (B) of the flat slice of the color image is to be formed, equal to the distance at which a “floating” color image of a flat slice of a color image corresponding to the original color image should be formed; wherein said voltage signal includes: a voltage signal for the red image channel (R) of a flat slice of a color image, the value of which corresponds to information about the distance from the floating image display device, on which a floating image of the red image channel (R) of a flat slice of a color image is to be formed, a voltage signal for the green image channel (G) of the flat slice of the color image, the value of which corresponds to the information about the distance from the floating image display device on which the floating image of the green image channel (G) of the flat slice of the color image is to be formed, a voltage signal for the blue image channel (B) of the flat slice of the color image, the value of which corresponds to the information about the distance from the floating image display device on which the floating image of the blue image channel (B) of the flat slice of the color image is to be formed, moreover, for each flat slice of the image from the sequence of digitized source color images that make up the video, steps (B) - (I) are repeated, and the sequence of "floating" images R, G, B of the components of flat slices of color images from the sequence of digitized source color images, constituting video in space, transmitted at a frequency exceeding the resolution of recognizing images as separate for the observer, forms for the observer a three-dimensional "floating" color video. 16. An interactive display system of a "floating" image, comprising: display device "floating" image according to any one of paragraphs. 1-10; beam splitter; infrared detector; an infrared waveguide located between the beam splitter and the waveguide system; infrared illumination module; a control module connected to an infrared detector and an electronic control unit; and the electronic control unit is connected to the infrared illumination module and is additionally configured to supply a control signal to the infrared illumination module; the system with tunable optical power is additionally configured to collimate infrared radiation scattered by the user; the waveguide system is transparent to infrared radiation; the infrared illumination module is configured to illuminate the entire area of the "floating" image; the beam splitter is configured to transmit scattered infrared radiation to an infrared detector; the infrared detector is configured to receive scattered infrared radiation that has passed through the beam splitter and transmit it to the control module; the control module is configured to recognize the fact of user interaction with the "floating" image plane, as well as the place of interaction on the "floating" image plane, and generate a command corresponding to the position of the place of interaction with the "floating" image plane. 17. The system of claim 16 wherein the infrared waveguide is integrated with the waveguide system. 18. The system according to any one of paragraphs. 16, 17, in which the infrared illumination module is embedded inside the projection module. 19. The system according to any one of paragraphs. 16-18, further comprising an array of ultrasonic transmitters. 20. The method of operating the floating image interactive display system according to claim 16, comprising the steps of: submitting, by means of the electronic control module, a control signal to the infrared illumination module; illuminate, by means of the infrared illumination module, the area of the "floating" image with infrared radiation; the user interacts with the plane of the "floating" image, due to the mentioned interaction, infrared radiation is scattered; collimating the scattered infrared radiation, by means of a system with a tunable optical power; directing collimated scattered infrared radiation through a waveguide system that is transparent to infrared radiation to an infrared waveguide; output infrared radiation from the infrared waveguide and through the beam splitter is sent to the infrared detector; detecting, by means of an infrared detector, scattered radiation and transmitting signals to the control module; recognize, by means of the control module, the fact of user interaction with the plane of the "floating" image, as well as the place of interaction on the plane of the "floating" image; form a command, by means of a control module, corresponding to the position of the place of interaction on the plane of the "floating" image.

Images