WO2021141565A2

WO2021141565A2 - Method for reducing the interlayer interference in multilayer holograms

Info

Publication number: WO2021141565A2
Application number: PCT/TR2021/050164
Authority: WO
Inventors: Fatih Omer Ilday; Onur TOKEL; Denizhan Koray KESIM; Ahmet TURNALI; Ozgun YAVUZ; Serim Kayacan ILDAY; Ghaith MAKEY; Parviz ELAHI
Original assignee: Ihsan Dogramaci Bilkent Universitesi
Priority date: 2020-01-08
Filing date: 2021-02-23
Publication date: 2021-07-15
Also published as: EP4088092A2; EP4088092A4; TR202000238A2; WO2021141565A3

Abstract

The present invention relates to a method for calculating and projecting 3-D holograms built in a multilayer manner. In particular, the present invention relates to a method for determining the image layers for forming the multilayer or multi-slice 3D volumetric images and the distances to be projected of said images and also for reducing the interlayer interference in holograms for forming the multilayer 3D volumetric images by means of the random selection of the phase of the image to be formed in each layer.

Description

METHOD FOR REDUCING THE INTERLAYER

INTERFERENCE IN MULTILAYER HOLOGRAMS

Related Technical Field of the Invention

The present invention relates to a method for calculating and projecting 3-D holograms built in a multilayer manner.

In particular, the present invention relates to a method for determining the image layers for forming the multilayer or multi slice 3D volumetric images and the distances to be projected of said images and also for reducing the interlayer interference in holograms for forming the multilayer 3D volumetric images by means of the random selection of the phase of the image to be formed in each layer.

State of the Art

Holograms, which can be identified as a recording of the light that falls upon the living and object, are three dimensional (3D) images. However, holography is the science and practice of making holograms. Since holograms feature depth and parallax, it ensures that the spatial environment of the object and the objects at its backside can be observed with more depth. Therefore, it possible to see the image from various angles and depths. Nowadays, holography is the most used method for transmitting an image from the real world to three-dimensional (3D) projections. However, the holography features some drawbacks. Multilayer holography is limited to just a few planes, and the problem of including the complex images in the full depth control still continues. Accordingly, it is one of the most challenging problems to reproduce and maintain the layer, that is, to form the three-dimensional images in different distances with high-resolution. Computer-generated holography (CGH) is a useful tool for final spatial control of the light in 3D. The expected application of 3D presentation popularizes this control. Furthermore, the current CGH screen technology is still limited in terms of the number of planes projected, depth of field, and full projection resolution. The digitally synthesized holograms that require no real objects to form a hologram also offer a possibility for the 3D dynamic video projection.

In real terms, 3D holography requires further full depth control and dynamic projection abilities; however, said requirements are limited to the high interference. The main difficulty herein is to store all information corresponding to a complex 3D image in a 2D form of the hologram and to achieve all original 3D image again without letting projections at different depths contaminate each other.

There are plenty of 3D holography studies conducted in the state of the art. However, these developments and studies are insufficient to overcome the fundamental drawbacks.

Following the preliminary survey conducted in the state of the art, the patent document bearing the number or "2011/04498" was examined. In the abstract of the invention, it is stated that "the object of the present invention is based on obtaining the real S3D deep images on the projectors as a result of projecting the 2D deep and real S3D images on the imaging device by way of placing the second projector behind the first projector with an angular difference of 1.5 and 7.5 degrees depending on the technique for obtaining S3D (stereoscopy 3-Dimension) real 3D image so as to form the hologram sensation of the images and projectors that project 2D deep (2-dimensional depth) and S3D (stereoscopy 3 dimension) real 3D image on a vertical plane by means of overcoming the technique for obtaining the 2D (two dimensional) hologram image of the hologram image projectors formed by the clear reflective projectors comprised of three different models, namely pyramid model, conical model, and semi pyramid model. It is designed as a unique model, on which an unlimited number of images can be positioned uninterruptedly on the conical model without any corner in addition to the corner projector models, such that said technique is overcome slightly".

Following the preliminary survey conducted in the state of the art, the patent document bearing the number or "2013/14925" was examined. In the abstract of the invention, it is stated that "device for obtaining 2D hologram image, that allows for sensing the 2D images projected through a first projection device as a 3D image, wherein it comprises a foil-coated reflecting surface, on which the image out of the first projector hits, a film- coated reflecting surface, on which the image out of said reflecting surface falls, a second projector positioned to form a second image to introduce depth in the hologram image in the area under said reflecting surface and a light source, of which color, intensity, and position are determined by the device".

Following the preliminary survey conducted in the state of the art, the patent document bearing the number or "2016/18804" was examined. The abstract of the invention states that "the present invention relates to an embodiment for converting the image played through screens of mobile devices such as mobile phones, tablet computers into a hologram. Said embodiment is developed to convert the image played t through screens of mobile devices such as mobile phones and tablet computers into a hologram. Said embodiment comprises basically a main body in a single-piece structure accommodating all elements therein and a foil that converts the image being played into a hologram".

The patent document numbered "US7548360" has been examined within the result of the preliminary survey conducted in the state of the art. The invention subjected to the application discloses a single method and apparatus for producing many of the most common types of hologram from digital data. In said invention, the data are generated entirely by a computer as a 3- D (animated) model or multiple 2-D camera images taken of a real 3-D (moving) object or scene from a plurality of different camera positions. The present invention permits the creation of restricted or full parallax master transmission or reflection type composite holograms, known as HI holograms, that can be copied using traditional methods.

Following the preliminary survey conducted in the state of the art, the patent document numbered "W02005099386" was examined. The invention described in said application discloses an apparatus and method for displaying three-dimensional images. In the inventive device, an active image reconstructor creates real-time, three-dimensional moving holograms from a single light beam. The inventive device comprises a coherent light beam generator that generates a light beam and a beam expander that expands a light beam. A digital micro-mirror device performs the holographic transform by transforming the expanded light beam into a holographic light beam. A lens receives the holographic light beam and transforms it. An image reconstruction unit forms a 3D holographic image from the modulated holographic light beam.

The light beam can be focused on the desired points in the 3D space by means of the methods formed by computers in the state of the art. However, the method is not basically suitable for generating images in a significant number of slices simultaneously, and it can generate two or three holograms at most, and those are for simple images, such that they are composed of dots.

In the state of the art, the hologram formed in the computer through two images in a similar complexity is calculated. However, the number of layers in these studies are not increased over two. In such a case, it cannot be mentioned an absolute 3D hologram.

Despite the intense studies aiming for the 3D holographic projection in the state of the art, the current methods are limited to forming images in a few planes, a narrow depth of field, and low resolution. Said holographic images allow for generating few layers. That is, the object should be split into hundreds of layers so as to image the same fully. However, only a few of said layers can be generated by the current hologram technologies. Besides, the images generated can be projected in a very narrow field or can feature a very low resolution. Both the object cannot be projected fully, and the object that is seen is not similar to the object to be intended to project.

Consequently, the abovementioned disadvantages and the inadequacy of available solutions in eliminating these disadvantages necessitated making an improvement in the relevant technical field.

Objects of the Invention

The most important object of the present invention is to successfully avoid the interference between the image layers by making use of the fact that the vectors random to each other are perpendicular to each other.

Another important object of the present invention is to keep all details of a 3D object in a 2D form while forming the holography and to prevent the adjacent layers from overlapping on other layers in the meanwhile.

Another important object of the present invention is to perform a shaping directly through laser without using any mask by means of the method developed. Thus, a much cheaper method is achieved compared to shaping the material by corroding by means of ions and electrons.

A further object of the present invention is to prevent the interlayer interference and achieve it through the image phase selected randomly for each layer in a controlled manner. Thus, it is ensured to control the random wave phases without causing any damage to the image formed on layers.

A further object of the present invention is to perform the shaping process through a direct laser without requiring any mask by way of implementing ultrafast laser shootings at an excessively high frequency. Thus, it is ensured to process the material with a less thermal fault in a more efficient and fast manner compared to the case occurring in a single and huge laser shooting .

The further object of the present invention is to reduce the Fresnel diffraction observed in the lasers locally. Thus, it is ensured to control the random wave phases, which are to allow for dominating on each layer generated, without causing any damage on the image formed in layers.

A further object of the present invention is to repeat similar representations with every sort of hologram medium, which is independent from holographic medium despite the use of SLM technology .

A further object of the present invention is to present a fundamental progression to reduce the interlayer interference in multilayer holograms. Thus, multilayer or slice 3D volumetric images can be formed.

A further object of the present invention is that it can be implemented in every sort of holographic medium, wherein it is not limited to SLM or similar specific technology. A further object of the present invention is to render the source portion of the volumetric imagining realizable, which can be separated into two parts, namely source and screen.

A further object of the present invention is to project the holograms formed independently from the angle.

A further object of the present invention is to ensure that the random phase for each depth introduces a random phase without changing the image projection at such a certain depth. Thus, the wavefront is locally determined in advance so as to reduce the Fresnel diffraction to the Fourier holography.

A further object of the present invention is that it can be employed in various fields, in which holograms formed by a computer such as laser-material processing, 3D printers, medical imaging, display systems for vehicle drivers or pilots are used or to be used.

A further object of the present invention is developing a method that can be used to generate dynamic holographies in real-time and video quality.

A further object of the present invention is to develop a method that can be used in various fields such as medical imaging, air traffic control, modern electro-optic devices, microscope studies, and laser-material interactions.

The subject matter invention's structural features and characteristics and all advantages thereof will be understood clearly by means of the figures given below and the detailed description provided with references to the figures. Therefore, assessment should be done by taking these figures and detailed description into consideration. Description of the Figures

FIGURE 1 illustrates a drawing of representation of the steps, wherein Fresnel holograms are obtained, of which the interlayer interference is suppressed by means of the inventive method.

FIGURE 2a illustrates a drawing of representation of the steps, wherein the holographic projection of the original object is formed by means of the inventive method.

FIGURE 2b illustrates a drawing of representation of the steps, wherein the multilayer 3D projection simulation is achieved by means of the Fresnel hologram of the inventive method.

FIGURE 2c illustrates a drawing of representation of the normalized vector inner product graph used in the inventive method.

FIGURE 2d illustrates a drawing of representation after and before adding the random phase of the planes obtained through the inventive method.

FIGURE 3 illustrates a drawing of representation of an exemplary hologram formed through the inventive method.

Description of the Invention

The present invention relates to a method for calculating and projecting 3D holograms built in a multilayer manner, for determining the image layers to be formed and the distances of said images to be projected, for randomly selecting the phase of an image to be formed on each layer and for reducing the interlayer interference in multilayer holograms that allows for forming multilayer or multi-slice 3D volumetric images. It is necessary to project simultaneously multiple images successively in the form of layers and slices in order to form a 3D hologram in a realistic manner. However, if multiple images are projected from the same hologram, said images are mixed into each other. Therefore, holograms have been composed of at most a few layers until today. Said problem will be solved by randomly selecting the image phase to be formed on each layer by means of the inventive method. The amplitude information of the image on each point is not affected if the wavefront is selected correctly. However, the random phase inactivates the projections of the adjacent and all other images.

The inventive method basically allows for successfully avoiding the interference between the image layers by making use of the fact that the vectors random to each other are perpendicular to each other.

Fourier and Fresnel holographies are initially combined using the piled Fresnel zone plates (FZP) through the inventive method. Locally straight Fresnel diffractions at specific depths are obtained by means of the addition of FZP to manipulate the propagation kernel. Thus, the Fresnel diffraction reduces the Fraunhofer diffraction effectively on the desired image planes. In addition to this process, images are randomized mutually by way of adding the pure random phase to each image plane, thereby avoiding the interference problem.

Every image plane is accepted as a vector with a dimension n in the complexed field. Herein, n is the number of the hologram pixels and typically is multiple. Afterward, the elimination of the interference among the image plane is equal to the claiming that said n -vectors are orthogonal. This approach constitutes great importance in projecting images successively to form 3D projections with a unique intensity and volume. Thus, 3D Fresnel holograms are formed, which exhibit 3D projection with lower interference, are multilayer, and project images successively.

Process steps of the inventive method start with determining the image layers and the distances to be projected of said images through a computer. Then, a random phase is added to each image by the computer. Thus, the vectors with the dimension n corresponding to each image layer obtained are statistically orthogonal to each other. Said orthogonality eliminates the wave interference therebetween, therefore the interference. Holograms of the layers, on which random phase is added, are calculated independently from each other. Fresnel Zone Plates corresponding to each layer's focal distances are positioned on the calculated holograms. When the obtained holograms are added to each other, a phase type hologram composed of a single layer is exhibited. When a projector projects said hologram, all images used in the calculation can be projected to 3D space without interlayer interference.

To provide detailed information on the process steps, the embodiment that comprises four steps of the inventive method is illustrated in Figure 1. The method is just an algorithmic representation of the subject matter of the invention, wherein the same or similar results may be achieved through other algorithms. In the first step, a group of target images forming the desired 3D projection is received by the computer. Each image is subjected to a pre-processing stage, in which the random phase is introduced. Each image is subjected to several iterations to form Fourier CGH (kinoform) in the second step. An Iterative Fourier-Transform Algorithm (IFTA) is implemented to project an image plane of the targeted 3D projection to each one and form a series of kinoforms Fi(x, h) to be used. IFTA cycle is employed, which is fast enough for the real-time application for this process. In the third step, each Fourier CGH (Computer generated holograms) projection is superposed on the Fresnel zone plate (FZP) to shift the same on the focal plane of the corresponding Fresnel zone plate. That is, the zone plate phase is added. In the third step, the transformed holograms are added in a complex manner to form a single complex-valued Fresnel hologram. Kinoform operator is employed for the super positioning process. After the complex superposition, the resulting total phase is used as the final hologram. Afterward, the final hologram is displayed through a displaying medium/screen . Said medium/screen may be any display used in the imaging process.

Inverse Fourier transform is initially performed on the pre- processed images in the IFTA cycle. After the Inverse Fourier Transform, the Fourier transform is performed by way of forcing the amplitude restriction. Afterward, an inverse process is carried out. If any cycle condition occurs, the phase addition process is carried out with Fresnel Zone Plates in the process of images by being removed from the IFTA cycle. The phase of the source image is varied at every IFTA iteration so as to form high-resolution images. For more optimizations, the amplitude of the source image is varied with the weighted addition of the source itself and the image of the hologram generated in this iteration. The number of iterations per image is determined based on Root Mean Square Error (RMSE) value.

The wave surface shaping process on each focal plane is not only for allowing for forming a single 3D hologram of multiple 2D holograms but also for preventing the random phase introduced from the image in which it is introduced. If the image reformed had a straight wavefront on the focal plane as in Fourier holography, this would occur almost automatically. However, Fourier holography is limited to the distanced field. Fresnel holograms may operate at almost every distance. Therefore, a Fresnel hologram is calculated, which projects a complex field distribution. The image layers to be formed and the distances thereof to be projected of said images are determined initially in the calculation process of Fresnel hologram. Said determinations are performed by way of using Fresnel diffraction (Equation -1).

Herein, while z represents the distance between the image and the hologram, (x,y) and (x,h) are spatial coordinates on the image and hologram planes, respectively, H(x,h) is a complex field distribution of the hologram and l is a wavelength. Its fundamental difference from a Fourier hologram is that the

term exists. If said term is canceled at a specific plane z = zo, this corresponds to the reduction of Fresnel diffraction to the Fourier transform on that plane.

To this end, the hologram H(x,h) is arranged as H(x,h) = E(x,h) jJjL + ) . Herein, E(x,h) defines the Fourier hologram of the

(_e- ^j<t>^(x,^y>) multiplication of the desired U(x,y) image and of the random phase that is to be detailed afterward. Since the second- order term eliminates the effect of the parabolic propagation kernel on the zo plane, it allows for reducing the projected field in this particular plane, such that it resembles a Fourier hologram in terms of a shape aspect. After this transform, the image, projected on the zo plane, is calculated using equation 2.

E(x,h) should be complex for maximum generality and the optimal results. If it is desired that the method is limited to the use of the phase holograms, a single SLM would be sufficient for the experimental representation. Superposition of a phase type of FZP on a Fourier hologram phase will generate a phase type of Fresnel hologram with a single plane. Focal control (zo) of said FZP is employed to project the desired image at any z distance beyond Talbot distance in a controlled manner.

Afterward, a single Fresnel hologram with an M multilayer projection is formed by using equation 3.

Herein, E₃(x,h) comprises a Fourier hologram of the image to be projected on the z = z_s plane.

Fresnel Zone Plates corresponding to each layer's focal distances are calculated by way of using equation 4 by the computer. Herein, M is the number of the total image plane; E₃(x,h) is a Fourier hologram of the E₃(x,h) multiplication of the random phase with the image (U(x,y)) to be projected on the z=z_s plane. Finally, the multilayer Fresnel hologram is calculated by way of using the equation-4.

Herein, the random phase (eii_s ^(x,^y>) is different for each plane and independent mutually. After the calculation of the image foreseen by the hologram on every plane, it is seen that the random phase does not deteriorate the image in which it is introduced; however, it prevents the different interplane undesired interference together with the random phase introduced in the other images.

After Fresnel Zone Plates corresponding to each layer's focal distances are calculated by way of using equation 4 by the computer, it is superposed on the holograms calculated (Equation-4).

The obtained holograms are added to each other by the computer (complex super-positioning process), thereby exhibiting a phase type of hologram composed of a single layer. After this process, the Kinoform operator is implemented. The resulting total phase obtained by using equation-4 is used as the final hologram, and the projection of the final hologram is displayed by a displayer.

As seen in Figure 2b, multiple images should be projected on the successive planes, and all those images should be embedded in the hologram. Successive lenses implemented as Fresnel zone plates (FZP) are used to focus every image on a specific plane. The first important step for this purpose is to shape the wavefront so as to reduce the Fresnel diffraction locally to the Fourier transform on every image plane. Thus, the structure of a single Fresnel hologram that can be projected on a desired number of planes is reduced to a superposition process. The second important step is to introduce the random phase in each image plane projected to prevent mutual interference (Equation 4).

As seen in Figure 1, the random phase is introduced to the source images, and consequently, the Fourier holograms are formed, of which the Fresnel zone plates and the distances thereof in the "z" direction are adjusted. It is formed a Fresnel hologram, of which the interlayer interference is suppressed. Herein, the random phase allows for reducing the interlayer interference significantly by means of the orthogonality of the random vectors to each other.

An image may be accepted as an N-dimensional vector, wherein N is the number of pixels. Random vectors are asymptotically orthogonal on the N border. The feature resulting from the central limit theorem and law of large numbers causes to remove tracks of the images during the hologram reconstruction. Such a case eliminates almost absolutely the interplane interference risk in the reconstructed images.

The hologram test formed by the computer is performed by means of the inventive method. The calculation performed may be a circuit with a processor and every sort of similar calculator, although other processes are performed by the computer. In case there is infrared illumination in the experiment, a laser source employs a collimator to invalidate the decomposition of the laser beam and expand the light spot size. A projecting liquid crystal- crystal spatial light modulator is also used in a digital camera with a pixel of 800 x 600 and with a pixel size of 20 pm so as to fill completely the hologram projected in SLM. After SLM is modulated with the Fresnel CGH, it projects a parallel and polarized laser beam, and then, the light beam is expanded with a 3x telescope to prevent the zero-degree diffraction optionally and afterward projected on a screen. The dimension of the hologram is selected as 512 x 512 pixel, and the phase quantization is adjusted as 202 level. Projecting distances and dimensions of the images vary on SLM dimensions that can be scaled with greater SLMs and smaller pixels and the pixel dimensions. Furthermore, the method can be employed for every sort of wavelength.

The performance of 3D Fresnel CGHs depends on the pixel dimension and the pixel density of the hologram, the modulation type, illumination wavelength, and the amplitude and phase distributions in all image planes. Practical restrictions may affect performance as the experimental restrictions in forming an image adjacent to a projecting-type of a hologram. Therefore, it would be absolutely complexing to find a precise analytic performance metric featuring all related parameters. Instead, two metrics are selected, which are to provide useful information on the performance of 3D Fresnel holograms. The first one is the root-mean-square error, and the second one is the depth of field. The first depends on the image quality and is a measure of similarity between the source images and the images projected on each plane. The second metric depends on the axial resolution and relates to a maximum number of planes that can be spared for given image quality. RMSE is initially calculated in the plane corresponding to this for each image and then is calculated through the average of all plane results for the purpose of providing a collective quality metric for a 3D hologram. Said value is used to evaluate how the projection quality varies as a function of the plane number. The number of image planes is selected for an error tolerance expressed in RMSE.

In parallel with that, DoF is employed to evaluate the axial resolution. DoF is a metric used commonly to define the maximum distance between two objects, in which the objects are still clearly seen in an acceptable manner, in professional photography. Thus, the interference between images may be evaluated through DoF on each plane.

The hologram obtained may be used stably and dynamically. For example, movable 3D holograms can be volumetrically projected by way of using a spatial light modulator available in many projection devices. Alternatively, the phase type may be written permanently on an optic element. When said optic element is illuminated by a laser, an image may be formed, which resembles the spatial light modulator. The number of layers is determined based on the image complexity and the resolution of the optical element with the phase type used. Therefore, the number of layers may be unlimited, providing that the resolution is sufficiently high. The method can be applied on every sort of hologram medium, wherein it is not limited to the spatial light modulator or similar specific technology. The volumetric imaging may be divided into two, namely source and screen. The inventive method renders the source portion of the real volumetric projection realizable. Multilayer or slice 3D volumetric images can be formed through the inventive method.

Holograms formed through the inventive method may be projected independently in terms of monitoring and imaging.

Holographic studies can be performed through the inventive method, which can be used in various fields such as medical imaging, air traffic control, modern electro-optic devices, microscope studies, and laser-material interactions. Furthermore, the holograms formed through the method may be employed in various fields, in which holograms formed by a computer such as laser-material processing, 3D printers, medical imaging, display systems for vehicle drivers or pilots are used or to be used.

Claims

1.Method for calculating and projecting multilayer and 3D holograms, characterized in that; it comprises the following process steps: - Adding random phase (0_S) to each of a group of target images (U_s) constituting 3D projection by computer,

- Adding Fresnel zone plates with a suitable value (Z_s) to focus every image layer on the projection to be conducted at a targeted distance by a computer, - Obtaining the total hologram (H_M) by gathering all layer information together,

- Displaying the final hologram through a displayer.

2.Method for calculating and projecting multilayer and 3D holograms according to Claim 1, characterized in that; in the process step of "obtaining the total hologram (H_M) by gathering all layer information together by a computer;

j Fresnel ¾one plate all layer information is gathered together by the computer by means of the equation given above.

3 . Method for calculating and projecting multilayer and 3D holograms according to Claim 1, characterized in that; in the process step of "obtaining the total hologram {H_M) by gathering all layer information together by a computer; the complex super-positioning process is applied to the layers.

4.Method for calculating and projecting multilayer and 3D holograms according to Claim 1, characterized in that; in the process step of adding the random phase to each of a group of target images constituting 3D projection by the computer, the random phase added and the images obtained at the N border are orthogonal asymptotically.

5.Method for calculating and projecting multilayer and 3D holograms, characterized in that; in the process step of projecting the final hologram through a displayer, the displayer used is a spatial light modulator.

6.Method for calculating and projecting multilayer and 3D holograms according to Claim 1, characterized in that; in the process step of adding Fresnel zone plates with a suitable value to focus every image layer on the projection to be conducted at a targeted distance by a computer; a series of successive lenses is employed, which are implemented as Fresnel zone plates (FZP) to focus each image on a specific plane.