JP7486381B2 - Hologram data generating device and program thereof - Google Patents

Description

The present invention relates to a hologram data generating device and a program for the same.

Holography is a technology that records and reproduces the wavefront of light as interference fringes. In principle, 3D images reproduced by holography can faithfully reproduce the light from an actual subject, so it is said that there is no problem with the so-called inconsistency between convergence and accommodation that is often seen in stereograms (two-eye stereoscopic methods).
As a method of generating hologram data based on information on interference fringes using holography technology, there is a method of generating hologram data through calculations using a computer. This method of generating hologram data on a computer is called a Computer Generated Hologram (hereinafter referred to as CGH).
There are various methods for generating hologram data for this CGH. For example, phase information is added to a two-dimensional image and a Fourier transform is performed to generate a complex amplitude distribution of the object light on the hologram plane, and hologram data is generated from the complex amplitude distribution of the reference light (see, for example, Patent Document 1).

The hologram data is displayed on a spatial light modulator (SLM) and reproduced as a three-dimensional image by irradiating it with a readout light.
The viewing angle of this 3D image (the angle at which the 3D image can be observed) is limited by the diffraction angle due to the pixel structure of the SLM that displays the hologram data, and depends on the pixel pitch and the wavelength of light. Here, when the pixel pitch of the SLM is p and the wavelength of light is λ, the viewing angle (full angle) θ can be expressed as θ = 2 sin ^-1 [λ/(2p)]. Therefore, in order to obtain a wide viewing area in hologram reproduction, it is necessary to narrow the pixel pitch of the SLM.

Moreover, in order to display a large 3D image by holography, it is necessary to increase the area of the SLM. Here, when the number of pixels in the width direction of the SLM is Nx, the number of pixels in the height direction of the SLM is Ny, and the pixel pitch of the SLM is p, the area S of the SLM can be expressed as S = p ² NxNy. Therefore, in order to widen the viewing area of a 3D image and increase the area of the SLM, it is necessary to narrow the pixel pitch and increase the number of pixels.
In recent years, research has been conducted into narrow-pixel-pitch and multi-pixel SLMs as devices for holographic stereoscopic display (see, for example, Patent Document 2).

JP 2007-279221 A JP 2019-144423 A

In hologram reproduction, for example, to obtain a viewing angle of 30° at a wavelength of 532 nm, an SLM with a pixel pitch of 1 μm is required. Also, an SLM with a size of ¹ cm2 requires 10,000 × 10,000 pixels (100 million pixels). Thus, the amount of information required for hologram data to display a large-screen 3D image with a wide viewing zone is enormous.
Therefore, the conventional method of generating hologram data by calculation using a computer has a problem of taking a long time for the calculation.

Furthermore, the in-plane resolution of a three-dimensional image by holography is determined by the diffraction limit of the SLM when a hologram pattern acting as a condenser lens is displayed. When the wavelength of light is λ and the condensing angle (half width) of light is φ, the in-plane resolution d can be expressed as d=λ/(2sinφ) as shown in FIG. 13. The condensing angle φ of light differs between the vicinity of the hologram surface and the distant case. As shown in FIG. 13(a), in the vicinity of the hologram surface, the condensing angle φ is determined by the diffraction angle θ _d , and when the pixel pitch of the SLM 100 is p, it can be expressed as φ=θ _d =sin ⁻¹ [λ/(2p)]. Furthermore, as shown in FIG. 13(b), at a distance from the hologram surface, the collection angle φ is determined by the angle of view θ _v , and can be expressed as φ = θ _v = tan ⁻¹ [W/(2|z ₀ |)] when the size of SLM 100 is W and the propagation distance from SLM 100 to the three-dimensional image is z ₀ .
For example, in the case of the SLM 100 with a pixel pitch of 1 μm at a wavelength of 532 nm, the in-plane resolution of a three-dimensional image reproduced near the hologram surface is 1 μm.
It should be noted that the larger the collection angle φ, the smaller the in-plane resolution value. In other words, since the collection angle φ is smaller at a distant location than at a nearby location, the in-plane resolution value is larger. In this way, although the in-plane resolution value is larger at a distant location compared to at a nearby location, it is still so small that it cannot be perceived by humans, and for display purposes, there is room for information reduction in the 3D data that is the basis for CGH.

The present invention was made in consideration of these conventional problems, and aims to provide a hologram data generating device and a program for the device that can generate hologram data by reducing the amount of data that serves as the basis for CGH and reducing the amount of calculations to the extent that it does not affect the visual appearance.

To solve the above problems, the hologram data generating device according to the present invention is a hologram data generating device that generates hologram data from a target image for hologram generation, and is configured to include a downsampling means, an upsampling light wave propagation means, and a hologram data calculation means.

In such a configuration, the hologram data generating device downsamples the image to be hologram-generated by the downsampling means at a pixel pitch such that the spatial frequency of the hologram-reconstructed image is at least equal to or greater than a predetermined value. The spatial frequency of the hologram-reconstructed image is preferably no less than 30 cpd (cycles per degree), which is generally considered to be the limit of perception for a person with 1.0 visual acuity.
The hologram data generation device then performs light wave propagation calculations using the upsampling light wave propagation means to upsample the complex amplitude distribution of the hologram generation target image downsampled by the downsampling means to the pixel pitch of the spatial light modulator that displays the hologram data on the hologram plane. At this time, the amount of data of the original data used for the light wave propagation calculation has been reduced by the downsampling means, so that the Fourier transform used in the light wave propagation calculation can be calculated at high speed.
Then, the hologram data generating device calculates hologram data using the complex amplitude distribution upsampled by the upsampling light wave propagation means as the complex amplitude distribution of the object light and the complex amplitude distribution of the reference light prepared in advance, using the hologram data calculation means.
This allows the hologram data generating device to generate hologram data to be displayed on a spatial light modulator of a predetermined size by reducing the amount of data.

The hologram data generating device can be operated by a hologram data generating program that causes a computer to function as each of the above-mentioned means.

The present invention provides the following excellent effects.
According to the present invention, when generating a computer-generated hologram, it is possible to reduce data to an extent that does not affect the visual appearance, and to suppress the amount of calculations, thereby generating hologram data.
As a result, according to the present invention, hologram data can be generated faster than before.

FIG. 2 is an explanatory diagram for explaining an overview of processing performed by a hologram data generating device according to an embodiment of the present invention. 1A and 1B are explanatory diagrams for explaining pixel pitch before and after downsampling performed by a hologram data generating device according to an embodiment of the present invention, where FIG. 1A shows the pixel pitch before downsampling and FIG. 1 is a block diagram showing a configuration of a hologram data generating device according to an embodiment of the present invention. 11 is an explanatory diagram for explaining a process of propagating a complex amplitude distribution of a tomographic image after downsampling onto a hologram surface as a light wave. FIG. 13A to 13E are diagrams illustrating the flow of data when upsampling the complex amplitude distribution of a downsampled tomographic image by angular spectrum propagation in a state having wide directivity. 13A to 13D are diagrams illustrating the flow of data when the complex amplitude distribution of a downsampled tomographic image is upsampled by Fresnel propagation in a state having wide directivity. 5 is a flowchart showing an operation of the hologram data generating device according to the embodiment of the present invention. 13A to 13C are example images showing an example of how hologram data is generated from a complex amplitude distribution, and 13E shows a reconstructed image. 11 is an explanatory diagram for explaining a process of propagating a complex amplitude distribution of a multi-viewpoint image after downsampling onto a hologram surface as a light wave. FIG. 11 is an explanatory diagram for explaining a process of propagating a complex amplitude distribution of a multi-viewpoint tomographic image after downsampling onto a hologram surface as a light wave. FIG. 13A to 13E are explanatory diagrams illustrating the flow of data when upsampling the complex amplitude distribution of a downsampled multi-view image by angular spectrum propagation in a state having narrow directivity. 13A to 13D are explanatory diagrams for explaining the flow of data when upsampling the complex amplitude distribution of a downsampled multi-viewpoint image by Fresnel propagation in a state having narrow directivity. 1A and 1B are diagrams for explaining the in-plane resolution of a three-dimensional image, in which (a) shows the vicinity of a hologram surface, and (b) shows the case far from the hologram surface.

[Processing Overview of Hologram Data Generator]
First, with reference to FIG. 1, an outline of the processing performed by a hologram data generating device 1 (see FIG. 3) according to an embodiment of the present invention will be described.
The hologram data generating device 1 down-samples the complex amplitude distribution U(x, y, z), which is the original data (three-dimensional data) for generating hologram data, within a range that does not affect the visual perception of the observer M. Then, from the down-amplified data, the hologram data generating device 1 determines the complex amplitude distribution U(x, y, 0) of the size of the SLM 100 corresponding to the original data on the hologram plane H by light wave propagation calculation, and generates hologram data. Note that the hologram plane H is the position of the display surface of the SLM 100.

[Downsampling performed by the hologram data generator]
Next, downsampling performed by a hologram data generating device 1 according to an embodiment of the present invention will be described with reference to FIG.
When performing light wave propagation calculations, the hologram data generation device 1 performs different light wave propagation calculations depending on the propagation distance _zO from the hologram surface H to the object O.
Here, the hologram data generating device 1 uses the angular spectrum propagation method when zO is less than ^Np2 /λ, and the Fresnel propagation method when _zO is Np2/λ or more, where N is the size of the SLM 100 that displays the hologram data, _p is the pixel pitch of the SLM ¹⁰⁰ , and λ is the wavelength of light. This makes it possible to avoid aliasing distortion due to light wave propagation calculations.

[Angular spectrum propagation method]
Generally, in the angular spectrum propagation method, when calculating the complex amplitude distribution U(x, y, 0) on the hologram plane H from the complex amplitude distribution U(x, y, z) of three-dimensional data, the calculations of the following equation (1) (equations (1-1) to (1-3)) are performed in sequence.

Here, FT is the Fourier transform, FT ^-1 is the inverse Fourier transform, A(u,v,z) is the spectral distribution of the complex amplitude distribution U(x,y,z), and u and v are the spatial frequencies corresponding to the x and y directions, respectively. Note that the angular spectrum propagation method uses a fast Fourier transform (FFT), and therefore can perform calculations at high speed, especially when the sampling number is a power of two.
Therefore, the hologram data generating device 1 samples the complex amplitude distribution U( _x , y, z) with sampling numbers _Nx ', Ny _' (sampling intervals px _' , py _' ) shown in the following equation (2), where Nx, _Ny (sampling intervals _px ', _py ') are the sampling numbers of the hologram data to be generated and displayed on the SLM 100.

In this way, even if the sampling numbers _Nx , _Ny (sampling intervals _px , _py ) are down-sampled to sampling numbers _Nx ', _Ny ' (sampling intervals _px ', _py '), the sizes of the analysis regions in the x and y directions of the complex amplitude distribution U(x, y, z) will be the same, as shown in Figures 2(a) and 2(b).
Here, floor(x) is an integer not greater than x, and << is the shift operator.
p(z _o ) is the sampling interval at z o , where z _o is the propagation distance from the hologram surface H to the object _O of the three-dimensional data, and is a value calculated by the following formula (3).

Here, L is the distance (visual distance) from the observer M to the hologram surface H. In addition, when a three-dimensional image is enlarged and observed by a lens or the like, such as a head-mounted display, each distance is the distance to the image enlarged by the lens.
Furthermore, β is the spatial frequency (cpd: cycles per degree) of the reproduced image. It is generally said that the limit of the spatial frequency that a person with 1.0 vision can perceive is 30 cpd. Therefore, it is preferable that β is 30 cpd or more.
As shown in Fig. 1, when the period T of the sampling interval p(z _o ) (one period is two pixels) is T = 2p(z _o ), the angle δθ per period is δθ = tan ^-1 [T/(L-z _o )] ≈ T/(L-z _o ) [rad] = T/(L-z _o ) x 180/π [deg], and the period (spatial frequency) β per degree is β = 1/δθ = (L-z _o )/T x π/180 [cpd]. From this, equation (3) can be derived.

[Fresnel propagation method]
In addition, in the Fresnel propagation method, when calculating the complex amplitude distribution U(x, y, 0) on the hologram surface H from the complex amplitude distribution U(x, y, z) of three-dimensional data, the following equation (4) is calculated.

Here, (ξ, η) is the spatial coordinate, and U(ξ, η, z) is used so that the variables do not overlap in the double integral calculation. Also, k is the wave number, and k = 2π/λ. By replacing this formula (4) with x' = x/(λz) and y' = y/(λz), the complex amplitude distribution U(x, y, 0) can be calculated using FFT.
If the sampling number of the complex amplitude distribution U(x, y, 0) on the hologram surface H is _Nx , _Ny (sampling interval _qx , _qy ), the sampling interval _qx , _qy after FFT involving coordinate transformation can be calculated using the following equation (5).

Therefore, the hologram data generating device 1 samples the complex amplitude distribution U( _x , y, z) with sampling numbers _Nx ', _{Ny '} (sampling intervals qx _' , qy _' ) shown in the following equation (6), where Nx, Ny (sampling interval _qx , _qy ) is the sampling number of the hologram data to be generated and displayed on the SLM 100.

Note that floor(x) is an integer not exceeding x, and << is a shift operator.
Furthermore, p(z _o ) is the sampling interval at z o, where z _o is the propagation distance from the hologram surface H to the object _O of the three-dimensional data, and is the value calculated by the above formula (3).

[Configuration of Hologram Data Generator]
Next, the configuration of the hologram data generating device 1 according to the embodiment of the present invention will be described with reference to FIG.

The hologram data generating device 1 generates hologram data from a complex amplitude distribution in which a phase is associated with each pixel of an image for which a hologram is to be generated.
The image for which a hologram is to be generated is three-dimensional data of a subject or scene. Note that this three-dimensional data is assumed to be stored in the three-dimensional data storage device 2 in advance.
The three-dimensional data is, for example, a tomographic image arranged by distance, a multi-viewpoint image corresponding to a plurality of viewpoint positions, a multi-viewpoint tomographic image arranged by distance, etc. Here, the three-dimensional data is a complex amplitude distribution in which a phase (for example, a random phase, an equal phase, a linear phase, etc.) is associated with each pixel of the image. Here, the three-dimensional data will be described using an example of a tomographic image.
As shown in FIG. 3, the hologram data generating device 1 includes a data receiving means 10 , a downsampling means 11 , an upsampling light wave propagating means 12 , and a hologram data calculating means 13 .

The data receiving means 10 receives three-dimensional data (image for generating a hologram) from the three-dimensional data storage device 2. The data receiving means 10 may read out three-dimensional data from a three-dimensional data storage device 2 connected to the outside as an external storage device, or may receive three-dimensional data from a three-dimensional data storage device 2 connected to a remote location via a network.
The data receiving means 10 outputs the received three-dimensional data to the downsampling means 11 .

The downsampling means 11 downsamples the three-dimensional data (image to be generated for hologram generation) at a pixel pitch such that the spatial frequency of the hologram reproduction image is at least equal to or greater than a predetermined value. Here, the downsampling means 11 downsamples the complex amplitude distribution within a range that does not have a visual effect when the hologram is reproduced.
As shown in Fig. 4, the downsampling means 11 performs downsampling for each image in the depth direction (indicated by a thick line in Fig. 4) of the tomographic image _Ui (x,y, _zi ), which is three-dimensional data. xy indicates two-dimensional coordinates, and _zi indicates the propagation distance of each tomographic image from the hologram plane H. Also, i is an integer of 1 or more that identifies a tomographic slice.
The downsampling means 11 calculates the sampling points for each image according to the propagation distance z _i to the hologram surface H using the above formula (2) or (6).

The wavefront U _down (x, y) after downsampling by the downsampling means 11 becomes point cloud data of U _down (x, y) = U _i (x, y) comb (x/ _px ', y/ _py ') where comb (x, y) is a comb function (z is omitted).
The downsampling means 11 outputs the downsampled sampling points and the values of the complex amplitude distribution at those positions to the upsampling light wave propagation means 12 for each tomographic image.

The upsampling light wave propagation means 12 calculates light wave propagation so as to upsample the complex amplitude distribution of the three-dimensional data (image to be generated as a hologram) downsampled by the downsampling means 11 to the pixel pitch of the SLM 100 on the hologram plane H.
Here, the upsampling light wave propagation means 12 performs different light wave propagation calculations according to the distance from the hologram surface H for the complex amplitude distribution for each tomographic image downsampled by the downsampling means 11 .
Specifically, when the size of the SLM 100 is N, the pixel pitch of the SLM 100 is p, and the wavelength of light is λ, the upsampling light wave propagation means 12 uses the angular spectrum propagation method when z _i is less than Np ² /λ, and uses the Fresnel propagation method when z _i is Np ² /λ or greater.

Here, with reference to FIG. 5, a process in which the angular spectrum propagation method is used for light wave propagation will be described.
5A shows the complex amplitude distribution U _down (x, y, z _o ) downsampled by the downsampling means 11. Note that a, b, p _x , p _y , N _x , and N _y are the values explained in equation (2) above.

The upsampling light wave propagation means 12 performs a fast Fourier transform (FFT) on the complex amplitude distribution U _down (x, y, z _o ) shown in Fig. 5(a) to generate a spectral distribution A _down (u, v, z _o ) shown in Fig. 5(b), where u and v represent spatial frequencies in the x and y directions, respectively.
The number of samples of the spectral distribution A _down (u, v, z _o ) is 1/a, 1/b compared to the complex amplitude distribution U before downsampling (see FIG. 2A).

Therefore, in order to upsample the spectral distribution A _down , the upsampling light wave propagation means 12 copies and arranges the spectral distribution A _down according to the original sampling numbers N _x and N _y as shown in FIG. 5( c ), and performs convolution integration to generate an upsampled spectral distribution A _up (u, v, z _o ).
A _up (u, v, z _o ) can be calculated by the following formula (7).

Therefore, the upsampling lightwave propagation means 12 can perform convolution integration by arranging the spectral distribution A _down (u, v, z _o ) at 1/p _x ' intervals in the u direction and at 1/p _y ' intervals in the v direction.
Then, the upsampling light wave propagation means 12 performs an angular spectral propagation calculation on the spectral distribution A _up (u, v, z _o ) to generate the spectral distribution A(u, v, 0) on the hologram plane H shown in FIG. 5( d ).
Then, the upsampling light wave propagation means 12 adds upsampled spectral distribution A by the number of times corresponding to the number of tomographic images, and performs an inverse fast Fourier transform (IFFT) to generate a complex amplitude distribution U(x, y, 0) shown in FIG. 5( e).

That is, when the number of samples in the depth direction of the tomographic image is N _z and z _i is the propagation distance to the hologram plane H of the i-th tomographic image, the upsampling light wave propagation means 12 generates a complex amplitude distribution U(u, v, 0) from the spectral distribution A _up (A _i in equation (8)) by the following equation (8).

In this way, the upsampling light wave propagation means 12 can generate a complex amplitude distribution U according to the original sampling numbers N _x and N _y from the downsampled complex amplitude distribution U _down .
In the upsampling light wave propagation means 12, a filtering process using a filter H shown in the following equation (9) may be applied.

This filter H works as shown in the following equation (10), and can change the units that make up a 3D image from points to other shapes.

Here, h(x,y)=FT ⁻¹ [H(u,v)].
This filtering process, for example, changes the units that make up a three-dimensional image from a point to a square, making it difficult for an observer to notice that the pixels that make up the three-dimensional image are sparse when observing the three-dimensional image from closer than the design viewing distance.

Next, a process in which the Fresnel propagation method is used for light wave propagation will be described with reference to FIG.
6A shows the complex amplitude distribution U _down (x, y, z _o ) downsampled by the downsampling means 11. Note that a, b, q _x , q _y , N _x , and N _y are the values explained in equation (5) above.
The upsampling light wave propagation means 12 performs an operation FT of the following equation (11) as the fast Fourier transform of the above equation (4) on the complex amplitude distribution U _down (x, y, z _o ) shown in Figure 6 (a) to generate a complex amplitude distribution U _s (x, y, z _o ) shown in Figure 6 (b) with a reduced sampling interval.

This formula (11) is a formula for calculating the double integral of the formula (4) using FFT. Here, (ξ, η) is a spatial coordinate, and U _down (ξ, η, z ₀ ) is used so that the variables do not overlap before and after the application of FFT. Also, as shown in the formula (4), this FFT involves coordinate transformation, and the complex amplitude distribution U (x, y, z ₀ ) of the three-dimensional data is downsampled at sampling intervals q _x , q _y , so that the sampling intervals after FFT can be p _x , p _y .
The number of samples of the complex amplitude distribution U _s (x, y, z _o ) is 1/a, 1/b compared to the complex amplitude distribution U before downsampling (see FIG. 2A).
Therefore, in order to upsample the complex amplitude distribution _Us , the upsampling light wave propagation means 12 copies and arranges the complex amplitude distribution _Us according to the original sampling numbers _Nx , _Ny , as shown in Figure 6 (c), and performs convolution integration to generate an upsampled complex amplitude distribution _Uup (x, y, _z0 ).
U _up (x, y, z _o ) can be calculated by the following formula (12).

Therefore, the upsampling light wave propagation means 12 can perform convolution integration by arranging the complex amplitude distribution U _s (x, y, z _o ) at 1/q _x ' intervals in the x direction and at 1/q _y ' intervals in the y direction.
Then, the upsampling light wave propagation means 12 multiplies the upsampled complex amplitude distribution U _up by the coefficient of the following equation (13) and adds up the results the same as the number of tomographic images, thereby generating a complex amplitude distribution U (x, y, 0) after Fresnel propagation shown in FIG. 6( d ).

The coefficient in this equation (13) is a coefficient that is multiplied by the double integral in the above equation (4).
In this way, the upsampling light wave propagation means 12 can generate a complex amplitude distribution U according to the original sampling numbers N _x and N _y from the downsampled complex amplitude distribution U _down .

Returning to FIG. 3, the description of the configuration of the hologram data generating device 1 will be continued.
The upsampling light wave propagation means 12 outputs the complex amplitude distribution U(x, y, 0) for each generated tomographic image to the hologram data calculation means 13 .

The hologram data calculation means 13 generates hologram data from the complex amplitude distribution U(x, y, 0) generated by the upsampling light wave propagation means 12 .
The hologram data calculation means 13 generates hologram data using the complex amplitude distribution U(x, y, 0) as the complex amplitude distribution of the object light and the complex amplitude distribution of the reference light prepared in advance.
The complex amplitude distribution of the reference light is stored in advance in the hologram data generation condition storage means 14. The wavefront of the reference light may be any wavefront, such as a plane wave or a spherical wave.

The hologram data generated by the hologram data calculation means 13 may be any of amplitude type, phase type, and complex amplitude type. When generating an amplitude type hologram (amplitude hologram), it is preferable to perform a process of separating the conjugate image from the reproduced image, such as a half zone plate process, in the light wave propagation calculation in the upsampling light wave propagation means 12 in order to avoid the phenomenon in which the conjugate image is multiplexed on the reproduced image.
A method for generating hologram data from the complex amplitude distribution of the object light and the complex amplitude distribution of the reference light can be a commonly used method, and therefore a detailed description thereof will be omitted here.

The conditions for generating hologram data are stored in advance in the hologram data generating condition storage means 14. The hologram data generating condition storage means 14 can be configured with a general storage medium such as a semiconductor memory.
The generation conditions stored in this hologram data generation condition storage means 14 include the structure of the three-dimensional data (identification of tomographic images, multi-viewpoint images, and multi-viewpoint tomographic images and image configuration), the size and pixel pitch of the SLM 100, and the type of hologram data to be generated (amplitude type, phase type, complex amplitude type), etc.
The hologram data generation condition storage means 14 is referenced by the downsampling means 11, the upsampling light wave propagation means 12 and the hologram data calculation means 13.

By configuring the hologram data generating device 1 as described above, the hologram data generating device 1 can reduce the amount of information in the three-dimensional data to the extent that it does not have a visual impact, and generate hologram data at high speed.
Furthermore, each component of the hologram data generating device 1 does not have to be configured as a single device, and may be configured separately.
The hologram data generating device 1 can operate a computer (not shown) according to a hologram data generating program for causing the computer to function as each of the above-mentioned means.

[Operation of the hologram data generating device]
Next, the operation of the hologram data generating device 1 according to the embodiment of the present invention will be described with reference to Fig. 7 (for the configuration, refer to Fig. 3 as appropriate). It is assumed that the three-dimensional data storage device 2 stores three-dimensional data (complex amplitude distribution) in which a phase is added to each pixel of an image in advance. It is also assumed here that the three-dimensional data is a tomographic image.

In step S 1 , the data receiving means 10 receives three-dimensional data (complex amplitude distribution) from the three-dimensional data storage device 2 .
In step S2, the downsampling means 11 sequentially selects complex amplitude distributions corresponding to the individual images of the tomographic images.
In step S3, the downsampling means 11 calculates sampling points for the image selected in step S2 according to the distance from the hologram surface H using the above formula (2) or (6), and downsamples the image.

In step S4, the downsampling means 11 determines whether or not downsampling has been completed for all images of the tomographic images.
If downsampling has not been completed for all images of the tomographic image (No in step S4), the hologram data generating device 1 returns to step S2, selects the complex amplitude distribution of another image, and continues the operation.

On the other hand, when downsampling has been completed for all images of the tomographic image (Yes in step S4), in step S5, the upsampling light wave propagation means 12 upsamples all the complex amplitude distributions downsampled in step S3 by light wave propagation calculation according to the distance from the hologram plane H, propagates them to the hologram plane H, and generates a complex amplitude distribution by adding them up.
In step S6, the hologram data calculation means 13 generates hologram data from the complex amplitude distribution on the hologram plane H generated in step S5, by using the complex amplitude distribution of the reference light.
Through the above operations, the hologram data generating device 1 can reduce the amount of three-dimensional data without affecting the visual appearance, and generate hologram data at high speed.

The hologram data generating device 1 according to the embodiment of the present invention may perform the operation of step S5 between steps S3 and S4. This allows the selection, downsampling, and upsampling of each cross-sectional image to be performed sequentially for each image, and the propagation calculations to the hologram surface H can be performed, thereby saving memory space required for the calculations.

[Example of hologram data generation]
Next, an example of hologram data generation by the hologram data generating device 1 will be described with reference to FIG. 8 (and also with reference to FIGS. 1 and 3 as appropriate).
1A is an image for which hologram data is to be generated. For the sake of simplicity, only one image is shown.
(a) shows a complex amplitude distribution with a pixel count of 2048×2048, a pixel pitch of 8 μm, and a random phase added. The wavelength of the light is 532 nm. The propagation distance z _O of the light wave is −10 mm (the reconstructed image is displayed at a position 10 mm behind the SLM 100), and the viewing distance L is 500 mm. Under these conditions, the pixel pitch at which the spatial frequency of the reconstructed image is 30 cpd or more is 145 μm or less. On the other hand, it is preferable that the number of data that can be calculated at high speed by FFT is a power of 2.
Therefore, the downsampling means 11 thins out the image (a) by 1/16 in both the horizontal and vertical directions, so that the number of pixels is 128 x 128 and the pixel pitch is 128 μm. Even if downsampling is performed in this way, the spatial frequency of the reproduced image is 30 cpd or more, so that the effect of thinning out is not perceived. In addition, since FFT is applied to the complex amplitude distribution that has been thinned out by 1/16 in both the horizontal and vertical directions and the number of samples is 1/256, it is possible to speed up the calculation compared to the case where it is applied to the original complex amplitude distribution.

(b) is an image obtained by propagating the downsampled complex amplitude distribution of (a) onto the hologram surface H. In other words, (b) is the result of the upsampling light wave propagation means 12 applying FFT to the downsampled complex amplitude distribution, duplicating and arranging the spectral distribution, propagating it a propagation distance of -10 mm using the angular spectrum propagation method, and then applying IFFT. Note that (b-1) shows the amplitude of the light wave after propagation, and (b-2) shows the phase of the light wave after propagation.

(c) shows the hologram data (phase hologram) generated by the hologram data calculation means 13 from the image propagated to the hologram surface H in (b). Here, the reference light is a plane wave, and the incidence angle is 1.5° in both the x and y directions.

(e) is an image obtained by calculating the reproduction of the hologram data of (c) and enlarging a part of the reproduction. Note that here, the readout light is a plane wave, the wavelength is 532 nm, and the incident angle is the same as the condition for generating the hologram data. As shown in (e), the reproduction image is expressed as a point cloud in which the image of (a) is thinned out to 1/16 in both the horizontal and vertical directions.
In this way, the hologram data generating device 1 can perform downsampling to generate hologram data that is visually unaffected.
In the above description, an example has been described in which a tomographic image is used as the three-dimensional data. In the following, an example will be described in which a multi-viewpoint image and a multi-viewpoint tomographic image are used.

[Other examples of 3D data]
[Multi-view images]
An outline of the process in which the hologram data generating device 1 generates hologram data from a multi-viewpoint image will be described with reference to FIG. 9 (and also with reference to FIG. 3 as necessary).
A multi-viewpoint image is an image of the same subject photographed from a plurality of viewpoint positions (or photographed virtually using CG or the like), with each image being taken from a plurality of viewpoint positions.
A multi-viewpoint image is a two-dimensional image with a direction to the viewpoint added to each of the two-dimensional viewpoints. Also, a multi-viewpoint image is a complex amplitude distribution with a phase (e.g., random phase, equal phase, linear phase, etc.) added to each pixel of the image.
In the multi-viewpoint image U _mn (x, y, z _o ) shown in Fig. 9, x and y indicate two-dimensional coordinates, and z _o indicates the propagation distance to the hologram surface H. Additionally, m and n are identifiers for identifying the multi-viewpoint image at the viewpoint position. Additionally, θ _m and θ _n are horizontal and vertical angles indicating the direction of the viewpoint position.

The hologram data generating device 1 downsamples the multi-viewpoint image U _mn (x, y, z _o ). Then, by propagation calculation, the hologram data generating device 1 generates a complex amplitude distribution U _mn (x, y, 0) that is upsampled in the direction of the viewpoint position (θ _m , θ _n ) and propagated to the hologram plane H. Then, the hologram data generating device 1 adds all the complex amplitude distributions U _mn (x, y, 0) on the hologram plane H to generate one complex amplitude distribution U(x, y, 0), and generates hologram data using the complex amplitude distribution of the reference light.

[Multi-view cross-sectional images]
An outline of the process in which the hologram data generating device 1 generates hologram data from a multi-viewpoint tomographic image will be described with reference to FIG. 10 (and also with reference to FIG. 3 as necessary).
A multi-viewpoint tomographic image is a tomographic image of the same subject photographed from a plurality of viewpoint positions (or photographed virtually using CG or the like), with each image being taken from each viewpoint position.
A multi-view tomographic image is obtained by adding a direction to a viewpoint position to each two-dimensional tomographic image. Also, a multi-view tomographic image is a complex amplitude distribution in which a phase (e.g., a random phase, an equal phase, a linear phase, etc.) is added to each pixel of an image. Note that a multi-view image group may be obtained by adding a direction to a viewpoint position and depth information to each two-dimensional viewpoint image.
In the multi-view tomographic image U _imn (x, y, z _i ) shown in Fig. 10, x and y indicate two-dimensional coordinates, and z _i indicates the propagation distance for each tomographic image to the hologram surface H. Furthermore, i is an integer of 0 or more that identifies a tomographic image. Furthermore, m and n are identifiers that identify the multi-view image at the viewpoint position. Furthermore, θ _m and θ _n are horizontal and vertical angles that indicate the direction of the viewpoint position.

The hologram data generating device 1 downsamples the multi-viewpoint tomographic image U _imn (x, y, z _i ). Then, the hologram data generating device 1 upsamples the multi-viewpoint tomographic image U _imn (x, y, z _i ) in the direction of the viewpoint position (θ _m , θ _n ) by propagation calculation to generate a complex amplitude distribution U _imn (x, y, 0) propagated to the hologram plane H. Then, the hologram data generating device 1 adds up all the complex amplitude distributions U _imn (x, y, 0) on the hologram plane H to generate one complex amplitude distribution U(x, y, 0), and generates hologram data using the complex amplitude distribution of the reference light.

[Narrow directional light wave propagation]
The multi-viewpoint images and multi-viewpoint tomographic images described with reference to FIGS. 9 and 10 require narrow-directivity light wave propagation in order to allow the images to be recognized in the direction of the viewpoint positions.
Therefore, here, the process of narrow directional light wave propagation in the upsampling light wave propagation means 12 will be described.
The process of the downsampling means 11 is omitted because the method of downsampling each image is the same whether it is a multi-view image or a multi-view tomographic image, and the process of the hologram data calculation means 13 is omitted because the method of downsampling each image is the same whether it is a multi-view image or a multi-view tomographic image.

First, with reference to FIG. 11, a process in which the upsampling light wave propagation means 12 uses the angular spectrum propagation method as the light wave propagation will be described.
Fig. 11(a) shows a complex amplitude distribution U _down (x, y, z _o ) downsampled by the downsampling means 11. Fig. 11(b) shows a spectral distribution A _down (u, v, z _o ) obtained by subjecting the complex amplitude distribution U _down (x, y, z _o ) to a fast Fourier transform (FFT). The process from (a) to (b) in Fig. 11 is the same as the process from (a) to (b) in Fig. 5.

The upsampling light wave propagation means 12 performs angular spectrum propagation calculation on the spectral distribution A _down (u, v, z _o ) to generate the spectral distribution A _down (u, v, 0) on the hologram plane H shown in FIG. 11( c ).
Then, the upsampling light wave propagation means 12 shifts the spectral distribution A _down (u, v, 0) to a position corresponding to the direction of the viewpoint position (θ _m , θ _n ) and pads other areas with zeros to generate the upsampled spectral distribution A(u, v, 0) shown in Figure 11 (d).
Then, the upsampling light wave propagation means 12 adds upsampled spectral distribution A as many times as the number of multi-viewpoint images and performs an inverse fast Fourier transform (IFFT) to generate a complex amplitude distribution U(x, y, 0) as shown in FIG. 11(e).

The shift and upsampling of the spectral distribution A _down (u, v, 0) and the addition process of the multi-viewpoint images may be performed all at once. That is, the upsampling light wave propagation means 12 may prepare an array (sampling number: N _x ×N _y ) whose amplitude and phase are initialized to zero in advance for the calculation of the upsampled spectral distribution A _down (u, v, 0), calculate the spectral distribution A _down (u, v, 0) corresponding to each multi-viewpoint image, and then shift it to a position corresponding to (θ _m , θ _n ) and add it to the prepared array. In this way, the upsampling light wave propagation means 12 can generate a complex amplitude distribution U corresponding to the direction of the viewpoint position and according to the original sampling numbers N _x and N _y from the downsampled complex amplitude distribution U _down .

Next, with reference to FIG. 12, a process in which the upsampling light wave propagation means 12 uses the Fresnel propagation method for light wave propagation will be described.
Fig. 12(a) shows a complex amplitude distribution U _down (x, y, z _o ) downsampled by the downsampling means 11. Fig. 12(b) shows a complex amplitude distribution U _s (x, y, z _o ) in which the sampling interval is reduced by performing the operation FT of the above-mentioned formula (11) on the complex amplitude distribution U _down (x, y, z _o ). The process from (a) to (b) in Fig. 12 is the same as the process from (a) to (b) in Fig. 6.

Then, the upsampling light wave propagation means 12 shifts the complex amplitude distribution _Us (x,y, _z0 ) to a position corresponding to the direction of the viewpoint position ( _θm , _θn ) and zero-pads other regions to generate the upsampled complex amplitude distribution _Uup (x,y, _z0 ) shown in Figure 12(c).
Then, the upsampling light wave propagation means 12 multiplies the upsampled complex amplitude distribution U _up by the coefficient of the above-mentioned formula (13) and adds up the results the number of times equal to the number of tomographic images, thereby generating a complex amplitude distribution U(x, y, 0) after Fresnel propagation shown in FIG. 12( d ).
As a result, the upsampling light wave propagation means 12 can generate a complex amplitude distribution U corresponding to the direction of the viewpoint position and according to the original sampling numbers N _x and N _y from the downsampled complex amplitude distribution U _down .

[Modification]
Although the embodiment of the present invention has been described above, the present invention is not limited to this embodiment.
For example, here, downsampling of the three-dimensional data is performed in the hologram data generating device 1, but it is also possible to input three-dimensional data that has been downsampled in advance externally and generate hologram data. In that case, the hologram data generating device 1 may be configured without downsampling means 11.
This allows the hologram data generating device 1 to reduce the amount of three-dimensional data input from outside, thereby increasing throughput.

Further, here, the light wave propagation calculation performed by the upsampling light wave propagation means 12 has been described using the angular spectrum propagation method and the Fresnel propagation method as examples.
However, other light wave propagation calculation methods may be used. For example, the angular spectrum propagation method may use a derivative calculation method such as the shift angular spectrum method. Also, for example, the Fresnel propagation method may use a derivative calculation method such as the shift diffraction calculation method.
Furthermore, in the Fresnel propagation method, when the propagation distance z _o (see FIG. 1) is (x ² +y ² )/(2λ)<<z ₀ , the Fraunhofer diffraction calculation shown in the following equation (14) may be used.

In addition, here, the three-dimensional data (image for which a hologram is to be generated) to which a phase has been added in advance is stored in the three-dimensional data storage device 2 .
However, the image input to the hologram data generating device 1 does not necessarily have to be a complex amplitude distribution with a phase added.
In that case, the hologram data generating device 1 may be configured to include a phase adding means for adding a phase to an image, located upstream of the upsampling light wave propagating means 12 and upstream or downstream of the downsampling means 11.

REFERENCE SIGNS LIST 1 Hologram data generating device 10 Data receiving means 11 Downsampling means 12 Upsampling light wave propagating means 13 Hologram data calculating means 14 Hologram data generating condition storing means 2 Three-dimensional data storage device

Claims (4)

A hologram data generating device that generates hologram data from a target image for hologram generation, comprising:
a downsampling unit that downsamples the image to be hologram-generated at a pixel pitch such that the spatial frequency of a hologram-reconstructed image is at least equal to or greater than a predetermined value;
an upsampling light wave propagation means for calculating light wave propagation so as to upsample the complex amplitude distribution of the image to be hologram-generated, downsampled by the downsampling means, to a pixel pitch of a spatial light modulator that displays the hologram data on a hologram plane;
a hologram data calculation means for calculating the hologram data by using the complex amplitude distribution upsampled by the upsampling light wave propagation means as a complex amplitude distribution of object light and a complex amplitude distribution of a reference light prepared in advance ,
The hologram generation target image is a tomographic image,
the downsampling means performs downsampling in accordance with a distance from an image for each depth of the tomographic image to the hologram surface,
A hologram data generating device , characterized in that the upsampling light wave propagating means adds up all complex amplitude distributions calculated for the images for each depth on the hologram plane . A hologram data generating device that generates hologram data from a target image for hologram generation, comprising:
a downsampling unit that downsamples the image to be hologram-generated at a pixel pitch such that the spatial frequency of a hologram-reconstructed image is at least equal to or greater than a predetermined value;
an upsampling light wave propagation means for calculating light wave propagation so as to upsample the complex amplitude distribution of the image to be hologram-generated, downsampled by the downsampling means, to a pixel pitch of a spatial light modulator that displays the hologram data on a hologram plane;
a hologram data calculation means for calculating the hologram data by using the complex amplitude distribution upsampled by the upsampling light wave propagation means as a complex amplitude distribution of object light and a complex amplitude distribution of a reference light prepared in advance ,
The hologram generation target image is a multi-viewpoint image consisting of a plurality of viewpoint images with different viewpoint directions,
the downsampling means performs downsampling for each of the viewpoint images in accordance with a distance to the hologram surface;
The upsampling light wave propagation means calculates light wave propagation to the hologram surface according to the viewpoint direction, and adds up all complex amplitude distributions of multi-viewpoint images at the hologram surface . A hologram data generating device that generates hologram data from a target image for hologram generation, comprising:
a downsampling unit that downsamples the image to be hologram-generated at a pixel pitch such that the spatial frequency of a hologram-reconstructed image is at least equal to or greater than a predetermined value;
an upsampling light wave propagation means for calculating light wave propagation so as to upsample the complex amplitude distribution of the image to be hologram-generated, downsampled by the downsampling means, to a pixel pitch of a spatial light modulator that displays the hologram data on a hologram plane;
a hologram data calculation means for calculating the hologram data by using the complex amplitude distribution upsampled by the upsampling light wave propagation means as a complex amplitude distribution of object light and a complex amplitude distribution of a reference light prepared in advance ,
The hologram generation target image is a multi-viewpoint tomographic image consisting of a plurality of tomographic images with different viewpoint directions,
the downsampling means performs downsampling in accordance with a distance from an image for each depth of the tomographic image to the hologram surface,
The upsampling light wave propagation means calculates light wave propagation to the hologram plane according to the viewpoint direction, and adds up all complex amplitude distributions of a plurality of tomographic images on the hologram plane . A hologram data generating program for causing a computer to function as the hologram data generating device according to any one of claims 1 to 3 .

Images