JP7393093B2 - Computer-generated hologram generation device, method and program - Google Patents

Description

The present invention relates to a computer-generated hologram generation device, method, and program, and particularly relates to a device, method, and program for generating a computer-generated hologram at high speed while suppressing quality deterioration.

Holography is a three-dimensional display technology that records and reproduces light from an object (object light) based on light interference and diffraction phenomena. In holography technology, light waves emitted from an object are caused to interfere with reference light emitted from a light source such as a laser, and the object light is recorded as interference fringes (holograms) on a hologram surface. By shining reproduction illumination light onto these interference fringes, it is possible to reproduce the light at the time of recording. Since holography technology can faithfully reproduce the light emitted from objects, it is considered to be an ideal three-dimensional display technology that satisfies all physiological factors for human three-dimensional perception.

Among these holography technologies, computer-generated holography performs light wave simulation inside a computer to calculate the light wave propagation and interference required for hologram calculation, and the interference fringes are represented by an image. This is a technology that outputs data as electronic data.

Compared to analog holograms recorded on photographic plates, etc., there is no need for a complicated optical system for photographing, and the CGH displayed on a spatial light modulator (SLM) can be switched one after another. Because it has the advantage of being able to easily create videos, it is expected to be applied to next-generation televisions and XR devices such as VR/AR.

In the following explanation, the name of the technology that generates interference fringes as electronic data is defined as computer-generated holography, and the image data of interference fringes output by this technology is referred to as computer-generated hologram (CGH). ).

On the other hand, computer-generated holography has a technical problem in that the processing time required for calculation is enormous. As a CGH calculation method, the object to be recorded is defined as a set of many point light sources (3D point cloud data), and the propagation of the light emitted from each point light source is calculated to record the object light wave. ” is famous.

However, in the point light source method, it is necessary to perform a light wave propagation simulation from a large number of point groups, which requires an enormous amount of calculation processing time. Furthermore, in order to obtain a sufficient viewing area during reproduction of computer-generated holography, an SLM with a pixel pitch of approximately 1 μm, which is close to the wavelength of visible light, is required.

For example, the number of pixels required to realize a 1cm x 1cm liquid crystal on the order of 1μm is 10,000 x 10,000 pixels. If we also assume that liquid crystals will become larger, in the future we will need liquid crystals with a capacity of several hundred K to several thousand K. Considering that the calculation time of the point light source method is generally "number of point light sources x number of pixels on the hologram surface," it is difficult to perform real-time calculations with a huge number of points as input, and research on speeding up CGH generation is difficult. It is being actively carried out.

Against this backdrop of demands for faster processing times, it is necessary to convert the object light wave distribution or interference fringes on the hologram surface of a particular frame to calculate the CGH of other frames with different object arrangements from a particular frame. Techniques have been proposed to rapidly produce . In other words, when the change between frames is small, such as in each frame of a continuous video, some transformation is applied based on the object light wave distribution calculated in the previous frame, and the object light wave distribution in the next frame is calculated. The approach is to create.

Non-Patent Document 1 describes that, as shown in FIG. 13, when the pixels on the object light wave distribution or interference fringes on the hologram surface are shifted, the reproduced image similarly moves in parallel in three-dimensional space. A technique has been disclosed for rapidly generating a current frame between frames using interference fringes of a previous frame.

According to Non-Patent Document 1, in a case where a certain 3D object undergoes parallel movement with no change in depth (movement such that the depth does not change with respect to the hologram surface), a movement vector is calculated, and a movement vector is calculated based on the movement vector. By shifting the pixels of the interference fringes calculated from the 3D object, we succeeded in reducing the number of recalculations in the point light source method and reducing the calculation time. According to this method, in a scene where the depth of a 3D object changes little, the interference fringes of the next frame can be generated by simply shifting the interference fringes of the previous frame, allowing efficient calculation.

In addition, Non-Patent Document 2 describes, for the purpose of XR applications, when a wearer wearing an XR device that can view a holographic reconstructed image slightly moves the position or direction of the viewpoint, A method has been disclosed for quickly calculating interference fringes tailored to the object by applying a transformation to the object light wave distribution in the frame before movement.

According to Non-Patent Document 2, similar to Non-Patent Document 1, by pixel-shifting the object light wave distribution on the hologram surface as shown in FIG. 7, the reproduced image can be moved in the pixel-shifted direction. It becomes possible to simulate a reconstructed image when the person's viewpoint moves parallel to the hologram surface.

This characteristic of parallel movement is useful for displaying reconstructed images from slightly different viewpoints for users with various interpupillary distances, such as in HMD (Head Mounted Display) type XR devices that can view holography. It is useful in cases such as In other words, one image of the object light wave distribution that serves as a reference is recorded, and only one reference image is calculated for users with other pupillary distances by moving the object light wave distribution in parallel to create the object light wave distribution. By simply doing this, you can quickly create CGHs that are adapted to the pupillary distances of various users.

Furthermore, Non-Patent Document 2 discloses a high-speed calculation method not only for parallel movement but also for rotational movement and back-and-forth movement of a hologram surface. The rotational movement of the viewpoint is simulated by calculating the optical path difference caused by the rotation of each pixel on the hologram surface and assigning to each pixel a phase that produces this optical path difference, as shown in FIG.

Figure 8 shows an example of rotating the viewpoint around the x-axis. In the equation in Figure 8, j is an imaginary unit, k is the wave number calculated from the wavelength of light, and θrot is the rotation angle. There is. The forward and backward movement of the viewpoint can be simulated by expanding or contracting the object light wave distribution on the hologram surface, as shown in FIG. According to Non-Patent Document 2, by combining the above-mentioned "parallel movement," "rotational movement," and "back-and-forth movement," any movement of the viewer's viewpoint can be simulated by converting the object light wave distribution calculated in the previous frame. This enables high-speed generation of CGH.

Patent application No. 2019-127852 Patent Application No. 2020-192088

Seung-Cheol Kim, Xiao-Bin Dong, Min-Woo Kwon, and Eun-Soo Kim, "Fast generation of video holograms of three-dimensional moving objects using a motion compensation-based novel look-up table" Opt. Express 21, 11568-11584 (2013) R. Watanabe, T. Nakamura, M. Mitobe, Y. Sakamoto, and S. Naito, "Fast calculation method for viewpoint movements in computer-generated holograms using a Fourier transform optical system," Appl. Opt. 58, G71-G83 (2019) Research on high-speed generation method of computer-generated hologram videos (https://eprints.lib.hokudai.ac.jp/dspace/handle/2115/78282) T. Ichikawa, T. Yoneyama, and Y. Sakamoto, "CGH calculation with the ray tracing method for the Fourier transform optical system," Opt. Express 21, 32019-32031 (2013).

Non-Patent Document 2 describes transformations for simulating "parallel movement", "rotation movement", and "back and forth movement" of the viewpoint (shifting each pixel on the hologram surface, giving a phase to each pixel on the hologram surface, There was a possibility that aliasing would occur when enlarging/reducing the hologram surface.

Generally, when generating interference fringes in holography, high frequency components are likely to occur because the intervals between the interference fringes become very narrow. However, there is a limit to the fineness of the pixel pitch of current electronic devices, and the spatial frequency that can be expressed is limited depending on the resolution of the electronic device used for display. At this time, aliasing occurs in a high frequency region exceeding the sampling frequency of the electronic device.

When such aliasing occurs, higher-order diffraction images (ghost images) are generated in which multiple identical images are displayed during reproduction, as shown in FIG. 5, which interferes with correct image observation. Therefore, when performing CGH calculations, it is necessary to "frequency limit" to remove high spatial frequency parts on the hologram surface.

Calculation of the frequency limit is performed based on the sampling theorem as disclosed in literature such as Non-Patent Document 3, and in reality, as shown in Figure 5, regions where the interval between fringes becomes narrower than a certain level are removed. It is done by If the spacing between the stripes becomes too narrow, as in the case of "no frequency restriction" in FIG. 5, high frequency components will occur as aliasing distortion, resulting in the same image being aliased and displayed. However, if high frequency components are lost, the spatial resolution of the reconstructed image will decrease, so it is desirable to leave as much high frequency components as possible within a range where aliasing does not occur.

The actual calculation procedure is to identify the area on the hologram surface where aliasing occurs (generally, this can be determined based on the conditions of the optical system) before calculating the light wave propagation simulation, and then select the pixels in the area where aliasing does not occur. Light wave propagation simulation is performed only for

At this time, since the area where frequency limitation is applied changes depending on the position of the object point used for calculation, the above processing is performed for each point individually, and the object light wave distribution from each frequency limited point is added. Obtain the final object light wave distribution from multiple 3D point clouds. By the above procedure, in addition to being able to avoid the occurrence of aliasing, it is also possible to perform efficient calculations because light wave propagation calculations are not required for pixels where aliasing occurs.

However, in the case of adding a transformation to simulate viewpoint movement to the calculated object light wave distribution as in Non-Patent Document 2, the frequency distribution changes before and after the transformation. Even if aliasing does not occur in the object light wave distribution, aliasing may occur after conversion.

That is, if the pixel shift amount, rotation amount, or enlargement or reduction of the hologram surface is excessively performed, aliasing will occur in large quantities, and the quality of the reproduced image will deteriorate as ghost images become visible. Therefore, in Non-Patent Document 2, whenever the amount of parallel movement, rotation movement, or back and forth movement of the viewpoint exceeds a certain amount, it is necessary to recalculate the entire hologram surface using the point light source method. This was a problem in speeding up time.

In addition, the object light wave distribution after conversion that simulates viewpoint movement is the object light wave distribution that is obtained by adding the light waves from object points at various positions. Therefore, it is not possible to perform frequency restriction processing using a general frequency restriction processing flow.

In particular, in an optical system that can expand the field of view using a lens adopted in Non-Patent Document 2 (hereinafter sometimes referred to as a "lens expansion optical system"), the calculation method is to use a "convergent spherical reference light" as a reference light. Since the optical path difference from the reference light to each pixel on the hologram surface is different and a phase difference occurs, aliasing occurs on the final interference fringes even with simple parallel movement. will occur.

The purpose of the present invention is to solve the above technical problems and provide an apparatus, method, and program for generating CGH at high speed while suppressing quality deterioration in computer-generated hologram generation in which viewpoint movement can be simulated by converting object light wave distribution. It is about providing.

In order to achieve the above object, the present invention is an apparatus for generating a computer-generated hologram based on interference calculation between an object beam and a reference beam on a hologram surface, and is characterized in that it includes the following configuration.

(1) A 3D point cloud acquisition means that acquires a 3D point cloud of an object, a light wave propagation calculation means that calculates the object light wave distribution on the hologram surface by calculating light wave propagation for each point of the 3D point cloud, and a light wave propagation calculation means that calculates the object light wave distribution on the hologram surface. An object light wave conversion means for simulating by converting an object light wave distribution; a post-conversion frequency limiting means for removing a high frequency region that causes aliasing from the object light wave distribution after the conversion; An interference calculation unit that performs interference calculation with the reference light and outputs a hologram is provided.

(2) The light wave propagation calculation means calculates the object light wave distribution for each group, and the object light wave conversion means divides each point of the 3D point cloud into a plurality of groups according to its position. The object light wave distribution is converted according to the movement of the viewpoint, and the post-conversion frequency limiting means removes high frequency regions that cause aliasing from the object light wave distribution for each group, and integrates the object light wave distributions after removing the high frequency regions of each group. The device further includes a light wave integration means.

(1) Frequency restriction is applied to prevent aliasing on the object light wave distribution that has been transformed to simulate viewpoint movement by pixel shift, so high-speed CGH generation that reuses the light wave propagation distribution of the previous frame is possible. is possible while suppressing quality deterioration.

(2) Divide the 3D point cloud into multiple groups, perform frequency restriction on the object light wave distribution after conversion for each group, and integrate the object light wave distribution of each group after frequency restriction, so the area of frequency restriction can be minimized. Therefore, deterioration in CGH quality can be further suppressed at the cost of a slight increase in memory consumption.

FIG. 1 is a functional block diagram of a computer-generated hologram generation device according to a first embodiment of the present invention. FIG. 3 is a diagram showing an example of dividing a 3D point cloud into multiple groups. FIG. 7 is a diagram showing another example of dividing a 3D point cloud into multiple groups. FIG. 3 is a diagram showing a method of calculating a frequency restriction region. FIG. 3 is a diagram showing an example in which a higher-order diffraction image (ghost image) is generated due to the occurrence of aliasing. FIG. 7 is a diagram showing an example of storing object light waves for each group. FIG. 7 is a diagram illustrating an example of simulating parallel movement of a viewpoint by pixel-shifting the object light wave distribution on a hologram surface. FIG. 7 is a diagram illustrating an example of simulating rotation of a viewpoint by imparting to each pixel a phase produced by an optical path difference caused by rotation on a hologram surface. FIG. 7 is a diagram showing an example of simulating forward and backward movement of the viewpoint by expanding and contracting the object light wave distribution on the hologram surface. FIG. 6 is a diagram illustrating an example in which frequency limitation is applied to an object light wave distribution after conversion. FIG. 7 is a diagram showing another example in which frequency limitation is applied to the object light wave distribution after conversion. FIG. 3 is a diagram for explaining the reason why aliasing occurs in interference fringes due to pixel shift of object light wave distribution. FIG. 7 is a diagram illustrating an example in which a reconstructed image is translated in parallel by pixel shifting the object light wave distribution. FIG. 2 is a functional block diagram of a computer-generated hologram generation device according to a second embodiment of the present invention. FIG. 3 is a functional block diagram of a computer-generated hologram generation device according to a third embodiment of the present invention.

Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of a computer-generated hologram generation device 1 according to the first embodiment of the present invention. The main components are a propagation calculation section 40, an object light wave conversion section 50, an object point group frequency restriction section 60, a light wave integration section 70, and an interference calculation section 80, and components unnecessary for explanation of the present invention are omitted from illustration. There is.

Such a computer-generated hologram generation device 1 can be configured by installing applications (programs) for realizing each function on at least one general-purpose computer or server. Alternatively, it can be configured as a dedicated machine or single-function machine in which part of the application is converted into hardware or software.

The 3D point cloud acquisition unit 10 acquires 3D point cloud data used for CGH calculation. In this embodiment, in order to handle a 3D point group of a scene to be reproduced by holography in a computer, data of each point pi (i is the index of the point) is imported into the device. The 3D point cloud is input as a universal format, such as a PLY file. In this embodiment, each point pi has position information (Xi, Yi, Zi) and brightness information Ai.

Luminance information Ai may be input as color information such as RGB, but when colorizing in a general CGH generation method, interference fringes are generated independently at red wavelength, green wavelength, and blue wavelength. generate. Therefore, in general, the same process is repeated only three times and there is no difference in flow between monochrome and color, so here we will continue the explanation assuming that monochrome brightness information Ai is available.

As preprocessing for the 3D point cloud acquisition unit 10, from the perspective of reducing calculation time, each position of the 3D point cloud is quantized by applying Octree, etc., and for points that overlap at the same position, the brightness is averaged and treated as one point. You may also perform downsampling like this. Further, preprocessing may be performed to remove in advance points that are hidden by points in front and cannot be seen when viewed from a specific viewpoint position.

Note that the input data is not limited to 3D point cloud data, and polygon models may also be input. As a method for obtaining 3D point cloud data from a 3D polygon model while maintaining motion parallax for holography, Non-Patent Document 4 proposes a 3D point cloud acquisition method based on a ray tracing method. Such a method may be adopted to input a polygon model and convert it into a point cloud.

The 3D point cloud dividing unit 20 divides the input 3D point cloud into a plurality of groups. In this embodiment, the position of each point group is used as a division criterion from the viewpoint that frequency restriction is applied to similar areas in pixels on the hologram surface for point groups located close to each other.

As shown in Figure 2, the simplest example of division is to arrange voxel grids in a three-dimensional orthogonal coordinate system expressed in the XYZ directions, and treat the points that fall within each voxel grid as the same group. . The voxel grid can have any rectangular shape.

Additionally, if a transformation that simulates parallel movement of the entire scene (a scene in which the viewer appears to move sideways) is planned for the object light wave distribution, the farther from the hologram surface, the smaller the movement of the actual reconstructed image will be. appear. In other words, when the viewer's viewpoint moves in parallel by 1 cm, an object at a depth of 50 cm appears to be moving relatively more than an object at a depth of 100 cm. Therefore, grouping may be performed such that the grid size increases as the distance from the hologram surface increases.

As an example of how to achieve this, consider the arrangement of an object point light source in a 3D polar coordinate system expressed as a distance l, an elevation angle θ, and an azimuth angle Φ centered on the center of the hologram surface, and each axis is set at a fixed length or angle. It is also possible to form a grid by dividing each area.

Alternatively, as shown in FIG. 3, the grid may be expressed as a viewing cone with a base parallel to the hologram surface. In this case, the space is divided by creating a grid of small viewing cones with a fixed length in the z-axis direction and a fixed angle of view within the viewing cone that encompasses the entire 3D area where the point exists. Good too.

Note that as the number of groups after division increases, more accurate frequency restriction can be achieved, but since the number of object light wave distributions held by the light wave propagation calculation unit 40, which will be described later, increases, not only does memory consumption increase, but also memory access As the number increases, the calculation time tends to increase as well. Therefore, in order to improve the trade-off between quality and computer resources, it is effective to divide the 3D point cloud so that the grid becomes larger as the distance from the hologram surface increases.

Furthermore, instead of grouping each point pi in grid units based on its position as described above, groups may be formed using any clustering method. For example, clustering may be performed based on the k-means method and each cluster may be treated as the same group.

Creating groups based on such a clustering method allows the object point cloud frequency limiter 60 in the subsequent stage to obtain frequency limits with less error when determining the frequency limit range based on the centroid positions of the point clouds in the group. There is a higher possibility that this can be implemented. On the other hand, when inputting a 3D point cloud consisting of a huge number of points, the processing time required for clustering may increase. Therefore, in this embodiment, as shown in FIG. 2, each point pi is divided into N groups (group 1 to group n, where n is the index of the group) based on the position information in the three-dimensional orthogonal coordinate system. did.

The frequency limiting unit 30 performs frequency limiting processing to remove a high frequency region that causes aliasing from the object light wave distribution on the hologram surface for each point pi of the 3D point cloud, and performs frequency limiting processing that limits the frequency to a band that does not cause aliasing. Calculate area. In this embodiment, as in Non-Patent Document 3, a pixel area on the hologram surface where aliasing occurs due to narrowing of the interval between interference fringes at each point pi is specified, and the subsequent light wave propagation calculation unit Frequency restriction is achieved by preventing 40 from performing light wave propagation calculations.

For example, the frequency restriction region when there is a phase difference between the object light and the reference light in the lens expansion optical system can be calculated using the optical path difference between adjacent pixels by the method shown in Non-Patent Document 3. This is because the spatial frequency in a hologram has a property of being proportional to the angle between the object light and the reference light.

As shown in FIG. 4, when a certain object light source and a reference light source exist, consider a pixel (x', 0) adjacent to a determination target pixel (x, 0). The distance from the object light source position (Xi, Yi, Zi) for which you want to calculate the frequency restriction region to each pixel on the hologram surface is defined as r _O , r _O ', and the distance from the reference light source position to each pixel is defined as r _R , r _R '.

Note that when using a convergent spherical reference light wave in a lens expansion optical system, the reference light is arranged at (0, 0, f) using the focal length f of the lens, as shown in Non-Patent Document 2. Therefore, the distances r _R and r _R' are the distances between each pixel and (0, 0, f). At this time, the area where aliasing does not occur (the area to be left) can be calculated using the following equations (1) to (5), where λ is the wavelength of light.

If the above formula (5) holds true, there is no need to limit the frequency of the determination target pixel (x,0). However, in reality, the determination formula is calculated for each of the two pixels adjacent to the left and right in the x-axis direction of the determination target pixel (x, 0), and if both satisfy formula (5), there is no need to perform frequency restriction. It is necessary to consider it as such. Furthermore, the above equations (1) to (5) need to be calculated not only in the x direction but also in the y direction, and only the areas that are determined to be left in either the x or y direction are shown in Figure 5. As shown, it remains as a frequency limited region.

In addition, in the present embodiment, the frequency limiting section 30 can leave a part of the area where aliasing would normally occur. That is, the above equation (5) can be multiplied by the variable m ₁ for controlling the degree of determination as shown in the following equation (6).

If m ₁ =1, equation (6) becomes the same as equation (5), so the larger m ₁ is, the higher frequency components remain. In the above equation (6), if m ₁ is set to 0 or less, the entire region will be frequency limited and nothing will remain, so it is desirable to set m ₁ to a value larger than 0. Note that if m1 is set to infinity, it is synonymous with no frequency restriction.

In this embodiment, m ₁ is set to 1.2. With this setting, aliasing will occur on the final interference fringes, but if it is planned to perform a conversion process on the object light wave distribution to simulate viewpoint movement as in this embodiment, It may be more effective to leave high frequency components. This effectiveness will be detailed later.

Note that if the interference fringes of the pre-conversion frame are also output, aliasing will occur if this is done, so the object light wave distribution may be calculated individually with m ₁ =1 for the interference fringes of the pre-conversion frame.

Note that if 0<m ₁ <1, there will be less area to perform propagation calculations, and since only low frequency components will remain, aliasing will be less likely to occur during transformation, so it can be generated quickly with low quality. It is effective if you want to do so. However, since many high frequency components are removed, there is a problem that the spatial resolution of the reconstructed image is reduced.

Note that the frequency limiting processing shown in equations (1) to (5) above may need to be calculated using different equations depending on the properties of the reference light. For example, Non-Patent Document 3 also shows a formula for a frequency limiting method when there is no phase difference in the reference light (that is, the reference light is parallel light). In this way, the frequency limit calculation method is not limited to the calculation methods of equations (1) to (5) above.

Essentially, it is sufficient to ensure that the spatial frequency calculated by the directional differential of the phase does not exceed the sampling frequency, and any calculation method can be used as long as this is satisfied. In that case as well, it is desirable to introduce a variable that can be adjusted so as to leave high frequency components as shown in equation (6) above.

The light wave propagation calculation unit 40 calculates the propagation of the object light wave from the 3D point group to the hologram surface in units of groups using the following equations (7) and (8), based on the conventional point light source method. However, the calculation of the following equation (8) is performed only on pixels for which the frequency limiting unit 30 determines that the frequency should not be limited.

Here, (x, y) is the pixel position on the hologram surface where the light wave is propagated, si (x, y) is the light wave distribution on the hologram surface that is propagated from each point (light source) pi, and Ai is the distribution of the light wave on the hologram surface that is propagated from each point (light source) pi. Brightness, ri is the distance between point pi and pixel (x, y) on the hologram surface, k is the wave number calculated from the wavelength of light, and u _n (x, y) is calculated from each point belonging to group n. This is the object light wave distribution. As shown in FIG. 6, this embodiment is characterized in that object light waves are stored for each group.

The amount of memory required to store the object light wave distribution u _n (x, y) is comparable to the interference fringes displayed on the final electronic device (SLM), so it is recommended to use an SLM with 4K resolution, for example. If so, u _n (x, y) will also have 4K resolution. As described above, in this embodiment, a trade-off relationship is established in which the quality improves as the number of groups N increases, but the memory consumption increases.

The object light wave conversion unit 50 performs "parallel _movement ", "rotation movement", and Conversion processing is performed to simulate either or a combination of "back and forth movement" by performing appropriate conversion processing on the object light wave distribution on the hologram surface. In this embodiment, viewpoint movement is simulated by applying transformation to at least one of amplitude information and phase information of an object light wave recorded as an object light wave distribution.

For example, to simulate parallel movement of the viewpoint as shown in FIG. 7, a pixel shift is performed on the object light wave distribution u _n (x, y). If the rotational movement of the viewpoint is simulated as shown in FIG. 8, a phase generated by the optical path difference caused by the rotation is given to the object light wave distribution u _n (x, y). As shown in FIG. 9, if the forward and backward movement of the viewpoint is simulated, u _n (x, y) is enlarged or reduced.

Such conversion processing is performed for each object light wave distribution u _n (x, y) calculated for each group. In this embodiment, by shifting each pixel on u _n (x, y) by Δx in the x direction and Δy in the y direction y according to the content of the transformation, the transformed object given by the following equation (9) Obtain the light wave distribution u' _n (x, y). Note that if a pixel does not exist at the position (x-Δx, y-Δy), the pixel (x, y) is filled with 0.

Note that the converted object light wave distribution u' _n (x, y) after rotational movement can be calculated by using the following equation (10) disclosed in Non-Patent Document 2 and the like.

Further, the converted object light wave distribution u' _n (x, y) after the forward and backward movement can be calculated using the following equation (11). However, s is a variable (s>0) representing the scaling ratio of the pixel, and when s is larger than 1, it is enlarged, and when s is smaller than 1, it is reduced.

The object point group frequency limiting unit 60 performs frequency limiting processing on the converted object light wave distribution u' _n (x, y) for the purpose of suppressing aliasing. However, since the converted object light wave distribution u' _n (x, y) is saved by superimposing object light waves from multiple points pi in groups using the above equation (6), the frequency limiting section It is not possible to apply a limiting method that removes the high frequency region for each point pi and calculates light wave propagation only in the remaining region, such as the frequency limiting method implemented in No. 30.

Therefore, the object point group frequency limiter 60 calculates the frequency limit for the representative point of each group, and approximately realizes the frequency limit by leaving only the frequency limit area with the high frequency area removed. The object light wave distribution after frequency limitation is expressed as u'' _n (x, y).

In this embodiment, the centroid position of the point group belonging to each group is calculated. For example, as shown in Figure 10, the point at the center of gravity of group 2 is set to p _g2 , and the area for frequency restriction is calculated using the following equation (12) in the same manner as the above equation (6) for the point p _g2 . . Here, a variable m ₂ was introduced to control the degree of judgment.

In this embodiment, m ₂ =0.8 is set, and as shown in FIG. 10, the object light wave is left only for pixels that satisfy the above equation (12) for the object light wave distribution u' _n (x, y) after conversion, and the object light wave is left for other pixels. By setting all to 0, we obtain the frequency-limited converted object light wave distribution u'' _n (x, y).

Note that since the center of gravity position p _g2 is different from each point in the group, it is not guaranteed that all aliasing can be removed by this frequency restriction. However, if each point in the group exists near the center of gravity p _g2 , the frequency restriction area of each point in the group will be the area near the frequency restriction area of the center of gravity p _g2 , resulting in aliasing. There is a high possibility that the outbreak can be suppressed.

Furthermore, the larger the amount of conversion to the object light wave distribution u _n (x, y), that is, the amount of pixel shift, rotation angle, or pixel expansion/contraction ratio, the more low-frequency regions may change to high-frequency regions. It is desirable that the variable m ₂ is set to a smaller value as the amount of conversion increases, thereby narrowing the band of the frequency restriction region to the lower frequency side.

The frequency limiting method after conversion is not limited to the method of representing the points in a group by the centroid position as described above, but as shown in Figure 11, frequency limiting is applied to all points pi constituting each group. It is also possible to calculate a region in which no processing is performed, and leave a region that is the product of the calculated regions as a frequency-limited region. In this way, aliasing can be prevented even if the frequency band of each area on the hologram surface is different for each point Pi, such as a low frequency area at one point becoming a high frequency area at another point. It becomes possible to set a frequency restriction area that can suppress the occurrence.

Alternatively, instead of calculating the product of each area, the area where frequency restriction is not applied may be calculated at a certain percentage or more. In this case, only areas where aliasing does not occur at any point are left, so aliasing can be completely suppressed, but if there are few areas where the product can occur, it may lead to deterioration in the quality of the reproduced image. be.

In this way, in this embodiment, the grouped light waves that are close to each other are converted, and the area for frequency restriction is calculated for each group for the converted light waves, and a part of the object light wave distribution is cut. It is distinctive in that it is done this way.

The reason why aliasing occurs in the interference fringes due to a simple pixel shift in the object light wave distribution is that as shown in Figure 12, the relative position to the reference light changes with the pixel shift, and the distance to the reference light changes, so the final This is because the frequency changes when forming interference fringes. This is because when using a reference light that has a phase difference for each pixel, or when performing a pixel shift in an optical system with a lens as in Non-Patent Document 2, the passing position relative to the lens changes. This is caused by things like this.

On the other hand, there is a possibility that a frequency range in which aliasing is originally supposed to occur may change to a frequency range in which aliasing does not occur due to conversion. Therefore, the meaning of specifying a value of 1 or more as m ₁ such as m ₁ = 1.2 in the frequency limiting section 30 is that the frequency limiting section 30 in the previous stage deliberately leaves the object light wave distribution up to the region where aliasing occurs. This is based on the fact that when a high frequency region changes to a low frequency region due to conversion, the low frequency region can be left and the quality may be improved.

The light wave integrating unit 70 obtains the final object light wave distribution by integrating the frequency-limited converted object light wave distributions u'' _n (x, y) for each group. The final converted object light wave distribution u(x, y) can be calculated by the following equation (13) since it is sufficient to add the converted object light wave distributions u'' _n (x, y) of each group.

Note that depending on a specific transformation such as parallel movement or scaling, a blank area where the light wave of the previous frame does not exist may occur on the hologram surface. Since there is no information in the frame before conversion regarding this region, the object light wave distribution may be recalculated from the 3D point cloud arrangement after conversion using the point light source method. The method of recalculating and filling the empty area is disclosed in Non-Patent Document 2, so the explanation will be omitted here.

The interference calculation unit 80 performs interference calculation by inserting a reference light wave into the object light wave u(x, y) on the hologram surface as a computer simulation. In this example, the optical system (lens expansion optical system, Fourier transform optical system) disclosed in Non-Patent Document 2 is adopted, and the focal length f of the lens of the reproduction optical system disclosed in Non-Patent Document 2 is used as a reference light. A convergent spherical reference light wave that converges at the position is used. The complex amplitude distribution R(x, y) of the light wave when this convergent spherical reference light wave is propagated onto the hologram surface is expressed by the following equation (14).

Here, Ro represents the intensity of the reference light, and r represents the distance from the position of the reference light to the position (x, y) on the hologram surface. As in Non-Patent Document 2, the position of the reference light is placed at the focal position (0,0,f) of the lens constituting the lens expansion optical system. f is the focal length of the lens.

Note that the reference light of the present invention is not limited to the above equation (14), and may be a simple spherical wave reference light expressed by the following equation (15), or a parallel light expressed by the following equation (16). It may also be a reference light. φ in equation (16) is the angle of incidence of the reference light onto the hologram surface.

The interference between the reference light wave and the object light wave is obtained by the following equation (17).

Here, I(x, y) is the brightness distribution of CGH. Finally, in this embodiment, I(x, y) is normalized to a range of 0-255 and output as an image. By reproducing this image on the SLM and irradiating it with reproduction illumination light, the reproduced image can be reproduced and viewed.

In the above embodiment, each point pi of the 3D point group is grouped based on its position, and the object light wave distribution u _n (x, y) and the converted object light wave distribution u' _n (x, y) are determined for each group. and the object light wave distribution u'' _n (x, y) after frequency limitation are calculated, and these are integrated to form the final object light wave distribution.

However, the present invention is not limited to this, and if the 3D point cloud is distributed in a narrow range of 3D space and the frequency domain of each point cloud is within a certain range, grouping is unnecessary. , as in the second embodiment shown in FIG. 14, the configuration may be such that the 3D point group division section 20 and the light wave integration section 70 are omitted.

Further, in each of the above embodiments, the frequency limiting section 30 is provided to perform frequency limiting by deleting the high frequency region from the object light wave distribution before conversion processing, but the present invention is not limited to this. Alternatively, as in the third embodiment shown in FIG. 15, the frequency limiting section 30 may be omitted and all functions related to frequency limiting for deleting high frequency regions may be assigned to the object point group frequency limiting section 60. .

According to each of the embodiments described above, high-quality CGH can be generated in a short time and can be provided in real time even via communication infrastructure, so it can be used in large quantities regardless of geographic or economic disparity. It will be possible to provide a variety of entertainment to people. As a result, Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations: ``Build resilient infrastructure and promote inclusive and sustainable industrialization'' and Goal 11: ``Make cities inclusive, safe and resilient.'' It will be possible to contribute to "making the world more sustainable and more sustainable."

1... Computer-generated hologram generation device, 10... 3D point cloud acquisition section, 20... 3D point group division section, 30... Frequency limiting section, 40... Light wave propagation calculation section, 50... Object light wave conversion section, 60... Object point group frequency Limiting section, 70...Light wave integration section, 80...Interference calculation section

Claims (16)

In a computer-generated hologram generation device that generates a computer-generated hologram based on interference calculation between an object beam and a reference beam on a hologram surface,
3D point cloud acquisition means for acquiring a 3D point cloud of an object;
a light wave propagation calculation means for calculating the object light wave distribution on the hologram surface by calculating the light wave propagation for each point of the 3D point group;
object light wave converting means for simulating viewpoint movement by converting at least one of amplitude information and phase information of the object light wave recorded as an object light wave distribution;
object point group frequency limiting means for removing a high frequency region that causes aliasing from the object light wave distribution after the conversion;
A computer-generated hologram generation device comprising: an interference calculation section that performs interference calculation with a reference light on the object light wave distribution from which the high frequency region has been removed, and outputs a hologram. Equipped with means for dividing each point of the 3D point cloud into a plurality of groups according to its position,
The light wave propagation calculation means calculates an object light wave distribution for each group,
The object light wave converting means converts the object light wave distribution for each group according to the movement of the viewpoint,
The object point group frequency limiting means removes a high frequency region that causes aliasing from the converted object light wave distribution for each group,
2. The computer-generated hologram generation device according to claim 1, further comprising a light wave integrating means for integrating object light wave distributions calculated for each group after high frequency region removal. The object point group frequency limiting means calculates the center of gravity position of each point for each group, and calculates a frequency limit region that does not include a high frequency region by assuming that a point light source exists at the center of gravity position. The computer-generated hologram generation device according to claim 2. 4. The computer-generated hologram generation apparatus according to claim 3, wherein the object point group frequency limiting means removes a part of the area that does not include the high frequency area from the frequency limiting area. The means for dividing into groups is to define a three-dimensional orthogonal coordinate system consisting of XYZ directions in the space of the 3D point cloud, arrange a rectangular parallelepiped-shaped voxel grid, and group the points located inside the same voxel grid. The computer-generated hologram generation device according to any one of claims 2 to 4, characterized in that the computer-generated hologram generation device divides the hologram. The means for dividing into groups defines a three-dimensional polar coordinate system consisting of a distance l from the center of the hologram surface, a horizontal deviation angle θ, and a vertical deviation angle φ in the space of the 3D point cloud, and arranges a voxel grid. 5. The computer-generated hologram generation apparatus according to claim 2, wherein the point group located inside the same voxel grid is divided into the same group. 7. The computer-generated hologram generation device according to claim 1, wherein the object light wave conversion means performs pixel shift on the object light wave distribution on the hologram surface. 8. The computer-generated hologram generation apparatus according to claim 1, wherein the object light wave conversion means imparts a different phase to each pixel to the object light wave distribution on the hologram surface. The computer-generated hologram according to any one of claims 1 to 8, wherein the object light wave conversion means performs one of expansion and contraction of a pixel area on the object light wave distribution on the hologram surface. generator. 10. The computer-generated hologram generation device according to claim 1, further comprising frequency limiting means for limiting the object light wave distribution on the hologram surface to a frequency limiting region that does not include the high frequency region. The frequency limiting means prohibits light wave propagation calculation to a high frequency region where the object light wave distribution causes aliasing on the hologram surface for each point of the 3D point cloud,
11. The computer-generated hologram generation device according to claim 10, wherein the light wave propagation calculation means does not calculate light wave propagation to the high frequency region. 12. The computer-generated hologram generation device according to claim 10, wherein the frequency limiting means sets the frequency limiting area so as to partially include a high frequency area that causes aliasing. In a computer-generated hologram generation method in which a computer generates a computer-generated hologram based on interference calculation between an object beam and a reference beam on a hologram surface,
Obtain a 3D point cloud of the object,
The object light wave distribution on the hologram surface is calculated by performing light wave propagation calculations for each point in the 3D point cloud.
The viewpoint movement is simulated by converting the object light wave distribution,
removing high frequency regions that cause aliasing from the object light wave distribution after the conversion;
A computer-generated hologram generation method, characterized in that a hologram is output by performing interference calculation with a reference light on the object light wave distribution from which the high frequency region has been removed. Divide each point in the 3D point cloud into multiple groups according to its position,
Calculate the object light wave distribution for each group,
Convert the object light wave distribution for each group according to the movement of the viewpoint,
For each group, remove high frequency regions that cause aliasing from the object light wave distribution after conversion,
14. The computer-generated hologram generation method according to claim 13, further comprising integrating the object light wave distributions calculated for each group after removal of the high frequency region. In a computer-generated hologram generation program that generates a computer-generated hologram based on interference calculations between an object beam and a reference beam on a hologram surface,
Steps to obtain a 3D point cloud of an object,
A procedure for calculating the object light wave distribution on the hologram surface by calculating light wave propagation for each point of the 3D point cloud,
A procedure for simulating viewpoint movement by converting the object light wave distribution;
a step of removing a high frequency region that causes aliasing from the object light wave distribution after the conversion;
A computer-synthesized hologram generation program characterized by causing a computer to execute a procedure of performing interference calculation with a reference light on the object light wave distribution from which the high frequency region has been removed and outputting a hologram. A procedure for dividing each point of a 3D point cloud into multiple groups according to its position,
A procedure for calculating object light wave distribution for each group,
A procedure for converting the object light wave distribution for each group according to the movement of the viewpoint;
A procedure for removing high frequency regions that cause aliasing from the object light wave distribution after conversion for each group;
16. The computer-generated hologram generation program according to claim 15, further comprising a step of integrating object light wave distributions after high-frequency region removal calculated for each group.

Images