TWI384337B - Method for generating video hologram in real time for a holographic display device - Google Patents

Abstract

The method involves dividing a scene into object points in a light modulator unit. The scene can be seen as a reconstruction from a visibility region. The region is located within a periodicity interval of the reconstruction of the video hologram. The region defines subholograms together with each object point of the scene to be reconstructed. The entire hologram is generated from superimposition of contributions of subholograms. The contributions at the entire reconstruction of the scene for each object point can be determined from a look-up table.

Description

Method for instantly generating video full image image by holographic display

The invention relates to instantaneously generating a full-view videovisual image from image data with deep information, in particular a computer-generated video full image (CGVH). During the holographic reconstruction of a three-dimensional object or a three-dimensional scene, the wavefront of the light wave is established by the interference and superposition of the homologous light waves.

The video hologram can be computed from the hologram of the three-dimensional scene sequence and stored in an electronic device, relative to a conventional hologram stored in the form of an image or in another suitable interference pattern.

The modulated modulated light that can produce interference propagates in the front space of the observer in a wavefront form that controls the amplitude and phase values of the light waves from which the wavefront of the light wave can reconstruct a three dimensional scene. The optical modulator device that controls the hologram value of the video hologram can cause a modulated wave field modulated by pixels, and reconstruct a desired three-dimensional scene in space by establishing interference of light waves.

A holographic display typically includes a controllable pixel configuration that reconstructs an object point by electronically affecting the amplitude and/or phase of the illumination light. A variety of optical modulator devices are currently known. For example, such a configuration can be a spatial light modulator (SLM) device. The display can be continuous rather than matrix. For example, a continuous type of optical modulator can be used, including a continuous SLM with matrix control or an acousto-optic modulator (AOM). A liquid crystal display LCD can be used as an example of such a suitable display. The image reconstruction of the video hologram is performed by the spatial amplitude modulation of the light pattern. However, the invention is also applicable to other controllable devices that use the same dimming to modulate the optical wavefront.

In this context, a so-called "pixel" is a controllable hologram pixel on a light modulator device, the pixels being individually addressable and controllable by discrete values of hologram points. Each pixel represents an hologram of the video hologram. In the LCD device, a so-called "pixel" is used to display image points of respective addresses of the screen. In digital light processing displays (DLP), so-called "pixels" are used on individual micromirrors or small groups of micromirrors. In a continuous SLM, a "pixel" is a transition region on a light modulator device that represents a complex hologram point. The so-called "pixel" usually means the smallest unit that represents or can display a complex hologram point.

The preferred holographic display used in the present invention is basically based on the principle that a scene that can be decomposed into object points is encoded as a complete hologram on at least one of the optical modulator devices. The scene can be seen from the visible region of the reconstructed image interval period of the video hologram. A sub-image is defined for each object point of the scene to be reconstructed, and the complete hologram can be formed by superposition of sub-images. In general, the principle is primarily used to reconstruct a wavefront that can be emitted by one object into one or more visible regions.

For the sake of detailed description, such a device is based on the principle that the reconstruction of individual object points requires only a sub-image as a subset of the complete hologram that is encoded in the optical modulator device.

The hologram display includes at least one screen device, which may be a light modulator itself, in which a full image of the scene can be encoded, or an optical component, such as a lens group or mirror, on which the light is encoded. The hologram or wavefront of the scene on the modulator can be projected.

The definition of the screen device and the corresponding principles of scene reconstruction in the visible region are described in other patent application documents filed by the applicant. In the documents WO 2004/044659 and WO 2006/027228, the screen device is the light modulator itself. In the document WO 2006/119760 "Projection device for holographic reconstruction", the screen device is an optical element which allows the hologram image encoded on the light modulator to be projected. In the document DE 10 2006 004 300 "Projection device for holographic reconstruction", the screen device is an optical element which allows the wavefront encoded on the light modulator to be projected.

The "visible area" is a limited range through which the observer can see the entire reconstructed scene. In the visible region, the wave field creates interference and forms a wavefront that allows the reconstructed scene to be seen by the observer. The visible area is located near the observer's eyes or eyes, and the visible area can be moved in the X, Y, and Z directions, and the actual observer position is tracked with the assistance of a known position detection and tracking system. Each observer may use two visible areas, one for each eye. In general, other embodiments of the visible area are also possible. It is also possible to encode the video hologram, which allows the individual objects or the entire scene to be located behind the light modulator.

Virtual, frustoconical reconstruction space in a holographic display optical modulator device Extending from the visible region, wherein the light modulator presents the bottom of the virtual frustum and the visible region presents the top portion of it. If the visible area is very small, the virtual frustum is approximately triangular. The observer can enter the omnidirectional display through the visible area and receive the wavefront representing the scene in the visible area.

The method of calculating a video hologram is described in the patent document WO/2006/066906 filed by the applicant. Typically, the profile is cut into planes of parallel light modulator planes, all of which are converted into visible areas and added in the visible area. The result of the addition is then inversely converted to the hologram plane, which also configures the optical modulator to determine the complex hologram value of the video hologram.

The method basically performs the following steps for a three-dimensional scene with the assistance of a computer: in terms of the plane of an observer, the diffracted image can be in the form of a separate two-dimensional distribution of the wave field, from each of the tomographic profiles The object data set calculates that the plane of the observer is located at a finite distance and parallel to the section, wherein the wave field of all the sections can perform data calculation on at least one common visible area; the calculated distribution of all the section layers is added, In the reference group relative to the observer plane, a progressive wave field is defined for the visible region; for generating a hologram group of the general computer generated scene hologram, the reference group content can be converted into a hologram plane, the plane The system is located at a finite distance and parallel to the reference plane, wherein the optical modulator device is located on the hologram plane.

The implementation of this method causes a large amount of computational burden because of the large amount of data conversion required. It can be seen that instant coding or generation of hologram values requires cost and high performance. Computer unit group. Such expensive computer unit groups can limit or reduce the acceptability of digital video holographic devices.

Accordingly, it is an object of the present invention to provide a method for instantly generating a video hologram from a three-dimensional image data having depth information. These holograms can be created using simple and inexpensive sets of computational units.

This can be solved in a way that, for all object points, the distribution of light waves into the visible area can be captured in at least one view. These distribution data of each object point can be described by the light wave propagation data set DP _VR , which is established by reference to the visible area.

According to the method of the present invention, it is applicable to a hologram display as defined in the content of claim 1 of the patent application. Such a holographic display incorporating a suitable light modulator device is based on the principle of superimposing the wavefield modulated by the object point information of the scene in at least one of the visible regions.

A preferred embodiment of the method of the present invention is illustrated as follows: In the processing steps of the preparation, the visible object points of the scene represented by the image data having the depth information are determined. The prepared data is preferably extracted from the image interface or image card. The method used in accordance with the present invention includes the following processing steps for each visible object point of the scene:

Step (1): The light wave propagation data set DP _VR is obtained by converting the light wave propagation into the visible region to obtain the light wave propagation data set DP _VR , the data group referring to the visible region.

Step (2): the conversion is repeated until all the object points of the entire scene are converted, and the results of the respective conversion DP _VR are added to describe the progressive wave field of the entire scene of the visible region in the data set D Σ _VR of the reference visible region. , DΣ _VR =Σ DP _{VR i}

The referenced data set D Σ _VR can thus represent or describe the entire scene that is converted into the visible area.

Step (3): inverse conversion, wherein the progressive data set D Σ _VR , which is found in step 2 and the reference visible region, is converted from the visible region into the hologram plane of the optical modulator device to generate a plurality of video holograms. Full image value.

The concept according to the present invention is that for each object point, the reference data set DP _VR established by the reference light wave propagating into the visible area can be extracted from at least one look-up table.

The look-up table includes these referenced data sets DP _VR , which can be quickly accessed by creating a look-up table. The look-up table can be implemented in any of the data carriers, memory and interfaces that provide the above data sets. The examples herein are memory, data carriers, databases or other storage media and interfaces dedicated to the present invention. The preferred interfaces are Internet, WLAN, Ethernet, and other regional and global networks.

In accordance with further aspects of the present invention, for example, to compensate for tolerances caused by the position or appearance of the light modulator device, or to improve the quality of the reconstructed image, an additional correction function can be used. For example, the correction value is tuned to the data set DP _VR that references the light wave propagation into the visible region, and/or to the progressive data set D Σ _VR that is referenced into the visible region, and/or converted to a complex hologram value, To correct the phase and/or amplitude of the complex values of these data.

The principle of using the look-up table can be better expanded, for example, parameter data from color and/or brightness information in the respective look-up tables can be used. In addition, the data set DP _VR or D Σ _VR can be modulated using the brightness and/or color values also extracted from the self-checking table. For color display, the color-related data set DP _VR can also be extracted from the individual look-up tables of the main colors.

The look-up table is, for example, by determining the input data set DP _VR corresponding to each possible object point in a defined space, by referring to a visible light wave by propagating an object point light wave that enters the visible region through the conversion, . The input values are stored in a suitable structure of the data carrier and/or storage module, or the storage interface provides input values. Another solution is to determine the correspondence of the data set DP _VR established by reference to the visible area by the aid of the optical tracking method. It is also possible to determine these data sets by optimization and/or approximation. For example, this space includes the range of motion of the observer and its eyes, within which a hologram can be seen.

In accordance with the method of the present invention, access to such data can be performed for all reconstructed object points. For example, the depth information of the object point can be used to query, read and process the corresponding data set D Σ _VR . This same method applies more detailed parameter values based on color and brightness related lookup tables. The data read can be processed further without any delays and complex operations that need to be performed. The generation of timely hologram values can thus be replaced by the method of the invention.

The visible area is composed of an image grid, which is used to describe the propagation of light waves from the object point to the visible area with the aid of the data set DP _VR . For example, the range consists of a matrix and a column of similar matrix, where each matrix element can be used to represent a complex value. All values can form the data set DP _VR . When there are m rows and n columns, the data set DP _VR includes a complex value of m times n. The data set architecture is suitable for fast access to data values, and the analogy principle is used for the entire scene data set DP _VR .

Another general concept of the present invention relates to the prevention of side effects of spots. The spot is the single spot seen by the observer in the hologram. The spot is usually generated by randomly distributed intensity extremes. According to the present invention, the progressive data set D Σ _VR value of the visible region can be smoothed. . This means, for example, that the amplitude extremes can be reduced. Furthermore, any discontinuities in the amplitude and/or phase curves can be corrected. The input values in the data set DΣ _VR can also be corrected by the optimization and self-learning algorithms to reduce the chance of speckles. Preferably, the previous image data of the video sequence can be included in the consideration of these situations, and the number and intensity of the spots can be reduced by homogenization or correction, and these principles can also be applied to the data set DP _VR .

Overall, assuming that the resolution of a commercial light modulator device provides a high-valued hologram, the cost of the computing unit used to generate holographic information in the past can be substantially reduced. When using a look-up table, the computational load can be reduced by the order of the amplitudes. Thus, the method of the present invention can be handled by a general computer system. For holographic applications, this approach ensures that the hologram is generated in an interactive and instant manner. Finally, by producing a reliable instant hologram, it is ensured that the delays required to track the observer's pupils are reduced. Instant holograms generated for a single observer are also guaranteed Simply a simple unit of arithmetic. The method of the present invention can also provide a temporal or spatially separated instant hologram for use by a plurality of observers.

For example, because the creation of a hologram requires only a small amount of computational load, there is no need to perform computations by the central processing unit CPU of the computer. According to an alternative solution, an image card component can be used to create an hologram, wherein the device is preferably a video central processing unit (GPU) and/or a specially configured operational unit group. This also allows for an increased rate of data transmission to be used.

The present invention can substantially provide a wide range of applicability and is acceptable for holographic displays, and has higher economic efficiency.

Figure 1 illustrates the general principles of the basis of a hologram display (HAE) for an observer. This principle is used in accordance with multiple observers whose position can be indicated by their eyes or pupils (VP). The apparatus includes a light modulator device (SLM) which is identical to the screen device (B) of the present embodiment in order to simplify the description and to view the scene (3D-S) in at least one visible area (VR). The wave field modulated by the object point information is superimposed. The eye can be traced to the visible area, and the reconstructed image of a single object point (OP) of the scene (3D-S) requires only a sub-image (SH) as a complete hologram encoded on the light modulator device (SLM). A subset of (HΣ _SLM ). As seen in this figure, the sub-image (SH) range contains only a few sub-regions of the light modulator device (SLM).

Figure 2 is a three-dimensional illustration of the principle of the holographic display in more detail. The reference is shown in Figure 1.

3 is a flow chart of the method of the present invention shown in accordance with an embodiment. The present embodiment provides an object point (OP) image color and depth map based on a three-dimensional scene (3D-S) containing a plurality of object points (OP). In this embodiment, the depth map of the image contains image depth information, and the color map contains pixelated color information provided by the image system.

The following steps can be performed for each visible object point:

Step (1): To obtain a light wave propagation data set (DP _VR ) by converting light wave propagation of an object point (OP) entering a visible area ( _VR ), the data set is referenced to a visible area (VR). According to the concept of the invention, the data set DP _VR can be extracted from at least one look-up table. The image depth information of the object points is used to query, read, and process the data. In the simplest embodiment, the normal distance of the object to the center of the visible area is used. The light waves of the object point (OP) can be simply converted into the visible area (VR), but the resulting values, such as the reference data set (DP _VR ), can be queried and read by at least one lookup table.

Step (2): Repeat the conversion until the entire scene (3D-S) has propagated all the object points. For each data point (OP), refer to the data area of the visible area (VR) (DΣ _VR ), according to the steps. 1 Read from at least one lookup table. For example, the data is extracted from the memory of the imaging system or captured by the interface, the data carrier, and the storage module.

In addition, the data set DP _VR can be modulated using the brightness and/or color values taken from the look-up table to correct the phase and/or amplitude of the complex values of the data set. For example, these complex values can be multiplied by a boost factor. Brightness and/or color values may optionally be extracted from at least one lookup table. It is also possible to extract the color data in the relevant data group from the self-checking table, or to superimpose the corresponding data table of the main color.

Furthermore, these data can be added to a data set (DΣ _VR ) that is referenced to the visible region (VR). It can also be explained by the following formula: DΣ _VR =Σ DP _VRi

Index i represents the data set DP _{VR of the} object point

In the present embodiment, the visible region constitutes a matrix similar to m columns and n columns. That is, the m value of the data set (DP _VR ) can be multiplied by the corresponding n value of the complex number, such as reading the matrix element in the self-check table. In this embodiment, the data set (DΣ _VR ) also has the same dimension.

Furthermore, the side effects of the spots can be reduced by smoothing and correcting the data set (DΣ _VR ) values. The amplitude extremes are reduced, ie all amplitude values are finite. Furthermore, corrections for any discontinuities in the amplitude and/or phase curves can be made. According to the simplest embodiment, the values of the matrix elements can be compared to the values of the adjacent elements of the horizontal and vertical, finding the discontinuities. These values can be further modified by a self-learning algorithm to reduce the chance of spotting, which is performed in the form of a neural network. Also taking into account the data set of the previous image of the video sequence, the probability of spot occurrence or the actual frequency is a suitable value function.

Step (3): inverse conversion, wherein the progressive data set (DΣ _VR ) extracted from the visible region is extracted from the visible region, and the scene is converted into the hologram of the optical modulator device (SLM) by the visible region (VR). On the plane (HP), a complete hologram (HΣ _SLM ), such as a video hologram, is generated to generate a complex hologram value.

It can be explained by the following formula: HΣ _SLM = T ¹ D Σ _VR where T ^{1 is} the inverse conversion operator.

The video hologram shows the hologram of all object points, and the video hologram can display and reconstruct the entire image.

In the last step (4), the hologram data may be encoded by a Burckhardt component, a bi-phase component, or any other suitable code to convert the complete hologram to the pixel value of the hologram display, preferably, based on the basis. The device disclosed in the documents WO 2004/044659, WO 2006/027228, WO 2006119760 and DE 10 2006 004 300.

The technology disclosed in this case can be implemented by a person familiar with the technology, and its unprecedented practice is also patentable, and the application for patent is filed according to law. However, the above embodiments are not sufficient to cover the scope of patents to be protected in this case. Therefore, the scope of the patent application is attached.

SH‧‧‧ child avatar

VR‧‧ visible area

VP‧‧‧ Observer

OP‧‧‧object points

3D-S‧‧‧ sight

SLM‧‧‧Space Light Modulator

HP‧‧‧Full image plane

HAE‧‧‧ holographic display

The present invention provides a more detailed description below with the aid of the embodiments and with reference to the accompanying drawings, wherein FIG. 1 is schematically illustrated as the principle of a holographic display; FIG. 2 is a three-dimensional schematic illustrating the principle of a holographic display. And Figure 3 shows a flow chart of the method of the invention in accordance with an embodiment.

Claims (13)

A method for a full-image display (HAE) to instantly generate a plurality of video holograms, wherein the hologram display (HAE) has at least one light modulator device (SLM) that is divided into a plurality of object points ( A scene of the OP) (3D-S) is encoded as a complete hologram (HΣ _SLM ) on the optical modulator device ( _SLM ), and the scene can be cycled from the reconstructed image of the video hologram A visible region (VR) within the interval sees the reconstruction result, wherein reconstruction of a single object point (OP) requires only one of the complete holograms (HΣ _SLM ) encoded on the optical modulator device A collection, characterized in that, for each object point (OP), the contribution of the transmission of light waves entering the visible region (VR) is extracted from at least one look-up table. The method of claim 1, wherein the position and viewing direction of an observer (VP) defines a field of view of the scene (3D-S), wherein at least one visible area (VR) assigned to the eye is assigned. To the observer, each visible object point (OP) of the scene (3D-S) includes the following steps: the transmission of the light wave of the object point (OP) by the visible area (VR) Internal conversion to obtain a data set (DP _VR ) of the transmission, the data set relates to the visible area (VR); the repetition of the conversion until all object points of the entire scene (3D-S) are converted, The result of summing the individual conversions (DP _VR ) to describe an Aggregated Wave Field of the complete scene (3D-S) of the visible region (VR) in a data set (DΣ _VR ), the progressive The wave field system relates to the visible region (VR); a back-transformation in which a newly determined progressive data set (DΣ _VR ) involving the visible region (VR) is converted from the visible region (VR) to the light tone The holographic image (HP) in which the transducer device (SLM) is located, thereby generating a complex holographic value of the holographic image of the image, characterized by : The transfer of each object point (OP), to enter the visible region (VR) of the light wave of the data set (DP _VR) system from at least one look-up table to retrieve. The method of claim 1 or 2, wherein for a plurality of object points (OP), the data set (DP _VR ) of the transmission of the light wave entering the visible area (VR) is involved. Modulated by the brightness and/or color values, or by the brightness and/or color values of at least one look-up table. The method of claim 1 or 2, wherein the correction value is modulated into the data set (DP _VR ) of the transmission involving the light wave entering the visible area ( _VR ), and/or The progressive data set D Σ _{VR of} the visible region, and/or the complex holographic value. The method of claim 1 or 2, wherein the value of the progressive data set (DΣ _VR ) related to the visible region (VR) is smoothed, or wherein the visible region (VR) is involved The value of the progressive data set (DΣ _VR ) is smoothed and the limit of the amplitude is limited. The method of claim 5, wherein the discontinuous portions of the amplitude and/or phase curves are homogenized. The method of claim 1 or 2, wherein the keying data in the data set (DΣ _VR ) is corrected with the aid of an optimization and/or self-learning algorithm to reduce the image. The probability of occurrence of spots is not expected in the hologram. The method of claim 1 or 2, wherein the complex hologram value is converted into a pixel value of the optical modulator device (SLM), or wherein the complex number The hologram value is converted to a Burckhardt component or a bi-phase component or any other suitable code. The method of claim 1 or 2, wherein the look-up table transmits the light wave of the object point (OP) by converting to the visible area (VR), thereby targeting the defined space each possible object type information corresponding to data sets (DP _VR) of the decision point is generated, the data set (DP _VR) system relates to the visible region. The method of claim 1 or 2, wherein the lookup table determines the type of the corresponding data set (DP _VR ) by using the ray tracing method for each possible object point in the defined space. Generated from the data, the data set (DP _VR ) is related to the visible area. The method of claim 1 or 2, wherein the lookup table determines the corresponding data set for each possible object point in the defined space with the aid of the optimization and approximation algorithms ( DP _VR ) is generated by typing data, and the data set (DP _VR ) is related to the visible area. The method of claim 1 or 2, wherein the screen device is the light modulator device itself on which the full image of the scene is encoded, or an optical component, wherein the scene is in the light adjusting device A hologram or wavefront of the upper code is projected onto the optical element. The method of claim 12, wherein the optical component of the display device is a lens or a mirror.