CN113973198A - Holographic image generation method, device and equipment and computer readable storage medium - Google Patents
Holographic image generation method, device and equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN113973198A CN113973198A CN202010712801.6A CN202010712801A CN113973198A CN 113973198 A CN113973198 A CN 113973198A CN 202010712801 A CN202010712801 A CN 202010712801A CN 113973198 A CN113973198 A CN 113973198A
- Authority
- CN
- China
- Prior art keywords
- image
- hologram
- images
- image processing
- holographic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 136
- 238000012545 processing Methods 0.000 claims abstract description 204
- 238000004364 calculation method Methods 0.000 claims abstract description 76
- 238000010586 diagram Methods 0.000 claims description 32
- 238000009826 distribution Methods 0.000 claims description 24
- 230000015654 memory Effects 0.000 claims description 11
- 238000001093 holography Methods 0.000 claims description 5
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 38
- 238000004587 chromatography analysis Methods 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 9
- 238000003325 tomography Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000003672 processing method Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000010287 polarization Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/388—Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume
Abstract
The embodiment of the application provides a holographic image generation method, a device, equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring at least two images and a time stamp corresponding to each image; determining at least two image processing units for processing the at least two images; inputting each image into an image processing unit in sequence according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain holograms of the images; and inputting the hologram of each image into a spatial light modulator in sequence to obtain the holographic image. By the method and the device, the efficiency of holographic processing can be improved, and accelerated calculation of the dynamic holographic image is effectively realized.
Description
Technical Field
The embodiment of the application relates to the field of holographic display, and relates to but is not limited to a holographic image generation method, a holographic image generation device and a computer readable storage medium.
Background
The holographic three-dimensional display technology refers to the technology that three-dimensional information can be observed by naked eyes. However, due to the demand for the size and resolution of the three-dimensional model, the amount of data to be calculated in the calculation process of the hologram is multiplied, so that the calculation time of the hologram becomes a bottleneck in development. In addition, because the hologram calculation time is long, static display is mainly performed at present, and no scheme for dynamic display is available.
In the related art, a single hologram is obtained mainly by performing accelerated calculation with a single Graphics Processing Unit (GPU), or a single hologram is obtained by performing accelerated calculation with multiple GPUs, that is, in the related art, there is no method for efficiently performing accelerated calculation on a dynamic hologram due to accelerated calculation on a static single hologram.
Disclosure of Invention
The embodiment of the application provides a holographic image generation method, a holographic image generation device, equipment and a computer readable storage medium.
The technical scheme of the embodiment of the application is realized as follows:
in a first aspect, an embodiment of the present application provides a method for generating a hologram, including: acquiring at least two images and a time stamp corresponding to each image; determining at least two image processing units for processing the at least two images; inputting each image into an image processing unit in sequence according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain holograms of the images; and inputting the hologram of each image into a spatial light modulator in sequence to obtain the holographic image.
In a second aspect, an embodiment of the present application provides a hologram generating apparatus, including: the acquisition module is used for acquiring at least two images and a time stamp corresponding to each image; a determining module for determining at least two image processing units for processing the at least two images; the first input module is used for sequentially inputting each image into one image processing unit according to the sequence of the timestamps, so that the images are subjected to holographic processing through the corresponding image processing units to obtain the holograms of the images; and the second input module is used for sequentially inputting the hologram of each image into the spatial light modulator to obtain the holographic image.
In a third aspect, an embodiment of the present application provides a hologram generating apparatus, including: a memory for storing executable instructions; and the processor is used for realizing the method when executing the executable instructions stored in the memory.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing executable instructions for causing a processor to implement the above-mentioned method when executed.
In the embodiment of the application, at least two images and a time stamp corresponding to each image are obtained; determining at least two image processing units for processing at least two images; and then, sequentially inputting each image into an image processing unit according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain holograms of the images so as to obtain holographic images. Therefore, the dynamic holographic image is generated by adopting the plurality of image processing units to carry out holographic processing on the plurality of images in sequence, so that the holographic processing efficiency can be improved, and the accelerated calculation of the dynamic holographic image is effectively realized.
Drawings
FIG. 1A is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present disclosure;
fig. 1B is a schematic view of an application scenario of the hologram generation method according to the embodiment of the present application;
FIG. 2 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present disclosure;
FIG. 4 is a schematic flow chart of an alternative tomography method in the hologram creation method provided in the embodiment of the present application;
FIG. 5 is a schematic flow chart of an alternative tomography method in the hologram creation method provided in the embodiment of the present application;
FIG. 6 is a schematic diagram illustrating the distribution of thread blocks in a GPU according to an embodiment of the present disclosure;
FIG. 7 is a schematic flow chart of an alternative tomography method in the hologram creation method according to the embodiment of the present application;
FIG. 8 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present disclosure;
FIG. 9 is a schematic flowchart of generating a dynamic hologram by using multi-GPU parallel computing based on chromatography according to an embodiment of the present disclosure;
FIG. 10 is a schematic diagram of a flow calculation process of a GPU according to an embodiment of the present disclosure;
FIG. 11 is a schematic flow chart illustrating an alternative method for generating a hologram according to an embodiment of the present disclosure;
FIG. 12 is a schematic diagram of a chromatography method provided in an embodiment of the present application;
FIG. 13 is a comparison graph of the calculated speed increase results provided by the embodiments of the present application;
fig. 14 is a schematic structural diagram of a hologram generating apparatus according to an embodiment of the present disclosure;
fig. 15 is a schematic structural diagram of a hologram generating apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of the present application belong. The terminology used in the embodiments of the present application is for the purpose of describing the embodiments of the present application only and is not intended to be limiting of the present application.
Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.
1) Holographic image technology: the holography technique is a technique for recording and reproducing a real three-dimensional image of an object by using the principles of interference and diffraction. The first step is to record object light wave information by using the interference principle, namely the shooting process: the shot object forms a diffused object beam under the irradiation of laser; the other part of laser beam is used as reference beam to irradiate the holographic film and is superposed with the object beam to generate interference, and the phase and amplitude of each point on the object light wave are converted into the intensity which is changed in space, so that the contrast and interval between interference fringes are used to record all information of the object light wave.
2) Chromatography: the method includes layering 3D objects along the depth direction, adding the complex amplitude distribution of two-dimensional sectional images of all layers imaged on the holographic surface respectively to serve as the complex amplitude distribution of the original three-dimensional object, adding reference light, and calculating by a Fresnel approximation method to obtain a hologram of a full object field.
In order to better understand the image processing method provided in the embodiment of the present application, first, an image processing method and problems in the related art are described:
the holographic three-dimensional display technology can observe three-dimensional information by naked eyes. However, due to the demand for the size and resolution of the three-dimensional model, the amount of data to be calculated in the calculation process of the hologram is multiplied, so that the calculation time of the hologram becomes a bottleneck in development. And because the hologram has long calculation time, the hologram is mainly displayed statically at present, and no scheme for dynamic display exists.
At present, two approaches are mainly used for improving the calculation speed, one is a software aspect, and the purpose of reducing the calculation amount is achieved by improving the improved algorithm and reducing the complexity of the algorithm. The other method is to change hardware devices, to accelerate performance by using the progress of the hardware devices with other high-performance architectures, and to accelerate the performance by using hardware devices such as a GPU, and mainly includes three methods: at present, the point cloud method and the triangular patch method are mainly used.
However, improvements in software or hardware devices have at least one of the following problems: firstly, the related art mainly uses a single GPU to perform acceleration to calculate a single hologram, or uses multiple GPUs to perform acceleration to calculate a single hologram, that is, both static single holograms and dynamic holograms are calculated in advance; secondly, the related technologies are calculated based on a point cloud method, so that the calculated data volume of different three-dimensional objects is variable, and the calculation time of each hologram is unpredictable; third, current research is mainly directed to the generation of a single static hologram, and no further research is done on dynamic holograms.
Based on at least one of the above problems in the related art, the embodiments of the present application provide a method for accelerating generation of a dynamic hologram by using multiple GPUs, which can calculate dynamic three-dimensional model data based on multiple GPUs and generate a dynamic hologram with a fixed refresh frequency. Acquiring at least two images and a time stamp corresponding to each image; and determining at least two image processing units (e.g., GPUs) for processing the at least two images; and then, sequentially inputting each image into an image processing unit according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain holograms of the images so as to obtain holographic images. Therefore, the dynamic holographic image is generated by adopting the plurality of image processing units to carry out holographic processing on the plurality of images in sequence, so that the holographic processing efficiency can be improved, and the accelerated calculation of the dynamic holographic image is effectively realized.
An exemplary application of the hologram generating apparatus provided in the embodiment of the present application is described below, and the hologram generating apparatus provided in the embodiment of the present application may be implemented as various types of terminals such as a notebook computer, a tablet computer, a desktop computer, a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device), and may also be implemented as a server. Next, an exemplary application when the hologram generating apparatus is implemented as a server will be described.
Referring to fig. 1A, fig. 1A is an alternative schematic flow chart of a hologram generation method provided in an embodiment of the present application, and will be described with reference to the steps shown in fig. 1A.
Step S101, at least two images and a time stamp corresponding to each image are obtained.
Here, the at least two images are images for generating a final hologram, and the images may be collected by an image collecting device on the terminal or downloaded on any website.
In the embodiment of the application, the server acquires the images and also acquires the time stamp corresponding to each image, wherein each image in the at least two images corresponds to one time stamp, and the time stamp of each image is different from the time stamps of other images. The timestamp may be the time when the server acquires the image, or may be a timestamp added to each image by the server based on a certain timestamp rule after the server acquires the image, or may be a timestamp of each image formed by the at least two images according to a chronological order. The time stamps are used for determining the sequence of each image relative to other images in the subsequent image processing process.
Step S102, at least two image processing units for processing the at least two images are determined.
Here, the image processing unit is used for holographically processing the images, in some embodiments, the image processing unit may be a GPU, and then, the method of the embodiment of the present application is to holographically process at least two images by a plurality of GPUs.
In the embodiment of the present application, the number of the image processing units may be smaller than the number of the images, or may be equal to the number of the images. And the server determines an image processing unit which can complete the holographic processing of the image at present according to the current hardware condition.
In some embodiments, after the at least two image processing units are determined, the at least two image processing units may be numbered, so that in a subsequent image holography processing process, each image processing unit may be sequentially called in an order of numbering the image processing units to implement holography processing of an image.
And step S103, sequentially inputting each image into an image processing unit according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain the holograms of the images.
Here, in the order of the time stamps, an image is input to an image processing unit one at a time, and the image is holographically processed by the image processing unit. When the number of the image processing units is equal to the number of the images, each image processing unit performs holographic processing on one image; when the number of the image processing units is smaller than the number of the images, traversing all the image processing units in sequence according to the numbers of the image processing units to enable all the image processing units to realize the holographic processing process on part of the images, then, starting from the initial number, continuously and circularly traversing all the image processing units, distributing the images for each image processing unit to complete the holographic processing until the holographic processing on all the images is completed. According to the scheme of the embodiment of the application, the holographic processing is performed on the image through the plurality of image processing units, and the plurality of images can be processed simultaneously through the plurality of image processing units in parallel, so that the image processing efficiency is greatly improved.
In the embodiment of the application, an image processing unit processes a hologram to form a hologram, wherein the hologram is a three-dimensional image which is greatly different from a traditional photo which presents a real physical image, and the hologram contains information such as the size, shape, brightness and contrast of a recorded object, and the information is stored in a tiny but complex interference pattern generated by laser.
The holographic processing process of the embodiment of the application adopts a holographic technology, and the holographic technology is a technology capable of realizing recording and reproduction of a real three-dimensional image. Unlike other three-dimensional "images," holograms provide "parallax". The presence of parallax allows the observer to observe different aspects of the image by moving back and forth, left and right, and up and down as if there were a real object.
And step S104, inputting the hologram of each image into a spatial light modulator in sequence to obtain the hologram.
Here, the Spatial Light Modulator (SLM) is a device that can modulate a parameter of an optical field through liquid crystal molecules under active control, for example, by modulating the amplitude of the optical field, modulating the phase through a refractive index, modulating the polarization state through rotation of a polarization plane, or realizing conversion of incoherent-coherent Light, so as to write a certain information into an optical wave, thereby achieving the purpose of optical wave modulation.
In the embodiment of the application, the hologram obtained after each image is subjected to holographic processing can be input into the spatial light modulator in sequence according to the sequence of the timestamps of the images, so that the obtained hologram is also an image formed according to the sequence of the timestamps of the images, and the dynamic display of the hologram of the three-dimensional image is realized.
Fig. 1B is a schematic view of an application scenario of a hologram generation method provided in an embodiment of the present application, and as shown in fig. 1B, a hologram generation system 10 for implementing the hologram generation method includes a terminal 100, a network 200, and a server 300, where a user acquires at least two images for generating a hologram and a timestamp corresponding to each image through the terminal 100, and sends the at least two images and the timestamp of each image to the server 300 through the network 200, so that the server 300 determines at least two image processing units for processing the at least two images; inputting each image into an image processing unit in sequence according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain holograms of the images; and inputting the hologram of each image into a spatial light modulator in sequence to obtain the holographic image. After obtaining the hologram, the server 300 transmits the hologram to the terminal 100 through the network 200, and the terminal 100 can play the hologram on the current interface 100-1.
The holographic image generation method provided by the embodiment of the application obtains at least two images and a timestamp corresponding to each image; determining at least two image processing units for processing at least two images; and then, sequentially inputting each image into an image processing unit according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain holograms of the images so as to obtain holographic images. Therefore, the dynamic holographic image is generated by adopting the plurality of image processing units to carry out holographic processing on the plurality of images in sequence, so that the holographic processing efficiency can be improved, and the accelerated calculation of the dynamic holographic image is effectively realized.
In some embodiments, the hologram generating method may be applied to a hologram generating system, where the hologram generating system includes a terminal (i.e., an image capturing device), a server, an image processing unit (in the embodiments of the present application, the image processing unit is exemplified as a GPU), and a spatial light modulator, the terminal performs image capturing to obtain at least two images, and sends the at least two captured images to the server, and the server controls the at least two GPUs to perform hologram generation on the at least two images.
Fig. 2 is a schematic flow chart of an alternative hologram generation method provided in an embodiment of the present application, and as shown in fig. 2, the method includes the following steps:
step S201, the terminal collects at least two images in sequence according to the time sequence.
Here, the terminal may sequentially acquire at least two images according to a time sequence by using its own image acquisition unit (e.g., a camera), or the terminal may sequentially download at least two images from any website according to the time sequence, or the terminal may sequentially receive at least two images sent by other terminals according to the time sequence. Of course, in other embodiments, the terminal may also acquire the image in any other manner, which is not limited in this embodiment of the application.
Step S202, the terminal sends at least two images and the time stamp corresponding to each image to the server.
Here, the timestamp may correspond to a time when the terminal acquires an image, or the timestamp may be added to each image according to a chronological order of the acquisition after the terminal acquires the image, or may be set by the user for each image according to an order of images played by a hologram that the user wants to form.
In step S203, the server determines at least two GPUs for processing the at least two images.
And step S204, the server inputs each image into a GPU in sequence according to the sequence of the timestamps.
For example, the determined number of GPUs is 3, which are GPU1, GPU2, and GPU3, the number of images to be processed is 5, which are image 1, image 2, image 3, image 4, and fig. 5, and the images 1 to 5 are numbered in sequence according to the sequence of time stamps, then, image 1 may be input into GPU1, image 2 is input into GPU2, image 3 is input into GPU3, holographic processing is performed on images 1 to 3 by GPU1 to GPU3, image 4 is input into GPU1 after the processing by GPU1 to GPU3 is completed, and image 5 is input into GPU2, so that the images are input into the GPU cyclically until all the images are input.
Step S205, the GPU performs holographic processing on the image to obtain a hologram of the image.
In the embodiment of the application, the GPU adopts a holographic technology to perform holographic calculation on the image so as to complete holographic processing on the image and obtain the hologram of the image.
In step S206, the server sequentially controls the GPU to input the hologram of each image into the spatial light modulator.
Here, after the GPU completes the holographic processing on the image to obtain the hologram, since the processing time of the GPU for each image is the same, the holograms may be sequentially formed in the order of the time stamps, and after the hologram is formed, the holograms obtained by the corresponding processing may be sequentially input into the spatial light modulator in the order of the time stamps of the image.
Step S207, the spatial light modulator performs spatial light beam modulation on the hologram of each of the at least two images to obtain a hologram corresponding to the at least two images.
Here, the spatial light modulator spatially modulates the beam of light on the hologram after acquiring the hologram, and thus, information of an image can be loaded into a one-dimensional or two-dimensional light field.
And step S208, the spatial light modulator sends the hologram to a terminal.
And step S209, the terminal dynamically displays the holographic image on the current interface.
In some embodiments, after receiving the hologram, the terminal may display the hologram on its current interface, or project the hologram onto a specific display screen or display curtain, so as to play the hologram.
According to the holographic image generation method provided by the embodiment of the application, the terminal, the server, the GPU and the spatial light modulator cooperate to complete the generation of the holographic image and the playing process after the generation, and because a plurality of GPUs are used for simultaneously processing a plurality of images in the process of generating the holographic image, a plurality of GPUs are used for simultaneously performing holographic processing on a plurality of images to generate the dynamic holographic image, the holographic processing efficiency can be improved, and the dynamic holographic image can be effectively accelerated to be calculated.
Fig. 3 is an optional schematic flow chart of the hologram image generation method according to the embodiment of the present application, and as shown in fig. 3, in some embodiments, before step S103, the method further includes the following steps:
and S301, sequentially extracting the image data of each image according to the sequence of the timestamps.
Here, the image data is a pixel value of each pixel point in the image, or a pixel value of each object point in the image.
Correspondingly, step S103 may be implemented by:
step S302, sequentially inputting the image data of each image into one image processing unit according to the sequence of the timestamps, and performing holographic calculation on the image data of the images through the corresponding image processing units to obtain the holograms of the images.
Here, a hologram of an image is obtained by performing holographic computation on image data using a holographic technique. When inputting image data of an image into an image processing unit, the image data may be randomly input into an image processing unit currently in an idle state, that is, the image data is input into an image processing unit which is not currently subjected to holographic processing.
Referring to fig. 3, in other embodiments, step S103 can be further implemented by:
step S303, sorting at least two image processing units to form an image processing unit sequence.
Here, the image processing units may be randomly ordered to form a sequence of image processing units.
Step S304, according to the sequence of the timestamps and the sequence of the image processing units in the image processing unit sequence, sequentially inputting each image into one image processing unit in the image processing unit sequence, and performing holographic processing on the images through the corresponding image processing units to obtain the holograms of the images.
Here, the at least two images form an image sequence according to the sequence of the time stamps, and then one image and one image processing unit may be sequentially selected according to the image sequence and the image processing unit sequence, and the image may be input to the image processing unit.
Referring to fig. 3, in still other embodiments, step S103 can be further implemented by:
in step S305, the image corresponding to the current timestamp is determined as the target image.
Step S306, among the at least two image processing units, determining any image processing unit currently in an idle state as a target image processing unit.
Step S307, inputting the target image into the target image processing unit, and performing holographic processing on the target image by the target image processing unit to obtain a hologram of the target image.
Step S308, determine whether the current timestamp is the last timestamp.
If yes, executing step S104; if the judgment result is no, when the time reaches the next time stamp, the process returns to the step S305 until the holography processing is completed for the images corresponding to all the time stamps.
In some embodiments, the step of performing holographic processing on the image in step S103 by using the corresponding image processing unit to obtain the hologram of the image may be implemented by using a tomography method, where the tomography method is to layer the 3D object in the depth direction, add the complex amplitude distributions of the two-dimensional cross-sectional images of each layer on the holographic surface, and use the sum as the complex amplitude distribution of the original three-dimensional object, add the reference light, and perform calculation by using a fresnel approximation method to obtain a hologram of a full object field.
Fig. 4 is a schematic flow chart of an alternative tomography method in the hologram generation method provided in the embodiment of the present application, and as shown in fig. 4, the tomography step in step S103 may be implemented by the following steps:
step S401, determining the number of layers of the image.
And S402, layering the images according to the layering quantity to obtain the cross-sectional diagram data of the layering quantity.
Here, the plane corresponding to the sectional view data may be a two-dimensional sectional view. In the implementation process, the image is an image in a three-dimensional space, and when layering, the image can be intercepted along a plane perpendicular to a Z axis in the three-dimensional space to obtain cross-sectional view data on different two-bit cross-sectional views under different Z-axis coordinates, wherein the cross-sectional view data reflects the parallel distribution of the image on the Z axis.
In step S403, a thread block in the corresponding image processing unit is allocated to each cross-sectional image data.
Here, the thread block is a calculation unit in the image processing unit, and each image processing unit can simultaneously realize calculation of a plurality of image data, and therefore, synchronous calculation of a plurality of processes can be realized using the thread block in the image processing unit.
And S404, performing holographic calculation on the cross-sectional diagram data by using the threads in the thread blocks to obtain the planar diffraction light waves corresponding to the cross-sectional diagram data.
Here, one thread block is used for processing one cross-sectional view data, and one thread block includes a plurality of threads, so that the plurality of threads in one thread block can be used to perform holographic calculation on the same cross-sectional view data at the same time to obtain the diffracted light wave of the plane corresponding to the cross-sectional view data.
Step S405, obtaining object light waves of the corresponding plane from the diffracted light waves of the plane corresponding to each of the cross-sectional view data.
And step S406, superposing the object light waves of the layered quantity to obtain the hologram data of the image.
Step S407, determining a hologram of the image according to the hologram data.
Here, after performing the holographic calculation on each cross-sectional view data, the same number of planes of diffracted light waves as the number of layers are obtained, and then, an object light wave, which is a diffracted light wave corresponding to an object in an image, is obtained from the diffracted light waves of each plane, and then all the object light waves are superimposed. Since the number of the sectional image data is the number of the layers, and therefore, the number of the planes is also the number of the layers, and the number of the determined object light waves is the number of the layers, the object light waves with the number of the layers can be superposed to obtain hologram data of the image, and finally, the hologram of the image is determined according to the hologram data. The sectional view data is data of sectional views corresponding to the same image and at different Z-axis coordinates, and the object light wave is a diffracted light wave corresponding to an object in a plane corresponding to the same image and at different Z-axis coordinates, and therefore, the object light waves are superimposed to obtain hologram data of a hologram of the image.
It should be noted that the number of layers in the chromatography determines the longitudinal resolution of the final hologram, and the larger the number of layers, the higher the resolution. Meanwhile, the increase of the number of layers also increases the amount of calculation, so that the number of layers needs to be reduced as much as possible while the resolution requirement is met.
Fig. 5 is an optional schematic flow chart of a tomography method in the hologram generation method provided in the embodiment of the present application, and as shown in fig. 5, step S401 may be implemented by the following steps:
step S501, determining the image depth of each object in the image to obtain at least two depth values.
Here, the image depth of the object corresponds to the above-mentioned Z-axis coordinate, and the object at different depth values reflecting the distance of the actual object from the photographing unit (or photographer) corresponds to different Z-axis coordinates.
Step S502, determining a maximum depth value and a minimum depth value among the at least two depth values.
Here, the object corresponding to the maximum depth value is farthest from the photographer, and the object corresponding to the minimum depth value is closest to the photographer.
Step S503, determining the number of layers of the image according to the maximum depth value and the minimum depth value.
Here, the integration may be performed between the maximum depth value and the minimum depth value, or by calculating a distance difference between the maximum depth value and the minimum depth value and determining the number of hierarchies according to the distance difference.
With continued reference to fig. 5, in some embodiments, each cross-sectional view includes at least one object point, and correspondingly, the step S404 can be implemented by:
step S504, object point data corresponding to each object point in the plane corresponding to the cross-sectional view data is obtained.
And step S505, performing Fourier transform calculation on object point data of one object point by adopting one thread in the thread blocks to obtain a distribution result of object point diffraction light waves of the corresponding object point.
As shown in fig. 6, which is a schematic distribution diagram of Thread blocks in a GPU provided by the embodiment of the present application, on a GPU 60, a computing unit on the GPU is divided into a plurality of (e.g., 2-3) grids (Grid)61, each Grid 61 includes a plurality of (e.g., 65535) Thread blocks (blocks) 62, and each Thread Block 62 includes a plurality of (e.g., 512) threads (threads) 63. A parallel program of a Unified computing Device Architecture (CUDA) is executed by a plurality of threads 63; a plurality of threads 63 are grouped into a thread block 62, and the threads 63 in the same thread block 62 can be synchronous or can communicate through a shared memory (shared memory); the thread blocks 62 are then reconstructed into a grid 61.
Step S506, performing summation on distribution results of the object point diffracted light waves of all or part of object points in the plane corresponding to the cross-sectional diagram data to obtain the diffracted light waves of the plane corresponding to the cross-sectional diagram data.
In the embodiment of the present application, the same GPU as in fig. 6 is used to process the same image; each image is divided into multiple cross-sectional views, and each thread block 62 in the GPU processes the same cross-sectional view; each cross-sectional view includes a plurality of object points, and each thread 63 in the thread block 62 processes object point data corresponding to one object point.
For example, there are 3 images for generating holograms, namely image 1, image 2 and image 3, and the number of GPUs is also 3, namely GPU1, GPU2 and GPU3, wherein GPU1 holographically processes image 1, GPU2 holographically processes image 2 and GPU3 holographically processes image 3. Here, the process of holographically processing the image 1 by the GPU1 is described as an example, and the process of holographically processing the image 2 by the GPU2 and the process of holographically processing the image 3 by the GPU3 may refer to the following process of holographically processing the image 1 by the GPU 1.
The process of the GPU1 to holographically process the image 1 is as follows: it is determined that the number of layers of the image 1 is 2, and therefore, the image 2 is divided into the cross-sectional view 11 and the cross-sectional view 12, and correspondingly, the GPU1 has a plurality of thread blocks, and the thread blocks 11 and 12 in the plurality of thread blocks can be used to perform holographic calculations on the cross-sectional view data of the cross-sectional view 11 and the cross-sectional view data of the cross-sectional view 12, respectively. The plane of the cross-sectional view 11 includes three object points, which are an object point 111, an object point 112, and an object point 113, the plane of the cross-sectional view 12 also includes three object points, which are an object point 121, an object point 122, and an object point 123, and the thread block 11 includes a plurality of threads. In the embodiment of the application, the threads 11a, 11b, and 11c in the thread block 11 are respectively used to perform fourier transform calculation on the object point data of the object points 111, 112, and 113, so as to correspondingly obtain the distribution result of the object point diffracted light waves of the object points 111, the distribution result of the object point diffracted light waves of the object points 112, and the distribution result of the object point diffracted light waves of the object points 113, and then the distribution result of the object point diffracted light waves of the object points 111, the distribution result of the object point diffracted light waves of the object points 112, and the distribution result of the object point diffracted light waves of the object points 113 are summed up to obtain the diffracted light waves of the plane where the cross-sectional view 11 is located. By the same method, the threads 12a, 12b and 12c in the thread block 12 are used to perform fourier transform calculation on the object point data of the object points 121, 122 and 123 respectively, so as to obtain the distribution result of the object point diffracted light waves of the object points 121, 122 and 123, and then the distribution result of the object point diffracted light waves of the object points 121, 122 and 123 is summed up to obtain the diffracted light waves of the plane where the cross-sectional view 12 is located.
In some embodiments, a method for dividing a cross-sectional view to obtain at least one object point is further provided, based on fig. 5, fig. 7 is an optional flowchart of a tomography method in the hologram generation method provided in the embodiment of the present application, and as shown in fig. 7, before acquiring object point data corresponding to each object point in step S504, the method further includes the following steps of dividing to obtain at least one object point:
step S701, obtaining a minimum resolution of the image and a pixel size of a display unit for playing the hologram.
Step S702, determining a pixel interval size between object points in a plane corresponding to each cross-sectional view data in the image according to a product of the minimum resolution and the pixel size.
Step S703, dividing the cross-sectional view data according to the pixel interval size to obtain the at least one object point.
Here, after the pixel interval size is determined, the sectional view data is divided according to the pixel interval size to obtain at least one object point. The spacing size reflects the size of the data space between each object point and the adjacent object point.
In some embodiments, in order to ensure that the input frequency of the holograms input into the spatial light modulator is synchronized with the refresh frequency of the spatial light modulator, the time interval for generating at least two images of the hologram may be predetermined, that is, the time stamp of each image is predetermined, so that the time interval between the images corresponding to each two adjacent time stamps is equal, and the frequency corresponding to the time interval between the images corresponding to each two adjacent time stamps is equal to the refresh frequency of the spatial light modulator. In order to achieve frequency synchronization, on the basis of any of the above embodiments, the hologram generating method may further include: step S11, obtaining a refresh frequency of the spatial light modulator. Step S12, sequentially determining the timestamp corresponding to each image according to the refresh frequency.
In the embodiment of the application, because the timestamp of the image is determined according to the refreshing frequency of the spatial light modulator, the input frequency and the processing frequency of the image are consistent with the refreshing frequency of the spatial light modulator, that is, the synchronization between the frequency of the hologram input to the spatial light modulator and the refreshing frequency of the spatial light modulator can be realized, so that the spatial light modulator performs common-frequency processing on the hologram, and a stable holographic image is obtained.
Next, an exemplary application of the embodiment of the present application in a practical application scenario will be described.
The embodiment of the application provides a holographic image generation method, which is characterized in that a three-dimensional object is segmented in a chromatography method to obtain target object optical data information, and then a multi-GPU calculation algorithm is improved and optimized, and a new GPU calculation process is provided for dynamic display of holographic images.
The embodiment of the present application provides a calculation process and a calculation process optimization scheme for generating a dynamic hologram by using multi-GPU parallel calculation based on a chromatography, fig. 8 is an optional flow diagram of a hologram generation method provided in the embodiment of the present application, and as shown in fig. 8, the implementation process includes the following steps:
step S801, a plurality of GPUs are sequenced and numbered according to a three-dimensional image (corresponding to the image) input sequence, dynamic three-dimensional model data (corresponding to the image data) are extracted according to a time sequence, and the image data are sequentially distributed to the GPU video memories;
here, the time interval of the three-dimensional image data is the refresh frequency of the spatial light modulator.
Step S802, each GPU is initialized, each GPU calculation algorithm program is started sequentially according to the refreshing time interval, three-dimensional model data at different moments are input into a GPU memory sequentially to wait for extraction calculation, and calculated calculation hologram results are projected into a spatial light modulator.
Here, since the principle of the tomographic calculation and the size of the image calculation data amount are fixed factors, the calculation time consumption of each hologram is the same, and thus, a relatively stable refresh frequency can be obtained, which is convenient for normal display viewing.
In some embodiments, the fresnel diffraction holographic algorithm may be optimized such that it may be improved by up to 60 times on average over the computation speed of a single GPU.
In step S803, the holograms are sequentially input under the spatial light modulator, so that the hologram of the three-dimensional model can be dynamically displayed.
Fig. 9 is a schematic flow chart of generating a dynamic hologram by using multi-GPU parallel computation based on a chromatography method according to the embodiment of the present application, and as shown in fig. 9, three-dimensional images (i.e., three-dimensional model data) at times t1, t2, t3, and t4 are sequentially and correspondingly input into a GPU G1, a GPU G2, a GPU G3, and a GPU G4, so as to correspondingly obtain a hologram 1, a hologram 2, a hologram 3, and a hologram 4; then, the hologram 1, the hologram 2, the hologram 3, and the hologram 4 are sequentially input to the spatial light modulator 91, and a hologram 92 is obtained and output.
In some embodiments, step S802 may be implemented by:
and S8021, reading the two-dimensional size of the three-dimensional model, solving the optimal layering number of the three-dimensional model, and determining the data volume of the total object point.
Here, the size of the interval between the two-dimensional plane object points may be set as the product between the minimum resolution of the three-dimensional model and the size of the pixel size of the display screen, and the optimum number of layers of the three-dimensional model is calculated using the following formula (1-1):
wherein N represents the optimal number of tiers; z represents a distance between the object and the photographing terminal (i.e., an image depth of the object); z is a radical ofmaxRepresents the maximum depth value; z is a radical ofminRepresenting a minimum depth value; Δ zeRepresenting an object andand shooting the change increment of the distance between the terminals in the three-dimensional model.
And step S8022, distributing the thread blocks and the thread quantity, and determining the optimal distribution and data transmission of the thread blocks, the threads and the computed hologram.
Here, since the GPU is experimentally measured when performing the hologram calculation, the maximum acceleration bottleneck is influenced by the data transmission bandwidth, so that the embodiment of the present application uses the processing procedure of the stream calculation of the GPU, so that bandwidth transmission is started in the calculation procedure, and the delay is reduced to the maximum extent. The optimal state of the GPU is that the data calculation time and the data transmission time just overlap.
As shown in fig. 10, which is a schematic diagram of a flow calculation processing process of a GPU provided in this embodiment of the present application, a Central Processing Unit (CPU) 110 divides three-dimensional image data into a plurality of cross-sectional views according to an image depth to obtain information such as arbitrary depth picture information 01 and arbitrary depth picture information 02, and sequentially inputs the arbitrary depth picture information 01 and the arbitrary depth picture information 02 into a thread block of the GPU 111, so as to implement data reception 1101 and Kernel function execution 1102 on the arbitrary depth picture information 01 or the arbitrary depth picture information 02 through the thread block, and correspondingly obtain complex amplitude 01 and complex amplitude 02. Finally, the complex amplitude 01, the complex amplitude 02, and the like are added and summed 1103 to obtain a hologram 1104, which is then output.
In some embodiments, the following rules may be followed when partitioning GPU threads and thread block distributions: the thread blocks are at least a multiple of the thread bundle 32 and at most the maximum number of threads of the thread blocks of the GPU.
And S8023, substituting the two-dimensional plane object point information obtained in the step S8021 into a chromatography formula, and superposing corresponding data calculated under different layer numbers to obtain the calculated hologram data at the moment.
Here, one thread bundle (or thread) corresponds to one object point, and the calculation of different layer planes can be performed randomly; when the number of the plane object points is larger than the number of the thread grids, traversing and sampling in a thread circulation mode; the results of the computations in each thread should be kept in the shared memory for the corresponding sequence number.
In the embodiment of the present application, the following formula (1-2) may be adopted to generate the computed hologram by the chromatography method:
wherein U (x, y) represents a result of the holographic computation; j is a plurality; k is a constant; z is a radical ofnRepresenting the distance of the object light from the reference plane; λ represents the wavelength of light; u shape0Representing the amplitude of the reference optical wave; (x)0,y0) Coordinates representing the diffraction plane, i.e. the reference coordinates of the object light waves; (x, y) represents the viewing plane coordinates.
Alternatively, the tomographic generation of the computed hologram can also be achieved by the following equations (1-3):
wherein U (xi, eta) represents the superposed holographic calculation result; u shapei(ξ, η) represent the ith holographic computation result to be superimposed; a (ξ, η) represents the magnitude of the superimposed complex amplitude; phi (xi, eta) represents the phase of the superimposed optical hologram.
In the embodiment of the application, the calculation hologram is generated by the formula (1-2) or the formula (1-3), so that the calculation precision can be simplified, and the calculation amount can be reduced. The calculation can be performed by using a simple formula in the GPU, for example, the tan function is replaced by the tan f function, so that the calculation amount can be greatly reduced.
In some embodiments, the holographic calculation algorithm of the fresnel diffraction is optimized, and each parameter variable in the fresnel diffraction algorithm is extracted and combined in the operation process, so that the calculation amount of repeated operation is further greatly reduced. Wherein, the formulas (1-3) can be extracted and combined, so that the formulas (1-3) can be regarded as the following formulas (1-4):
wherein the content of the first and second substances,
representing the number of reduction generations of the PC.
Fig. 11 is an alternative schematic flowchart of a hologram image generating method according to an embodiment of the present application, and as shown in fig. 11, the method includes the following steps: in step S1101, object depth and image information of the three-dimensional object are extracted. Step S1102 selects an arbitrary depth, and calculates a hologram cross-section of a cross-section of the depth. Step S1103, the superimposition depth is translated, and the hologram cross-sectional image is calculated. And step S1104, generating a hologram according to the calculated hologram cross-section.
It should be noted that the principle of the chromatography is that a three-dimensional object can be discretized into a series of parallel cross sections perpendicular to the Z-axis, as shown in fig. 12, which is a schematic diagram of the chromatography provided in the embodiment of the present application, each plane pattern (e.g., the plane patterns 121, 122, and 123 in fig. 12) is a two-dimensional array, the distribution of the diffracted light waves can be conveniently calculated by using fast two-dimensional fourier transform, then the diffracted light waves of each plane pattern are calculated, finally the object light waves of all the plane patterns are superimposed and the phase information thereof is extracted, and finally the phase information superimposed on the plane is the hologram data result 124 to be obtained.
According to the holographic image generation method provided by the embodiment of the application, the dynamic three-dimensional virtual object is dynamically and holographically displayed by using the GPU, and dynamic refreshing can be realized. Furthermore, the processing time of the hologram of the three-dimensional object is controlled within the same range by the chromatography, and the refresh frequency of the hologram generation is stabilized. Meanwhile, the transmitted video images are respectively circularly put into the GPU according to a certain time sequence, so that the generated holograms can be sequentially output.
The holographic image generation method provided by the embodiment of the application at least has the following technical advantages: firstly, compared with other methods such as a point cloud method and a triangular patch method, the other methods have the advantages that the calculated data amount is changed and the calculation time is also changed.
Secondly, a calculation algorithm of the calculation hologram under the GPU is optimized, so that the calculation speed of the calculation hologram is greatly improved, wherein fig. 13 is a comparison graph of calculation speed improvement results provided in the embodiment of the present application, as shown in fig. 13, an abscissa is the number of layers (number of layers), and an ordinate is calculation time, wherein three curves from bottom to top in fig. 13 are a change curve of calculation time with number of layers calculated by using GPU 1050Ti + CUDA, a change curve of calculation time with number of layers calculated by using CPU I5+ mathematical Kernel Library (MKL, Math Kernel Library) function, and a change curve of calculation time with number of layers calculated by using CPU I5+ Matrix Laboratory (MATLAB, Matrix Laboratory) function, respectively. Obviously, the method of the embodiment of the present application (i.e., the method using GPU 1050Ti + CUDA) has the shortest calculation time, i.e., the highest calculation rate, and the calculation time does not rise significantly as the number of layers increases.
Thirdly, compared with other methods in which data transmission and data calculation are separately performed, the embodiment of the present application uses a stream calculation processing manner in the GPU, so that calculation and data transmission are performed simultaneously, and GPU cores are allocated, so that the dynamic equivalence of the calculation time and the transmission time is achieved.
Fourthly, the other methods have the problem that dynamic holographic display cannot be performed because the three-dimensional model processing time at each moment is uncertain, and the method of the embodiment of the application can achieve dynamic holographic display.
Based on the foregoing embodiments, the embodiments of the present application provide a hologram generating apparatus, where the apparatus includes modules and components included in the modules, and may be implemented by a processor in a hologram generating device; of course, it may also be implemented by logic circuitry; in the implementation process, the Processor may be a Central Processing Unit (CPU), a microprocessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like.
Fig. 14 is a schematic structural diagram of a holographic image generating apparatus provided in an embodiment of the present application, and as shown in fig. 14, the holographic image generating apparatus 1400 includes:
an obtaining module 1401, configured to obtain at least two images and a timestamp corresponding to each image; a determining module 1402 for determining at least two image processing units for processing the at least two images; a first input module 1403, configured to sequentially input each image into one image processing unit according to the sequence of the timestamps, so as to perform holographic processing on the image through a corresponding image processing unit, so as to obtain a hologram of the image; a second input module 1404, configured to input the hologram of each image into a spatial light modulator in sequence, so as to obtain the hologram.
In some embodiments, the apparatus further comprises: the extraction module is used for sequentially extracting the image data of each image according to the sequence of the timestamps; correspondingly, the first input module is further configured to sequentially input the image data of each image into one of the image processing units according to the sequence of the timestamps, so that the image data of the image is subjected to holographic computation through the corresponding image processing unit to obtain a hologram of the image.
In some embodiments, the first input module is further configured to: sequencing the at least two image processing units to form an image processing unit sequence; and sequentially inputting each image into one image processing unit in the image processing unit sequence according to the sequence of the timestamps and the sequence of the image processing units in the image processing unit sequence, so as to perform holographic processing on the images through the corresponding image processing units, and obtain the holograms of the images.
In some embodiments, the first input module is further configured to: determining an image corresponding to the current timestamp as a target image; determining any image processing unit which is in an idle state currently in the at least two image processing units as a target image processing unit; and inputting the target image into the target image processing unit, and performing holographic processing on the target image through the target image processing unit to obtain a hologram of the target image.
In some embodiments, the apparatus further comprises a holographic processing module to determine a number of layers of the image; layering the images according to the layering quantity to obtain cross-sectional diagram data of the layering quantity; allocating a thread block in the corresponding image processing unit for each section data; performing holographic calculation on the cross-sectional diagram data by using threads in the thread blocks to obtain a planar diffraction light wave corresponding to the cross-sectional diagram data; acquiring object light waves of corresponding planes from the diffraction light waves of the planes corresponding to the data of each cross-sectional diagram; superposing the object light waves of the layered quantity to obtain hologram data of the image; determining a hologram of the image from the hologram data.
In some embodiments, the holographic processing module is further to: determining the image depth of each object in the image to obtain at least two depth values; determining a maximum depth value and a minimum depth value among the at least two depth values; and determining the layering number of the image according to the maximum depth value and the minimum depth value.
In some embodiments, each cross-sectional view data corresponds to a plane that includes at least one object point; the holographic processing module is further to: acquiring object point data corresponding to each object point in a plane corresponding to the cross-sectional drawing data; performing Fourier transform calculation on object point data of an object point by adopting one thread in the thread block to obtain a distribution result of object point diffraction light waves of the corresponding object point; and accumulating and summing the distribution results of the object point diffracted light waves of all or part of object points in the plane corresponding to the cross-sectional diagram data to obtain the diffracted light waves of the plane corresponding to the cross-sectional diagram data.
In some embodiments, the apparatus further comprises: the second acquisition module is used for acquiring the minimum resolution of the image and the size of the pixel size of a display unit for playing the hologram; a second determining module, configured to determine, according to a product of the minimum resolution and the size of the pixel size, a pixel spacing size between object points in a plane corresponding to each cross-sectional view data in the image; and the dividing module is used for dividing the data of the cross-sectional diagram according to the pixel interval size to obtain the at least one object point.
In some embodiments, the apparatus further comprises: a third obtaining module, configured to obtain a refresh frequency of the spatial light modulator; and the third determining module is used for sequentially determining the time stamp corresponding to each image according to the refreshing frequency.
It should be noted that the description of the apparatus in the embodiment of the present application is similar to the description of the method embodiment, and has similar beneficial effects to the method embodiment, and therefore, the description is not repeated. For technical details not disclosed in the embodiments of the apparatus, reference is made to the description of the embodiments of the method of the present application for understanding.
In the embodiment of the present application, if the image processing method is implemented in the form of a software functional module and sold or used as a standalone product, the image processing method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a terminal to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
Correspondingly, an embodiment of the present application provides a holographic image generating apparatus, fig. 15 is a schematic structural diagram of the holographic image generating apparatus provided in the embodiment of the present application, and as shown in fig. 15, the holographic image generating apparatus 1500 at least includes: a processor 1501, a communication interface 1502, and a computer-readable storage medium 1503 configured to store executable instructions, wherein the processor 1501 generally controls the overall operation of the hologram generating apparatus. The communication interface 1502 may enable the hologram creation apparatus to communicate with other terminals or servers via a network. The computer-readable storage medium 1503 is configured to store instructions and applications executable by the processor 1501, and may also cache data to be processed or processed by each module in the processor 1501 and the hologram image generating apparatus 1500, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
It should be noted that, in this document, the terms "comprises", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion, so that a process, a method or an apparatus including a series of elements includes not only those elements but also other elements not explicitly listed or inherent to such process, method or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented.
The above description is only for the embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (12)
1. A method of generating a hologram, comprising:
acquiring at least two images and a time stamp corresponding to each image;
determining at least two image processing units for processing the at least two images;
inputting each image into an image processing unit in sequence according to the sequence of the timestamps, and performing holographic processing on the images through the corresponding image processing units to obtain holograms of the images;
and inputting the hologram of each image into a spatial light modulator in sequence to obtain the holographic image.
2. The method of claim 1, further comprising: sequentially extracting the image data of each image according to the sequence of the timestamps;
correspondingly, the image data of each image is sequentially input into one image processing unit according to the sequence of the time stamps, so that the holographic calculation is performed on the image data of the images through the corresponding image processing units, and the holograms of the images are obtained.
3. The method according to claim 1, wherein the sequentially inputting each of the images into an image processing unit according to the sequence of the timestamps to perform holographic processing on the image by the corresponding image processing unit to obtain the hologram of the image comprises:
sequencing the at least two image processing units to form an image processing unit sequence;
and sequentially inputting each image into one image processing unit in the image processing unit sequence according to the sequence of the timestamps and the sequence of the image processing units in the image processing unit sequence, so as to perform holographic processing on the images through the corresponding image processing units, and obtain the holograms of the images.
4. The method according to claim 1, wherein the sequentially inputting each of the images into an image processing unit according to the sequence of the timestamps to perform holographic processing on the image by the corresponding image processing unit to obtain the hologram of the image comprises:
determining an image corresponding to the current timestamp as a target image;
determining any image processing unit which is in an idle state currently in the at least two image processing units as a target image processing unit;
and inputting the target image into the target image processing unit, and performing holographic processing on the target image through the target image processing unit to obtain a hologram of the target image.
5. The method according to claim 1, wherein the holography processing of the image by the corresponding image processing unit to obtain the hologram of the image comprises:
determining the number of layers of the image;
layering the images according to the layering quantity to obtain cross-sectional diagram data of the layering quantity;
distributing a thread block in the corresponding image processing unit for each section data;
performing holographic calculation on the cross-sectional diagram data by using threads in the thread blocks to obtain a planar diffraction light wave corresponding to the cross-sectional diagram data;
acquiring object light waves of corresponding planes from the diffraction light waves of the planes corresponding to the data of each cross-sectional diagram;
superposing the object light waves of the layered quantity to obtain hologram data of the image;
determining a hologram of the image from the hologram data.
6. The method of claim 5, wherein the determining the number of tiers of the image comprises:
determining the image depth of each object in the image to obtain at least two depth values;
determining a maximum depth value and a minimum depth value among the at least two depth values;
and determining the layering number of the image according to the maximum depth value and the minimum depth value.
7. The method of claim 5, wherein the plane corresponding to each cross-sectional view data includes at least one object point;
the performing holographic calculation on the cross-sectional diagram data by using the thread in the thread block to obtain the planar diffraction light wave corresponding to the cross-sectional diagram data includes:
acquiring object point data corresponding to each object point in a plane corresponding to the cross-sectional drawing data;
performing Fourier transform calculation on object point data of an object point by adopting one thread in the thread block to obtain a distribution result of object point diffraction light waves of the corresponding object point;
and accumulating and summing the distribution results of the object point diffracted light waves of all or part of object points in the plane corresponding to the cross-sectional diagram data to obtain the diffracted light waves of the plane corresponding to the cross-sectional diagram data.
8. The method of claim 7, further comprising:
acquiring the minimum resolution of the image and the size of the pixel size of a display unit for playing the hologram;
determining the pixel interval size between object points in a plane corresponding to each section data in the image according to the product of the minimum resolution and the pixel size;
and dividing the data of the cross-sectional diagram according to the pixel interval size to obtain the at least one object point.
9. The method according to any one of claims 1 to 8, further comprising:
acquiring the refreshing frequency of the spatial light modulator;
and sequentially determining the time stamp corresponding to each image according to the refreshing frequency.
10. A hologram image generating apparatus, comprising:
the acquisition module is used for acquiring at least two images and a time stamp corresponding to each image;
a determining module for determining at least two image processing units for processing the at least two images;
the first input module is used for sequentially inputting each image into one image processing unit according to the sequence of the timestamps, so that the images are subjected to holographic processing through the corresponding image processing units to obtain the holograms of the images;
and the second input module is used for sequentially inputting the hologram of each image into the spatial light modulator to obtain the holographic image.
11. A hologram creation apparatus, comprising:
a memory for storing executable instructions; a processor for implementing the method of any one of claims 1 to 9 when executing executable instructions stored in the memory.
12. A computer-readable storage medium having stored thereon executable instructions for causing a processor, when executing, to implement the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010712801.6A CN113973198B (en) | 2020-07-22 | 2020-07-22 | Holographic image generation method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010712801.6A CN113973198B (en) | 2020-07-22 | 2020-07-22 | Holographic image generation method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113973198A true CN113973198A (en) | 2022-01-25 |
| CN113973198B CN113973198B (en) | 2024-04-09 |
Family
ID=79585118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010712801.6A Active CN113973198B (en) | 2020-07-22 | 2020-07-22 | Holographic image generation method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113973198B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101088053A (en) * | 2004-12-23 | 2007-12-12 | 视瑞尔技术公司 | Method for computing computer-generated video holograms |
| US20090128619A1 (en) * | 2005-06-14 | 2009-05-21 | Light Blue Optics Ltd. | Signal processing system for synthesizing holograms |
| JP2010079514A (en) * | 2008-09-25 | 2010-04-08 | Sony Corp | Image output device and three-dimensional image display system |
| US20100118117A1 (en) * | 2006-10-26 | 2010-05-13 | Seereal Technologies S.A. | 3d content generation system |
| CN103336420A (en) * | 2013-05-31 | 2013-10-02 | 青岛海信电器股份有限公司 | Holographic image display method and holographic image display device |
| JP2014209247A (en) * | 2006-10-26 | 2014-11-06 | シーリアル テクノロジーズソシエテ アノニムＳｅｅｒｅａｌ Ｔｅｃｈｎｏｌｏｇｉｅｓ Ｓ．Ａ． | 3d content generation system, method thereof and communication method |
| EP3206087A1 (en) * | 2016-02-12 | 2017-08-16 | Samsung Electronics Co., Ltd | Method and apparatus for processing holographic image |
| WO2018190163A1 (en) * | 2017-04-12 | 2018-10-18 | 国立研究開発法人情報通信研究機構 | Hologram generation device, hologram generation program, and holographic projection system |
-
2020
- 2020-07-22 CN CN202010712801.6A patent/CN113973198B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101088053A (en) * | 2004-12-23 | 2007-12-12 | 视瑞尔技术公司 | Method for computing computer-generated video holograms |
| US20090128619A1 (en) * | 2005-06-14 | 2009-05-21 | Light Blue Optics Ltd. | Signal processing system for synthesizing holograms |
| US20100118117A1 (en) * | 2006-10-26 | 2010-05-13 | Seereal Technologies S.A. | 3d content generation system |
| JP2014209247A (en) * | 2006-10-26 | 2014-11-06 | シーリアル テクノロジーズソシエテ アノニムＳｅｅｒｅａｌ Ｔｅｃｈｎｏｌｏｇｉｅｓ Ｓ．Ａ． | 3d content generation system, method thereof and communication method |
| JP2010079514A (en) * | 2008-09-25 | 2010-04-08 | Sony Corp | Image output device and three-dimensional image display system |
| CN103336420A (en) * | 2013-05-31 | 2013-10-02 | 青岛海信电器股份有限公司 | Holographic image display method and holographic image display device |
| EP3206087A1 (en) * | 2016-02-12 | 2017-08-16 | Samsung Electronics Co., Ltd | Method and apparatus for processing holographic image |
| WO2018190163A1 (en) * | 2017-04-12 | 2018-10-18 | 国立研究開発法人情報通信研究機構 | Hologram generation device, hologram generation program, and holographic projection system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113973198B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sahin et al. | Computer-generated holograms for 3D imaging: a survey | |
| RU2434258C2 (en) | Method of calculating digital video holograms and device for realising said method | |
| Su et al. | Fast computer-generated hologram generation method for three-dimensional point cloud model | |
| Chen et al. | Rapid hologram generation utilizing layer-based approach and graphic rendering for realistic three-dimensional image reconstruction by angular tiling | |
| Zhao et al. | Time-division multiplexing holographic display using angular-spectrum layer-oriented method | |
| Bayraktar et al. | Method to calculate the far field of three-dimensional objects for computer-generated holography | |
| Yi et al. | Fast 3D computational integral imaging using graphics processing unit | |
| CN108107700B (en) | Hologram generating method and hologram generating apparatus | |
| Xia et al. | Performance comparison of bilinear interpolation, bicubic interpolation, and B-spline interpolation in parallel phase-shifting digital holography | |
| Zhang et al. | Three-dimensional display technologies in wave and ray optics: a review | |
| Zhao et al. | Quality enhancement and GPU acceleration for a full-color holographic system using a relocated point cloud gridding method | |
| US9727023B2 (en) | Apparatus and method for generating hologram pattern | |
| Lee et al. | High-performance computer-generated hologram by optimized implementation of parallel GPGPUs | |
| Chen et al. | Implementation of shading effect for reconstruction of smooth layer-based 3D holographic images | |
| Pi et al. | Accelerating calculation method for curved computer-generated hologram using look-up table in holographic display | |
| Chang et al. | Fast calculation of computer generated hologram based on single Fourier transform for holographic three-dimensional display | |
| US20240027961A1 (en) | Method and apparatus for generating computer-generated hologram | |
| CN113973198B (en) | Holographic image generation method, device, equipment and computer readable storage medium | |
| CN111812956B (en) | Volume data-based computer generated hologram method, device and electronic equipment | |
| Min et al. | Wigner inverse transform based computer generated hologram for large object at far field from its perspective light field | |
| Liu et al. | Occlusion calculation algorithm for computer generated hologram based on ray tracing | |
| US20210003967A1 (en) | Method and apparatus for processing holographic image | |
| JP2004517354A (en) | Improved method of generating computer generated hologram | |
| Liu et al. | Overview of fast algorithm in 3D dynamic holographic display | |
| CN111443583A (en) | Rapid hologram calculation method based on hologram optimization segmentation calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |