CN113973198B

CN113973198B - Holographic image generation method, device, equipment and computer readable storage medium

Info

Publication number: CN113973198B
Application number: CN202010712801.6A
Authority: CN
Inventors: 时家幸
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Priority date: 2020-07-22
Filing date: 2020-07-22
Publication date: 2024-04-09
Anticipated expiration: 2040-07-22
Also published as: CN113973198A

Abstract

Embodiments of the present application provide a method, an apparatus, a device, and a computer readable storage medium for generating a hologram, where the method includes: acquiring at least two images and a time stamp corresponding to each image; determining at least two image processing units for processing the at least two images; inputting each image into an image processing unit in turn according to the sequence of the time stamps so as to carry out holographic processing on the images through the corresponding image processing units and obtain holograms of the images; and inputting the hologram of each image into a spatial light modulator in turn to obtain the hologram. According to the method and the device, the efficiency of holographic processing can be improved, and the acceleration calculation of the dynamic holographic image is effectively realized.

Description

Holographic image generation method, device, equipment and computer readable storage medium

Technical Field

Embodiments of the present application relate to the field of holographic display, and relate to, but are not limited to, a method, apparatus, device, and computer-readable storage medium for generating a holographic image.

Background

The holographic three-dimensional display technology is that three-dimensional information can be observed through naked eyes. However, due to the increase in the size and resolution of the three-dimensional model, the amount of data to be calculated in the calculation process of the hologram increases exponentially, making the hologram calculation time a bottleneck for development. Moreover, since the hologram calculation time is long, the hologram calculation time is mainly static display at present, and no dynamic display scheme is adopted.

In the related art, a single hologram is obtained by performing acceleration calculation by using a single graphics processor (GPU, graphics Processing Unit), or a single hologram is obtained by performing acceleration calculation by using a plurality of GPUs, that is, in the related art, the acceleration calculation of a static single hologram is performed, but there is no method for effectively performing acceleration calculation on a dynamic hologram.

Disclosure of Invention

The embodiment of the application provides a holographic image generation method, device, equipment and computer readable storage medium, which adopt a plurality of image processing units to sequentially carry out holographic processing on a plurality of images to generate dynamic holographic images, so that the holographic processing efficiency can be improved, and the acceleration calculation of the dynamic holographic images is effectively realized.

The technical scheme of the embodiment of the application is realized as follows:

in a first aspect, an embodiment of the present application provides a method for generating a hologram, including: acquiring at least two images and a time stamp corresponding to each image; determining at least two image processing units for processing the at least two images; inputting each image into an image processing unit in turn according to the sequence of the time stamps so as to carry out holographic processing on the images through the corresponding image processing units and obtain holograms of the images; and inputting the hologram of each image into a spatial light modulator in turn to obtain the hologram.

In a second aspect, an embodiment of the present application provides a hologram generating apparatus, including: the acquisition module is used for acquiring at least two images and the corresponding time stamp of each image; a determining module for determining at least two image processing units for processing the at least two images; the first input module is used for inputting each image into one image processing unit in sequence according to the sequence of the time stamps so as to carry out holographic processing on the images through the corresponding image processing units to obtain holograms of the images; and the second input module is used for inputting the hologram of each image into the spatial light modulator in sequence to obtain the hologram.

In a third aspect, embodiments of the present application provide a hologram image generating apparatus, including: a memory for storing executable instructions; and the processor is used for realizing the method when executing the executable instructions stored in the memory.

In a fourth aspect, embodiments of the present application provide a computer readable storage medium storing executable instructions for causing a processor to perform the above-described method.

In the embodiment of the application, at least two images and a time stamp corresponding to each image are acquired; and determining at least two image processing units for processing the at least two images; and then inputting each image into an image processing unit in turn according to the sequence of the time stamps, so as to carry out holographic processing on the images through the corresponding image processing units, obtain holograms of the images, and further obtain holograms. Therefore, as the plurality of image processing units are adopted to sequentially carry out holographic processing on the plurality of images, dynamic holographic images are generated, the holographic processing efficiency can be improved, and the acceleration calculation on the dynamic holographic images is effectively realized.

Drawings

FIG. 1A is a schematic flow chart of an alternative method for generating a holographic image according to an embodiment of the present application;

fig. 1B is a schematic view of an application scenario of a hologram generating method according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present application;

FIG. 4 is a schematic flow chart of an alternative chromatography method in a method for generating a holographic image according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of an alternative chromatography method in a hologram generating method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of the distribution of line blocks in a GPU provided by an embodiment of the present application;

FIG. 7 is a schematic flow chart of an alternative chromatography method in a hologram generating method according to an embodiment of the present application;

FIG. 8 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present application;

FIG. 9 is a schematic flow chart of generating dynamic holograms using multiple GPU parallel computing based on chromatography provided in an embodiment of the present application;

FIG. 10 is a schematic diagram of a flow computation process of a GPU according to an embodiment of the present application;

FIG. 11 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present application;

FIG. 12 is a schematic diagram of a chromatography provided in an embodiment of the present application;

FIG. 13 is a graph comparing calculated speed improvement results provided in embodiments of the present application;

FIG. 14 is a schematic view of the constitution of a hologram generating apparatus according to an embodiment of the present application;

fig. 15 is a schematic diagram of the composition structure of a hologram generating apparatus provided in an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the present application will be described in further detail with reference to the accompanying drawings, and the described embodiments should not be construed as limiting the present application, and all other embodiments obtained by those skilled in the art without making any inventive effort are within the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict. Unless defined otherwise, all technical and scientific terms used in the embodiments of the present application have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of the present application belong. The terminology used in the embodiments of the present application is for the purpose of describing the embodiments of the present application only and is not intended to be limiting of the present application.

Before further describing embodiments of the present application in detail, the terms and expressions that are referred to in the embodiments of the present application are described, and are suitable for the following explanation.

1) Holographic imaging technology: the hologram technology is a technology of recording and reproducing a real three-dimensional image of an object using interference and diffraction principles. The first step is to record the object light wave information by utilizing the interference principle, namely the shooting process: the shot object forms a diffuse object beam under laser irradiation; the other part of laser is used as reference beam to be emitted onto the holographic negative film, and is overlapped with the object beam to generate interference, so that the phase and amplitude of each point on the object light wave are converted into the intensity which is changed in space, and the whole information of the object light wave is recorded by utilizing the contrast and interval between interference fringes.

2) Chromatography: layering the 3D object along the depth direction, adding the complex amplitude distribution of the two-dimensional sectional images of each layer on the holographic surface respectively, taking the complex amplitude distribution as the complex amplitude distribution of the original three-dimensional object, adding reference light, and calculating by a Fresnel approximation method to prepare a hologram of the whole object field.

In order to better understand the image processing method provided in the embodiment of the present application, first, an image processing method and problems existing in the related art will be described:

And in the holographic three-dimensional display technology, three-dimensional information can be observed through naked eyes. However, due to the increase in the size and resolution of the three-dimensional model, the amount of data to be calculated in the calculation process of the hologram increases exponentially, making the hologram calculation time a bottleneck for development. And because the hologram calculation time is long, the hologram calculation time is mainly static display at present, and a dynamic display scheme is not adopted.

At present, two main approaches are used for improving the calculation speed, one is software, and the purpose of reducing the calculation amount is achieved by improving an algorithm and reducing the complexity of the algorithm. Another method is to change the hardware device, and utilize the progress of the hardware device of other high-performance architectures to accelerate the performance, and employ the hardware device such as GPU to accelerate the speed, and mainly includes three methods: the point cloud method and the triangular patch method are mainly used for the point cloud method at present.

However, whether an improvement is made in terms of software or hardware devices, there is at least one of the following problems: firstly, the related art mainly uses a single GPU to accelerate to calculate a single hologram, or uses multiple GPUs to accelerate to calculate a single hologram, that is, the single hologram is calculated by a static single hologram, but the acceleration calculation of a dynamic hologram is not performed; secondly, the related technologies are all based on the point cloud method, so that the calculated data amount is changed for different three-dimensional objects, and the calculation time of each hologram is unpredictable; third, current research is mainly directed to the generation of single static holograms, and no further research is done on dynamic holograms.

Based on at least one of the above problems in the related art, the embodiments of the present application provide a method for generating a dynamic hologram using multiple GPUs acceleration, which can generate a dynamic hologram with a fixed refresh frequency based on multiple GPUs to calculate dynamic three-dimensional model data. Acquiring at least two images and a time stamp corresponding to each image; and determining at least two image processing units (e.g., GPUs) for processing the at least two images; and then inputting each image into an image processing unit in turn according to the sequence of the time stamps, so as to carry out holographic processing on the images through the corresponding image processing units, obtain holograms of the images, and further obtain holograms. Therefore, as the plurality of image processing units are adopted to sequentially carry out holographic processing on the plurality of images, dynamic holographic images are generated, the holographic processing efficiency can be improved, and the acceleration calculation on the dynamic holographic images is effectively realized.

An exemplary application of the hologram generating apparatus provided in the embodiment of the present application is described below, and the hologram generating apparatus provided in the embodiment of the present application may be implemented as a notebook computer, a tablet computer, a desktop computer, a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device), and other various types of terminals, and may also be implemented as a server. Next, an exemplary application when the hologram generating apparatus is implemented as a server will be described.

Referring to fig. 1A, fig. 1A is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present application, and will be described with reference to the steps shown in fig. 1A.

Step S101, at least two images and a time stamp corresponding to each image are acquired.

Here, the at least two images are images for generating a final hologram, which may be acquired by an image acquisition device on the terminal, or may be downloaded on an arbitrary website.

In this embodiment of the present application, the server acquires the image and also acquires a timestamp corresponding to each image, where each image of at least two images corresponds to a timestamp, and the timestamp of each image is different from the timestamps of other images. The time stamp may be a time when the server acquires the image, or may be a time stamp added to each image by the server based on a certain time stamp rule after the server acquires the image, or may be a time stamp of each image correspondingly formed according to a time sequence between the at least two images. The time stamps are used for determining the sequence of each image relative to other images in the process of carrying out subsequent image processing.

Step S102, determining at least two image processing units for processing the at least two images.

Here, the image processing unit is used to perform holographic processing on the image, and in some embodiments, the image processing unit may be a GPU, and then, in the method of the embodiment of the present application, holographic processing is performed on at least two images by using multiple GPUs.

In this embodiment of the present application, the number of image processing units may be smaller than or equal to the number of images. And the server determines an image processing unit capable of completing the holographic processing of the image according to the current hardware condition.

In some embodiments, after determining at least two image processing units, the at least two image processing units may be further numbered, so in a subsequent image holographic processing process, each image processing unit may be sequentially called according to the sequence of the image processing unit numbers to implement holographic processing of the image.

Step S103, inputting each image into an image processing unit in turn according to the sequence of the time stamps, so as to carry out holographic processing on the images through the corresponding image processing units, and obtaining holograms of the images.

Here, one image is input to one image processing unit at a time in the order of the time stamps, and the image is holographically processed by the image processing unit. When the number of the image processing units is equal to the number of the images, each image processing unit carries out holographic processing on one image; when the number of the image processing units is smaller than that of the images, firstly traversing all the image processing units in sequence according to the numbers of the image processing units to enable all the image processing units to realize the holographic processing process of part of the images, then starting from the initial numbers, continuously traversing all the image processing units in a circulating way, distributing the images to each image processing unit to complete the holographic processing until the holographic processing of all the images is completed. According to the scheme for carrying out holographic processing on the images through the plurality of image processing units, the plurality of images can be processed simultaneously through the plurality of image processing units in parallel, so that the efficiency of image processing is greatly improved.

In this embodiment, after an image processing unit performs holographic processing on an image, a hologram is formed, and the hologram is a three-dimensional image, which is very different from a conventional photo, where the conventional photo shows a real physical image, and the hologram contains information such as size, shape, brightness, contrast, etc. of an object to be recorded, and the information is stored in a very small but very complex interference pattern, which is generated by a laser.

The hologram processing process of the embodiment of the application adopts a hologram technology, which is a technology capable of realizing the recording and reproduction of a real three-dimensional image. Unlike other three-dimensional "images," holograms provide "parallax". The presence of parallax allows the observer to observe the different aspects of the image by moving back and forth, left and right, and up and down as if there were a real object there.

Step S104, inputting the hologram of each image into a spatial light modulator in turn to obtain the hologram.

Here, the spatial light modulator (SLM, spatial Light Modulator) refers to a device that can modulate a certain parameter of an optical field by liquid crystal molecules under active control, for example, modulate the amplitude of the optical field, modulate the phase by refractive index, modulate the polarization state by rotation of a polarization plane, or realize conversion of incoherent-coherent light, thereby writing certain information into an optical wave and achieving the purpose of optical wave modulation.

In the embodiment of the application, holograms obtained after holographic processing of each image can be sequentially input into the spatial light modulator according to the sequence of the time stamps of the images, so that the obtained holograms are also images formed according to the sequence of the time stamps of the images, and the dynamic display of the holograms of the three-dimensional images is realized.

Fig. 1B is a schematic view of an application scenario of a hologram generating method according to an embodiment of the present application, as shown in fig. 1B, a hologram generating system 10 for implementing the hologram generating method includes a terminal 100, a network 200, and a server 300, where a user obtains at least two images for generating a hologram and a timestamp corresponding to each image through the terminal 100, and sends the at least two images and the timestamp of each image to the server 300 through the network 200, so that the server 300 determines at least two image processing units for processing the at least two images; inputting each image into an image processing unit in turn according to the sequence of the time stamps so as to carry out holographic processing on the images through the corresponding image processing units and obtain holograms of the images; and inputting the hologram of each image into a spatial light modulator in turn to obtain the hologram. After obtaining the hologram, the server 300 transmits the hologram to the terminal 100 through the network 200, and the terminal 100 may play the hologram on the current interface 100-1.

According to the holographic image generation method, at least two images and the corresponding time stamp of each image are obtained; and determining at least two image processing units for processing the at least two images; and then inputting each image into an image processing unit in turn according to the sequence of the time stamps, so as to carry out holographic processing on the images through the corresponding image processing units, obtain holograms of the images, and further obtain holograms. Therefore, as the plurality of image processing units are adopted to sequentially carry out holographic processing on the plurality of images, dynamic holographic images are generated, the holographic processing efficiency can be improved, and the acceleration calculation on the dynamic holographic images is effectively realized.

In some embodiments, the method for generating a hologram may be applied to a hologram generating system, where the hologram generating system includes a terminal (i.e., an image capturing device), a server, an image processing unit (in this embodiment, the image processing unit is taken as a GPU to describe the image processing unit), and a spatial light modulator, the terminal captures images to obtain at least two images, and sends the captured at least two images to the server, and the server controls the at least two GPUs to generate a hologram for the at least two images.

FIG. 2 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present application, as shown in FIG. 2, the method includes the following steps:

step S201, the terminal sequentially collects at least two images according to time sequence.

Here, the terminal may sequentially collect at least two images through its own image collection unit (e.g., a camera) according to a time sequence, or the terminal sequentially downloads at least two images from any website according to a time sequence, or the terminal sequentially receives at least two images sent by other terminals according to a time sequence. Of course, in other embodiments, the terminal may also collect the image in any other manner, which is not limited in this embodiment.

Step S202, the terminal sends at least two images and the corresponding time stamp of each image to the server.

Here, the time stamp may correspond to the time when the terminal acquires the images, or the time stamp may be added to each image according to the acquired time sequence after the terminal acquires the images, or the time stamp may be set for each image by the terminal according to the sequence of the images played by the hologram to be formed by the user.

In step S203, the server determines at least two GPUs for processing the at least two images.

In step S204, the server sequentially inputs each image into a GPU according to the sequence of the time stamps.

For example, the determined number of GPUs is 3, namely GPU1, GPU2 and GPU3, the number of images to be processed is 5, namely image 1, image 2, image 3, image 4 and image 5, and the images 1 to 5 are numbered sequentially according to the sequence of the time stamps, then the image 1 may be input into the GPU1, the image 2 may be input into the GPU2, the image 3 may be input into the GPU3, the images 1 to 3 are first subjected to holographic processing by the GPUs 1 to 3, the image 4 is input into the GPU1 after the processing of the GPUs 1 to 3 is completed, and the image 5 is input into the GPU2, so that the images are input into the GPU in a circulating manner until all the images are input.

In step S205, the GPU performs holographic processing on the image to obtain a hologram of the image.

In the embodiment of the application, the GPU adopts a holographic technology to carry out holographic calculation on the image so as to complete holographic processing on the image and obtain a hologram of the image.

In step S206, the server sequentially controls the GPU to input the holograms of each image into the spatial light modulator.

Here, after the GPU completes the hologram processing on the images to obtain the holograms, since the processing time of the GPU on each image is the same, the holograms may be sequentially formed according to the sequence of the time stamps, and after the holograms are formed, the holograms obtained by the corresponding processing may be sequentially input into the spatial light modulator according to the sequence of the time stamps of the images.

In step S207, the spatial light modulator spatially modulates the hologram of each of the at least two images to obtain holograms corresponding to the at least two images.

Here, the spatial light modulator spatially modulates a light beam of the hologram after the hologram is acquired, so that information of the image can be loaded into a one-dimensional or two-dimensional light field.

In step S208, the spatial light modulator transmits the hologram to the terminal.

Step S209, the terminal dynamically displays the hologram on the current interface.

In some embodiments, after receiving the hologram, the terminal may display the hologram on its current interface, or may project the hologram onto a specific display screen or display curtain, so as to play the hologram.

According to the hologram generation method, the terminal, the server, the GPU and the spatial light modulator cooperate to complete the generation of the hologram and the playing process after the generation, and in the process of generating the hologram, the plurality of GPUs are used for processing the plurality of images at the same time, so that the plurality of GPUs are used for carrying out hologram processing on the plurality of images at the same time to generate the dynamic hologram, the efficiency of the hologram processing can be improved, and the acceleration calculation of the dynamic hologram is effectively realized.

Fig. 3 is a schematic flowchart of an alternative method for generating a hologram according to an embodiment of the present application, as shown in fig. 3, in some embodiments, before step S103, the method further includes the following steps:

step S301, sequentially extracting image data of each image according to the sequence of the time stamps.

Here, the image data is a pixel value of each pixel point in the image or a pixel value of each object point in the image.

Correspondingly, step S103 may be implemented by:

step S302, sequentially inputting the image data of each image into one of the image processing units according to the sequence of the time stamps, so as to perform holographic calculation on the image data of the image through the corresponding image processing unit, thereby obtaining a hologram of the image.

Here, a hologram of an image is obtained by performing hologram calculation on image data using a hologram technique. When inputting image data of an image into an image processing unit, the image data may be randomly input into an image processing unit currently in an idle state, that is, the image data is input into an image processing unit which is not currently subjected to hologram processing.

With continued reference to fig. 3, in other embodiments, step S103 may be implemented by:

step S303, ordering at least two image processing units to form a sequence of image processing units.

Here, the image processing units may be randomly ordered to form a sequence of image processing units.

Step S304, sequentially inputting each image into one image processing unit in the image processing unit sequence according to the sequence of the time stamps and the sequence of the image processing units in the image processing unit sequence, so as to perform holographic processing on the image through the corresponding image processing unit, thereby obtaining a hologram of the image.

Here, the at least two images form an image sequence according to the sequence of the time stamps, and then one image and one image processing unit may be sequentially selected according to the image sequence and the image processing unit sequence, and the images may be input into the image processing unit.

With continued reference to fig. 3, in still other embodiments, step S103 may be implemented by:

in step S305, an image corresponding to the current time stamp is determined as a target image.

Step S306, determining any one of the at least two image processing units currently in an idle state as a target image processing unit.

Step S307, inputting the target image into the target image processing unit, so as to perform holographic processing on the target image by the target image processing unit, and obtain a hologram of the target image.

Step S308, it is determined whether the current timestamp is the last timestamp.

If yes, executing step S104; if the judgment result is no, when the time reaches the next time stamp, returning to the loop execution step S305 until the hologram processing is completed for the images corresponding to all the time stamps.

In some embodiments, in step S103, the step of performing holographic processing on the image by using a corresponding image processing unit to obtain a hologram of the image may be implemented by using a tomography method, where the tomography method is to layer the 3D object along the depth direction, add complex amplitude distributions of two-dimensional cross-sectional views of each layer imaged on the holographic surface respectively, and use the complex amplitude distributions as complex amplitude distributions of the original three-dimensional object, add reference light, and calculate by using fresnel approximation to obtain a hologram of the whole object field.

Fig. 4 is a schematic flow chart of an alternative chromatography method in the hologram generating method according to the embodiment of the present application, as shown in fig. 4, the chromatography step in step S103 may be implemented by the following steps:

step S401, determining the layering number of the image.

And step S402, layering the image according to the layering quantity to obtain sectional view data of the layering quantity.

Here, the plane to which the sectional view data corresponds may be a two-dimensional sectional view. In the implementation process, the image is an image in a three-dimensional space, and the image can be cut along a plane perpendicular to a Z axis in the three-dimensional space during layering to obtain section image data on different two-position section images under different Z axis coordinates, wherein the section image data reflects parallel distribution of the image on the Z axis.

Step S403, assigning a thread block in the corresponding image processing unit to each section view data.

Here, the thread blocks are calculation units in the image processing units, each of which can simultaneously perform calculation of a plurality of image data, and thus, the thread blocks in the image processing units can be employed to perform synchronous calculation of a plurality of processes.

And step S404, carrying out holographic calculation on the section image data by adopting threads in the thread blocks to obtain the diffraction light waves of the plane corresponding to the section image data.

Here, one thread block is used for processing one section image data, and one thread block includes a plurality of threads, so that the plurality of threads in one thread block can be adopted to simultaneously perform holographic calculation on the same section image data, so as to obtain the diffraction light wave of the plane corresponding to the section image data.

Step S405, obtaining object light waves corresponding to the planes from the diffracted light waves of the planes corresponding to each piece of the sectional view data.

And step S406, overlapping the object light waves with the layering quantity to obtain hologram data of the image.

Step S407, determining a hologram of the image according to the hologram data.

After carrying out holographic calculation on each section image data, obtaining the diffraction light waves of the planes with the same number as the layering number correspondingly, and then obtaining object light waves from the diffraction light waves of each plane, wherein the object light waves are diffraction light waves corresponding to objects in an image, and then superposing all object light waves. Since the number of the cross-sectional image data is the layering number, the number of planes is also the layering number, so that the determined number of the object light waves is the layering number, the object light waves with the layering number can be overlapped to obtain hologram data of the image, and finally, the hologram of the image is determined according to the hologram data. The cross-sectional image data is data of a cross-sectional image corresponding to the same image and in different Z-axis coordinates, and the object light waves are diffraction light waves corresponding to an object in a plane corresponding to the same image and in different Z-axis coordinates, and therefore, these object light waves are superimposed to obtain hologram data of a hologram of the image.

The number of layers in the chromatography determines the longitudinal resolution of the final hologram, and the larger the number of layers, the higher the resolution. Meanwhile, the increase of the layering number also brings about an increase in the calculation amount, so that the layering number needs to be reduced as much as possible while the resolution requirement is met.

Fig. 5 is a schematic flow chart of an alternative chromatography method in the hologram generating method according to the embodiment of the present application, as shown in fig. 5, step S401 may be implemented by the following steps:

step S501, determining an image depth of each object in the image, and obtaining at least two depth values.

Here, the image depth of the object corresponds to the above-described Z-axis coordinates, and objects at different depth values, which reflect the distance between the actual object and the photographing unit (or photographer), correspond to different Z-axis coordinates.

Step S502, determining a maximum depth value and a minimum depth value from the at least two depth values.

Here, the object corresponding to the maximum depth value is farthest from the photographer, and the object corresponding to the minimum depth value is closest to the photographer.

Step S503, determining the layering number of the image according to the maximum depth value and the minimum depth value.

Here, the integration may be performed between the maximum depth value and the minimum depth value, or the number of layers may be determined by calculating a distance difference between the maximum depth value and the minimum depth value and determining the number of layers according to the distance difference.

With continued reference to fig. 5, in some embodiments, each cross-sectional view includes at least one object point, and correspondingly, step S404 may be implemented by:

step S504, obtaining object point data corresponding to each object point in the plane corresponding to the section view data.

In step S505, fourier transform calculation is performed on object point data of an object point by using one thread in the thread block, so as to obtain a distribution result of object point diffraction light waves of the corresponding object point.

As shown in fig. 6, a schematic distribution diagram of Thread blocks in a GPU according to an embodiment of the present application is provided, on a GPU 60, a computing unit on the GPU is divided into a plurality (e.g., 2-3) of grids (Grid) 61, each Grid 61 includes a plurality (e.g., 65535) of Thread blocks (blocks) 62, and each Thread Block 62 includes a plurality (e.g., 512) of threads (threads) 63. A parallel program of a unified computing device architecture (CUDA, compute Unified Device Architecture) is executed by a plurality of threads 63; several threads 63 are grouped into a thread block 62, and the threads 63 in the same thread block 62 may be synchronized or may communicate via shared memory (shared memory); the plurality of thread blocks 62 reconstruct the grid 61.

Step S506, performing accumulation and summation on the distribution results of the object point diffracted light waves of all or part of object points in the plane corresponding to the section image data, to obtain the diffracted light waves of the plane corresponding to the section image data.

In the embodiment of the application, the same GPU as in fig. 6 is used to process the same image; each image is divided into multiple cross-sectional views, each thread block 62 in the GPU processes the same cross-sectional view; each cross-sectional view further includes a plurality of object points, and each thread 63 in the thread block 62 correspondingly processes object point data corresponding to one object point.

For example, there are 3 images for generating the hologram, namely, image 1, image 2 and image 3, and the number of GPUs is also 3, namely, GPU1, GPU2 and GPU3, wherein GPU1 performs holographic processing on image 1, GPU2 performs holographic processing on image 2, and GPU3 performs holographic processing on image 3. Here, taking the procedure of performing the hologram processing on the image 1 by the GPU1 as an example, the procedure of performing the hologram processing on the image 2 by the GPU2 and the procedure of performing the hologram processing on the image 3 by the GPU3 may refer to the following procedure of performing the hologram processing on the image 1 by the GPU 1.

The process of carrying out holographic processing on the image 1 by the GPU1 is as follows: since the number of layers of the image 1 is determined to be 2, the image 2 is divided into the cross-section 11 and the cross-section 12, and correspondingly, the GPU1 has a plurality of thread blocks, and the thread blocks 11 and 12 in the plurality of thread blocks can be used for respectively carrying out holographic calculation on the cross-section data of the cross-section 11 and the cross-section data of the cross-section 12. The plane of the cross-sectional view 11 includes three object points, namely an object point 111, an object point 112 and an object point 113, and the plane of the cross-sectional view 12 also includes three object points, namely an object point 121, an object point 122 and an object point 123, and the thread block 11 includes a plurality of threads. In the embodiment of the present application, the thread 11a, the thread 11b and the thread 11c in the thread block 11 are used to perform fourier transform computation on object point data of the object point 111, the object point 112 and the object point 113 respectively, so as to correspondingly obtain a distribution result of the object point diffraction light wave of the object point 111, a distribution result of the object point diffraction light wave of the object point 112 and a distribution result of the object point diffraction light wave of the object point 113, and then, the distribution result of the object point diffraction light wave of the object point 111, the distribution result of the object point diffraction light wave of the object point 112 and the distribution result of the object point diffraction light wave of the object point 113 are summed up, so as to obtain the diffraction light wave of the plane where the cross-section 11 is located. By adopting the same method, carrying out Fourier transform calculation on object point data of object point 121, object point 122 and object point 123 by adopting thread 12a, thread 12b and thread 12c in thread block 12 respectively, correspondingly obtaining a distribution result of object point diffraction light waves of object point 121, a distribution result of object point diffraction light waves of object point 122 and a distribution result of object point diffraction light waves of object point 123, and then carrying out accumulation summation on the distribution result of object point diffraction light waves of object point 121, the distribution result of object point diffraction light waves of object point 122 and the distribution result of object point diffraction light waves of object point 123, thereby obtaining the diffraction light waves of the plane of the section view 12.

In some embodiments, a method for dividing a cross-sectional view to obtain at least one object point is further provided, based on fig. 5, fig. 7 is a schematic flow chart of an alternative chromatography in the hologram generating method according to the embodiments of the present application, as shown in fig. 7, before the step S504 of obtaining object point data corresponding to each object point, the method further includes the following steps of dividing to obtain at least one object point:

step S701, obtaining the minimum resolution of the image and the pixel size of the display unit for playing the hologram.

Step S702, determining a pixel interval size between object points in a plane corresponding to each section image data in the image according to a product of the minimum resolution and the pixel size.

Step S703, dividing the cross-sectional image data according to the pixel interval size to obtain the at least one object point.

Here, after determining the pixel interval size, the cross-sectional image data is divided by the pixel interval size to obtain at least one object point. The space dimension reflects the data space size between each object point and the adjacent object point.

In some embodiments, to ensure that the frequency of the input of the hologram into the spatial light modulator is synchronized with the refresh frequency of the spatial light modulator, the time interval of at least two images used to generate the hologram may also be predetermined, i.e., the time stamp of each image is predetermined such that the time interval between the images corresponding to each adjacent two time stamps is equal and the frequency corresponding to the time interval between the images corresponding to each adjacent two time stamps is equal to the refresh frequency of the spatial light modulator. In order to achieve frequency synchronization, on the basis of any one of the above embodiments, the hologram generating method may further include the steps of: step S11, the refreshing frequency of the spatial light modulator is obtained. And step S12, determining the time stamp corresponding to each image in sequence according to the refreshing frequency.

In the embodiment of the application, since the time stamp of the image is determined according to the refresh frequency of the spatial light modulator, the input frequency and the processing frequency of the image are consistent with the refresh frequency of the spatial light modulator, namely, the synchronization of the frequency of inputting the hologram into the spatial light modulator and the refresh frequency of the spatial light modulator can be realized, and the spatial light modulator can perform the same-frequency processing on the hologram to obtain a stable holographic image.

In the following, an exemplary application of the embodiments of the present application in a practical application scenario will be described.

The embodiment of the application provides a holographic image generation method, which is characterized in that a three-dimensional object is segmented in a chromatography mode to obtain optical data information of a target object, then a multi-GPU computing algorithm is improved and optimized, and a new GPU computing flow is provided for dynamic display of the holographic image.

An embodiment of the present application provides a calculation process and a calculation flow optimization scheme for generating a dynamic hologram by using multiple GPUs in parallel based on a chromatography, and fig. 8 is an optional flow diagram of a hologram generating method according to an embodiment of the present application, as shown in fig. 8, where the implementation process includes the following steps:

step S801, a plurality of GPUs are numbered according to the input sequence of three-dimensional images (corresponding to the images), then dynamic three-dimensional model data (corresponding to the image data) is extracted according to the time sequence, and the image data are sequentially distributed to each GPU video memory in sequence;

Here, the time interval of the three-dimensional image data is the refresh frequency of the spatial light modulator.

Step S802, initializing each GPU, starting each GPU calculation algorithm program in sequence according to the refreshed time interval, inputting three-dimensional model data at different moments into a GPU memory in sequence to wait for extraction and calculation, and projecting calculated calculation hologram results into a spatial light modulator.

Here, since the principle of the tomographic calculation and the size of the image calculation data amount are fixed factors, the calculation time for averaging each hologram is also the same, so that a relatively stable refresh frequency can be obtained for normal display viewing.

In some embodiments, the fresnel diffraction holographic algorithm may also be optimized such that it can be up to 60 times higher on average in the computation speed of a single GPU.

In step S803, holograms are sequentially input under the spatial light modulator, so that holograms of the three-dimensional model can be dynamically displayed.

FIG. 9 is a schematic flow chart of generating a dynamic hologram image by using multiple GPUs in parallel based on a chromatography method according to an embodiment of the present application, as shown in FIG. 9, three-dimensional images (i.e. three-dimensional model data) at times t1, t2, t3 and t4 are sequentially and correspondingly input into GPU G1, GPU G2, GPU G3 and GPU G4 to correspondingly obtain hologram 1, hologram 2, hologram 3 and hologram 4; then, holograms 1, 2, 3 and 4 are sequentially input to the spatial light modulator 91, and a hologram 92 is obtained and output.

In some embodiments, step S802 may be implemented by:

step S8021, the two-dimensional size of the three-dimensional model is read, the optimal layering quantity is obtained, and the total object point data quantity is determined.

Here, the size of the interval between the two-dimensional plane object points may be set as a product between the minimum resolution of the three-dimensional model and the size of the pixel size of the display screen, and the optimal layering number of the three-dimensional model is calculated using the following formula (1-1):

wherein N represents the optimal number of layers; z represents the distance between the object and the photographing terminal (i.e., the image depth of the object); z _max Representing a maximum depth value; z _min Representing a minimum depth value; Δz _e Representing the increment of the change in the three-dimensional model of the distance between the object and the photographing terminal.

In step S8022, the thread blocks and the thread numbers are allocated, and the optimal distribution and data transmission of the thread blocks, threads and the computation holograms are determined.

Here, since the GPU is tested by experiments when performing hologram calculation, the maximum acceleration bottleneck is the influence of the data transmission bandwidth, so the embodiment of the application uses the processing procedure of stream calculation of the GPU, so that the bandwidth transmission is started in the calculation process, and the delay is reduced to the greatest extent. The optimal state of the GPU is that the data calculation time and the data transmission time just overlap.

As shown in fig. 10, which is a schematic diagram of a flow calculation processing procedure of the GPU according to the embodiment of the present application, a central processing unit (CPU, central processing unit) 110 divides three-dimensional image data into a plurality of cross-sectional views according to image depth, obtains information such as arbitrary depth picture information 01 and arbitrary depth picture information 02, and sequentially inputs the arbitrary depth picture information 01 and the arbitrary depth picture information 02 into a thread block of the GPU 111, and performs data receiving 1101 and Kernel execution 1102 on the arbitrary depth picture information 01 and the arbitrary depth picture information 02 through the thread block, so as to obtain complex amplitude 01 and complex amplitude 02. Finally, the hologram 1104 is obtained by performing addition summation 1103 on the complex amplitude 01, the complex amplitude 02, and the like, and output.

In some embodiments, the following rules may be followed when partitioning the GPU threads and thread block distributions: the minimum thread blocks are multiples of the thread bundles 32, and the maximum thread blocks are the maximum number of threads of the GPU.

And S8023, substituting the information of each two-dimensional plane object point obtained in the step S8021 into a chromatography formula, and superposing the corresponding data calculated under different layers to obtain the calculated hologram data at the moment.

Here, one thread bundle (or thread) corresponds to one object point, and the calculation of different layer planes can be performed randomly; when the number of plane object points is larger than the number of thread grids, traversing sampling in a thread circulation mode; the results of the computation in each thread should be kept in the shared memory of the corresponding sequence number.

In the embodiment of the application, the following formula (1-2) can be adopted to realize the chromatographic method to generate the calculation hologram:

wherein U (x, y) represents the holographic calculation result; j is a complex number; k is a constant; z _n Representing the distance of the object light from the reference surface; λ represents the wavelength of the light wave; u (U) ₀ Representing the reference light wave amplitude; (x) ₀ ,y ₀ ) Representing the coordinates of the diffraction surface, namely the reference coordinates of the object light wave; (x, y) represents the viewing surface coordinates.

Alternatively, the computed hologram may be generated by a chromatography implemented by the following formula (1-3):

wherein U (ζ, η) represents the superimposed holographic calculation result; u (U) _i (ζ, η) represents the i-th hologram calculation result to be superimposed; a (ζ, η) represents the magnitude of the superimposed complex amplitude; phi (xi, eta) represents the holographic phase of the superimposed light.

In the embodiment of the application, the calculation hologram is generated by the formula (1-2) or the formula (1-3), so that the calculation accuracy can be simplified, and the calculation amount can be reduced. The calculation can be performed by adopting a simple formula in the GPU, for example, the tan function is replaced by the tan function, so that the calculation amount can be greatly reduced.

In some embodiments, the holographic calculation algorithm of the Fresnel diffraction is optimized, and each parameter variable in the Fresnel diffraction algorithm is extracted and combined in the operation process, so that the calculated amount of repeated operation is further greatly reduced. Wherein, the formula (1-3) can be extracted and combined, so that the formula (1-3) can be regarded as the following formula (1-4):

wherein, representing the reduction algebra of the PC.

Fig. 11 is a schematic flow chart of an alternative method for generating a hologram according to an embodiment of the present application, as shown in fig. 11, including the following steps: in step S1101, object depth and image information of the three-dimensional object are extracted. Step S1102, selecting an arbitrary depth, and calculating a hologram cross-section of a cross-section of the depth. Step S1103, the superimposed depth is translated, and the hologram cross-section is calculated. Step S1104, generating a hologram according to the calculated hologram cross-section.

It should be noted that, the principle of the chromatography is that a three-dimensional object may be considered as a series of multi-layer parallel cross sections perpendicular to the Z axis, as shown in fig. 12, which is a schematic diagram of the chromatography provided in the embodiment of the present application, each planar graph (such as planar graphs 121, 122 and 123 in fig. 12) is a two-dimensional array, and the distribution of the diffracted light waves of each planar graph may be conveniently calculated by using fast two-dimensional fourier transform, then the diffracted light waves of each planar graph are calculated, finally the object light waves of all the planar graphs are superimposed and the phase information thereof is extracted, and finally the phase information superimposed on the plane is the hologram data result 124 to be obtained.

According to the holographic image generation method provided by the embodiment of the application, the dynamic three-dimensional virtual object uses the GPU to carry out dynamic holographic display, so that the dynamic three-dimensional virtual object can be refreshed dynamically. The processing time of the hologram of the three-dimensional object is controlled to be within the same range by using the chromatography, and the refresh frequency of the hologram generation is stabilized. Meanwhile, the transmitted video images are respectively and circularly put into the GPU according to a certain time sequence, so that the generated holograms can be sequentially output.

The holographic image generation method provided by the embodiment of the application has at least the following technical advantages: first, compared with other methods such as a point cloud method and a triangular patch method, the calculated data volume and the calculated time of the other methods are changed, and the chromatographic method adopted in the embodiment of the application can enable the total calculated data amount of each layer of three-dimensional model slice to be the same, so that the generated calculated hologram time is the same, and the method is convenient to use in dynamic display.

Secondly, the calculation algorithm of the calculation hologram under the GPU is optimized, so that the calculation speed of the calculation hologram is greatly improved, wherein fig. 13 is a comparison graph of calculation speed improvement results provided by the embodiment of the application, as shown in fig. 13, the abscissa is the layering number (layer number), and the ordinate is the calculation time, wherein three curves from bottom to top in fig. 13 are respectively a change curve of the calculation time calculated by using the GPU 1050Ti+CUDA with the layer number, a change curve of the calculation time calculated by using the CPU I5+mathematical core function library (MKL, math Kernel Library) function with the layer number, and a change curve of the calculation time calculated by using the CPU I5+matrix laboratory (MATLAB, matrix Laboratory) function with the layer number. Obviously, the method of the embodiment of the application (that is, the method of GPU 1050ti+cuda) has the shortest calculation time, that is, the highest calculation rate, and the increase of the calculation time with the increase of the layer number is not obvious.

Thirdly, compared with other methods in which data transmission and data calculation are performed separately when in use, the method of the present invention uses a stream calculation processing mode in the GPU, so that calculation and data transmission are performed simultaneously, and GPU cores are allocated, so as to achieve dynamic equality of calculation time and transmission time.

Fourth, since the processing time of the three-dimensional model at each moment is uncertain in other methods, dynamic holographic display cannot be performed, and the method of the embodiment of the present application can achieve dynamic holographic display.

Based on the foregoing embodiments, embodiments of the present application provide a hologram generating apparatus, including each module included, and each component included in each module, which may be implemented by a processor in a hologram generating device; of course, the method can also be realized by a logic circuit; in practice, the processor may be a central processing unit (CPU, central Processing Unit), a microprocessor (MPU, micro Processor Unit), a digital signal processor (DSP, digital Signal Processing), or a field programmable gate array (FPGA, field Programmable Gate Array), or the like.

Fig. 14 is a schematic diagram of the composition structure of a hologram generating apparatus according to an embodiment of the present application, and as shown in fig. 14, the hologram generating apparatus 1400 includes:

An acquiring module 1401, configured to acquire at least two images and a timestamp corresponding to each image; a determining module 1402 for determining at least two image processing units for processing the at least two images; a first input module 1403, configured to sequentially input each of the images into an image processing unit according to the sequence of the time stamps, so as to perform holographic processing on the image by using the corresponding image processing unit, thereby obtaining a hologram of the image; a second input module 1404, configured to sequentially input the holograms of each image into a spatial light modulator, so as to obtain the holograms.

In some embodiments, the apparatus further comprises: the extraction module is used for sequentially extracting the image data of each image according to the sequence of the time stamps; correspondingly, the first input module is further configured to sequentially input the image data of each image into one of the image processing units according to the sequence of the time stamps, so as to perform holographic calculation on the image data of the image through the corresponding image processing unit, and obtain a hologram of the image.

In some embodiments, the first input module is further to: sequencing the at least two image processing units to form an image processing unit sequence; and inputting each image into one image processing unit in the image processing unit sequence in turn according to the sequence of the time stamps and the sequence of the image processing units in the image processing unit sequence so as to carry out holographic processing on the image through the corresponding image processing unit to obtain a hologram of the image.

In some embodiments, the first input module is further to: determining an image corresponding to the current time stamp as a target image; determining any image processing unit currently in an idle state as a target image processing unit in the at least two image processing units; inputting the target image into the target image processing unit, and carrying out holographic processing on the target image by the target image processing unit to obtain a hologram of the target image.

In some embodiments, the apparatus further comprises a holographic processing module for determining a number of layers of the image; layering the images according to the layering quantity to obtain sectional view data of the layering quantity; distributing thread blocks in the corresponding image processing units to each section view data; carrying out holographic calculation on the section image data by adopting threads in the thread blocks to obtain plane diffraction light waves corresponding to the section image data; obtaining object light waves of the corresponding planes from the diffraction light waves of the planes corresponding to the sectional view data; superposing the layering number of object light waves to obtain hologram data of the image; and determining a hologram of the image according to the hologram data.

In some embodiments, the holographic processing module is further to: determining the image depth of each object in the image to obtain at least two depth values; determining a maximum depth value and a minimum depth value from the at least two depth values; and determining the layering quantity of the image according to the maximum depth value and the minimum depth value.

In some embodiments, each section data includes at least one object point in a plane corresponding to the section data; the holographic processing module is further configured to: acquiring object point data corresponding to each object point in a plane corresponding to the section image data; carrying out Fourier transform calculation on object point data of one object point by adopting one thread in the thread block to obtain a distribution result of object point diffraction light waves of the corresponding object point; and accumulating and summing the distribution results of the object point diffraction light waves of all or part of object points in the plane corresponding to the section graph data to obtain the diffraction light waves of the plane corresponding to the section graph data.

In some embodiments, the apparatus further comprises: the second acquisition module is used for acquiring the minimum resolution of the image and the pixel size of a display unit used for playing the holographic image; a second determining module, configured to determine a pixel interval size between object points in a plane corresponding to each section image data in the image according to a product of the minimum resolution and the pixel size; and the dividing module is used for dividing the section image data according to the pixel interval size to obtain the at least one object point.

In some embodiments, the apparatus further comprises: a third acquisition module, configured to acquire a refresh frequency of the spatial light modulator; and the third determining module is used for sequentially determining the time stamp corresponding to each image according to the refreshing frequency.

It should be noted that, the description of the apparatus in the embodiment of the present application is similar to the description of the embodiment of the method described above, and has similar beneficial effects as the embodiment of the method, so that a detailed description is omitted. For technical details not disclosed in the embodiments of the present apparatus, please refer to the description of the embodiments of the method of the present application for understanding.

In the embodiment of the present application, if the image processing method is implemented in the form of a software functional module and sold or used as a separate product, the image processing method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or part contributing to the related art, and the computer software product may be stored in a storage medium, including several instructions for causing a terminal to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the application provides a hologram generating device, fig. 15 is a schematic diagram of a composition structure of the hologram generating device provided in the embodiment of the application, and as shown in fig. 15, the hologram generating device 1500 at least includes: a processor 1501, a communication interface 1502, and a computer readable storage medium 1503 configured to store executable instructions, wherein the processor 1501 generally controls the overall operation of the holographic image generating apparatus. The communication interface 1502 may enable the hologram generating apparatus to communicate with other terminals or servers via a network. The computer-readable storage medium 1503 is configured to store instructions and applications executable by the processor 1501, and may also cache data to be processed or processed by each module in the processor 1501 and the hologram generating apparatus 1500, and may be implemented by FLASH memory (FLASH) or random access memory (RAM, random Access Memory).

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application. The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.

The foregoing is merely an embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of generating a hologram, comprising:

acquiring at least two images and a time stamp corresponding to each image;

determining at least two image processing units for processing the at least two images;

inputting each image into an image processing unit in turn according to the sequence of the time stamps so as to carry out holographic processing on the images through the corresponding image processing units and obtain holograms of the images; the hologram processing is performed on the image by a corresponding image processing unit to obtain a hologram of the image, including: determining a layering number of the image; layering the images according to the layering quantity to obtain sectional view data of the layering quantity; assigning a thread block in the corresponding image processing unit to each of the cross-sectional view data; carrying out holographic calculation on the section image data by adopting threads in the thread blocks to obtain plane diffraction light waves corresponding to the section image data; obtaining object light waves of the corresponding planes from the diffraction light waves of the planes corresponding to the sectional view data; superposing the layering number of object light waves to obtain hologram data of the image; determining a hologram of the image from the hologram data;

And inputting the hologram of each image into a spatial light modulator in turn to obtain the hologram.

2. The method according to claim 1, wherein the method further comprises: sequentially extracting the image data of each image according to the sequence of the time stamps;

correspondingly, inputting the image data of each image into one image processing unit in turn according to the sequence of the time stamps, so as to perform holographic calculation on the image data of the images through the corresponding image processing units, and obtaining holograms of the images.

3. The method according to claim 1, wherein inputting each of the images into an image processing unit in turn according to the sequence of the time stamps, so as to perform holographic processing on the images by the corresponding image processing unit, to obtain a hologram of the image, includes:

sequencing the at least two image processing units to form an image processing unit sequence;

and inputting each image into one image processing unit in the image processing unit sequence in turn according to the sequence of the time stamps and the sequence of the image processing units in the image processing unit sequence so as to carry out holographic processing on the image through the corresponding image processing unit to obtain a hologram of the image.

4. The method according to claim 1, wherein inputting each of the images into an image processing unit in turn according to the sequence of the time stamps, so as to perform holographic processing on the images by the corresponding image processing unit, to obtain a hologram of the image, includes:

determining an image corresponding to the current time stamp as a target image;

determining any image processing unit currently in an idle state as a target image processing unit in the at least two image processing units;

inputting the target image into the target image processing unit, and carrying out holographic processing on the target image by the target image processing unit to obtain a hologram of the target image.

5. The method of claim 1, wherein the determining the number of layers of the image comprises:

determining the image depth of each object in the image to obtain at least two depth values;

determining a maximum depth value and a minimum depth value from the at least two depth values; and determining the layering quantity of the image according to the maximum depth value and the minimum depth value.

6. The method of claim 1, wherein each cross-sectional view data corresponds to a plane that includes at least one object point;

Carrying out holographic calculation on the section image data by adopting threads in the thread blocks to obtain diffraction light waves of a plane corresponding to the section image data, wherein the method comprises the following steps:

acquiring object point data corresponding to each object point in a plane corresponding to the section image data;

carrying out Fourier transform calculation on object point data of one object point by adopting one thread in the thread block to obtain a distribution result of object point diffraction light waves of the corresponding object point;

and accumulating and summing the distribution results of the object point diffraction light waves of all or part of object points in the plane corresponding to the section graph data to obtain the diffraction light waves of the plane corresponding to the section graph data.

7. The method of claim 6, wherein the method further comprises:

acquiring the minimum resolution of the image and the pixel size of a display unit for playing the holographic image;

determining the pixel interval size between object points in a plane corresponding to each section image data in the image according to the product of the minimum resolution and the pixel size;

and dividing the sectional image data according to the pixel interval size to obtain the at least one object point.

8. The method according to any one of claims 1 to 7, further comprising:

acquiring the refresh frequency of the spatial light modulator;

and determining the time stamp corresponding to each image in turn according to the refreshing frequency.

9. A hologram generating apparatus, comprising:

the acquisition module is used for acquiring at least two images and the corresponding time stamp of each image;

a determining module for determining at least two image processing units for processing the at least two images;

the first input module is used for inputting each image into one image processing unit in sequence according to the sequence of the time stamps so as to carry out holographic processing on the images through the corresponding image processing units to obtain holograms of the images;

the second input module is used for inputting the holograms of each image into the spatial light modulator in sequence to obtain the holograms;

a holographic processing module for determining a layering number of the image; layering the images according to the layering quantity to obtain sectional view data of the layering quantity; distributing thread blocks in the corresponding image processing units to each section view data; carrying out holographic calculation on the section image data by adopting threads in the thread blocks to obtain plane diffraction light waves corresponding to the section image data; obtaining object light waves of the corresponding planes from the diffraction light waves of the planes corresponding to the sectional view data; superposing the layering number of object light waves to obtain hologram data of the image; and determining a hologram of the image according to the hologram data.

10. A hologram generating apparatus, comprising:

a memory for storing executable instructions; a processor for implementing the method of any one of claims 1 to 8 when executing executable instructions stored in said memory.

11. A computer readable storage medium storing executable instructions for causing a processor to perform the method of any one of claims 1 to 8.