KR102277101B1 - A ultra-high-resolution hologram generation method based on artificial intelligence and deep learning to learn fringe patterns by area - Google Patents

Abstract

The present invention relates to an ultra-high-resolution hologram generation method based on artificial intelligence (AI) and deep learning to learn fringe patterns by area, wherein a deep neural network is divided into areas of hologram and learned and fringe patterns obtained by area are suitably assembled together and accumulated to generate a hologram. The method of the present invention comprises the following steps of: (a) generating a first fringe pattern by a computer generated hologram (CGH) algorithm for points of a three-dimensional object and a second fringe pattern by a fringe pattern generator and optimizing parameters with a possibility value of similarity of the first and second fringe patterns to train the fringe pattern generator, wherein conditions for training are set to a position of an object point and corresponding hologram areas; and (b) generating third fringe patterns by area through the trained fringe pattern generator, multiplying the third fringe pattern generated for each area by a value of brightness, accumulating a value resulting from the multiplication, and assembling the third fringe pattern by area to generate a digital hologram. According to the present invention, a deep neural network is divided into areas of a hologram and learned and fringe patterns obtained by area are suitably assembled together and accumulated, thereby enabling an ultra-high-resolution hologram in a big size to be readily processed by dividedly generating fringe patterns.

Description

{ A ultra-high-resolution hologram generation method based on artificial intelligence and deep learning to learn fringe patterns by area }

The present invention uses a deep neural network that uses a generative network based on deep learning technology to generate a digital hologram that is not limited in size, but includes an object point and a region coordinate (or segmented region) of a high-resolution hologram corresponding thereto. It relates to a method for generating a super-resolution hologram based on artificial intelligence and deep learning that learns a fringe pattern by area, which trains a deep neural network through branches.

In addition, the present invention obtains a high-resolution fringe pattern for one object point by connecting some regions (or divided regions) of a high-resolution fringe pattern obtained through a deep neural network to each region coordinates appropriately, and a high-resolution fringe pattern made for each object point It relates to an artificial intelligence and deep learning-based ultra-high-resolution hologram generation method that learns fringe patterns for each area by accumulating and accumulating a single high-resolution hologram.

In general, a hologram is composed of fringe patterns generated as a result of interfering a light wave (reference wave) as a reference with a light wave (object wave) reflected from an object. A digital hologram (DH) is obtained by acquiring a pattern as digital data. Therefore, although a hologram is 2D data, it is 3D data including distance information to an object due to a phase difference between an object wave and a reference wave [Non-Patent Document 1]. When the reference wave is irradiated to this hologram, the original image of the object at the original position is reproduced. Therefore, although the holographic image has a viewing angle limitation, since it displays a three-dimensional image, it is considered as the ultimate 3D image [Non-Patent Document 2].

From the perspective of broad classification, a method for generating a traditional digital hologram can be divided into a method using optical imaging and a computer generated hologram (CGH) method using a computer. 1 shows a method using optical imaging. Countless studies on optical imaging and CGH methods have been conducted so far. Each of these methods will have different advantages and disadvantages.

In the case of the CGH method, the biggest disadvantage is that the operation processing speed is very slow. In order to improve this speed, methods such as improving an algorithm, using a GPU, making a field programmable gate array (FPGA), or making a dedicated chipset have been suggested.

In addition, as shown in FIG. 2 , a method for generating a hologram using a deep learning technology that has been rapidly developed in recent years has been proposed [Patent Document 1].

Korean Patent Publication No. 10-2018-0064884 (published on June 15, 2018)

S. A. Benton and V. M. Bove Jr., Holographic imaging, John Wiley and Sons Inc., 2008. Ki-Pyung Han, "Giga Korea," TTL Journal, Vol. 146, pp. 16-21, March 2013 (March 2013).

An object of the present invention is to solve the above problems, and to generate a digital hologram that is not limited in size, a deep neural network based on deep learning technology and using a generative network is used, but the object point and the It is to provide a method for generating a super-high-resolution hologram based on artificial intelligence and deep learning that learns fringe patterns for each region, which trains a deep neural network through two division regions of the corresponding high-resolution hologram.

In addition, an object of the present invention is to obtain a high-resolution fringe pattern for one object point by connecting some regions of a high-resolution fringe pattern obtained through a deep neural network to each region coordinate appropriately, and to obtain a high-resolution fringe pattern made for each object point and accumulate Accordingly, it is to provide a method for generating a high-resolution hologram based on artificial intelligence and deep learning that learns the fringe pattern generated by a single high-resolution hologram by area.

In order to achieve the above object, the present invention relates to an artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning a fringe pattern for each area, (a) a fringe that receives object points and area information and outputs a fringe pattern for each area optimizing the parameters of the pattern generator, but optimizing by determining the fringe pattern for each area for the object point generated by the computer generated hologram (CGH) algorithm as correct answer data; and, (b) generating a fringe pattern for each region corresponding to all regions using the optimized fringe pattern generator, connecting the fringe patterns for each region of all regions to generate a high-resolution fringe pattern, and from the generated high-resolution fringe pattern and generating a high-resolution hologram.

In addition, the present invention provides an artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning a fringe pattern for each area, in the step (a), for the sampled points and area information, the CGH algorithm is applied to each point. The degree of similarity of the generated fringe patterns for each first and second areas by generating a fringe pattern for each first area for each region, and applying each point and each area information to the fringe pattern generator to generate a fringe pattern for each second area. It is characterized in that the parameters are optimized so that n is maximized.

In addition, the present invention provides an artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning a fringe pattern for each region, wherein the region information is information indicating each region when the size of the entire hologram is divided into a grid of the same size. characterized by being.

In addition, the present invention is a method for generating a super-high-resolution hologram based on artificial intelligence and deep learning that learns a fringe pattern for each region, characterized in that the region information is displayed as region coordinates.

In addition, the present invention relates to an artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning a fringe pattern for each area. In the step (a), a fringe indicating the degree of similarity of the fringe pattern for each first and second area as a probability value. It is characterized in that the pattern discriminator is configured, the parameters of the fringe pattern discriminator are optimized, and the parameters of the fringe pattern generator are optimized so that the output value of the optimized fringe pattern discriminator is maximized.

In addition, the present invention provides an artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning a fringe pattern for each region, wherein the fringe pattern generator is composed of a real part generator and an imaginary part generator, and each generator is a real fringe for each region. It is characterized by generating an imaginary fringe pattern for each pattern and region.

In addition, the present invention is a method for generating a super-resolution hologram based on artificial intelligence and deep learning for learning a fringe pattern for each area, the fringe pattern generator is characterized in that it is composed of a deep neural network.

In addition, the present invention is a method for generating a super-high-resolution hologram based on artificial intelligence and deep learning for learning a fringe pattern by region, the fringe pattern discriminator is characterized in that it is composed of a deep neural network.

In addition, the present invention provides an artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning a fringe pattern for each area, in the step (b), for each of the points of a given three-dimensional object, each point and each point By applying all area information to the fringe pattern generator, all area-specific fringe patterns corresponding to all area information of each point are generated, and high-resolution fringe patterns are generated by connecting the area-specific fringe patterns of all generated areas, and the generated A digital hologram is generated by multiplying the high-resolution fringe pattern by the brightness value of each point, and accumulating the random phase Prince pattern of all points multiplied by the brightness value.

In addition, the present invention relates to a computer-readable recording medium recording a program for performing an artificial intelligence and deep learning-based ultra-high-resolution hologram generating method for learning a fringe pattern for each area.

As described above, according to the artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning the fringe pattern for each area according to the present invention, the deep neural network is trained by dividing the object point and the hologram area into a single object point. By properly connecting and accumulating the fringe patterns obtained for each area, it is possible to easily process the fringe patterns by dividing and generating the fringe patterns, even for a hologram with a large size.

In addition, according to the artificial intelligence and deep learning-based ultra-high-resolution hologram generation method for learning the fringe pattern for each area according to the present invention, by generating the fringe pattern using a neural network, the memory capacity is saved and the calculation is performed at a very high speed. possible effects are obtained.

That is, compared to the lookup table method according to the related art, the memory capacity used can be very small. Specifically, in the case of the lookup table method, a memory capacity of several giga to several tera is required, whereas the method of the neural network method according to the present invention uses only tens of megabytes to a maximum of several hundred megabytes.

In addition, the method according to the present invention has a very high operation speed compared to the prior art. That is, in the case of the software method, compared to the method using the formula, the method using the formula requires repeated calculations as much as the size of the hologram to generate the fringe pattern for one light source, whereas the method of the neural network method according to the present invention is one Since a single fringe pattern is output with one repetition, theoretically, the speed increases in proportion to the size of the hologram.

1 is an exemplary view showing a method of generating and restoring a digital hologram by an optical acquisition method according to the prior art.
2 is an exemplary view showing a digital hologram generation method using artificial intelligence according to the prior art.
3 is a diagram showing the configuration of an overall system for implementing the present invention.
4 is a diagram illustrating a process of generating a fringe pattern based on object points according to the method for generating a hologram of the present invention.
5 is a view showing a process of generating a fringe pattern (diffraction pattern for a single object pointer) using a deep neural network (deep learning model) according to the hologram generation method of the present invention.
6 is a flowchart illustrating a method for generating a super-high-resolution hologram based on artificial intelligence and deep learning for learning a fringe pattern for each area according to an embodiment of the present invention.
7 is a diagram illustrating a process of training a deep neural network (deep learning model) to generate a diffraction pattern (a part of an ultra-high-resolution diffraction pattern for a single object point) according to an embodiment of the present invention.
8 is a view showing a region of a fringe pattern according to an embodiment of the present invention.
9 is a flowchart illustrating a step of generating an ultra-high-resolution hologram using a deep neural network for generating a fringe pattern according to an embodiment of the present invention.
10 is a view for explaining a hologram generation process according to an embodiment of the present invention.
11 is an exemplary view of a fringe pattern generated using a deep neural network according to an experiment of the present invention, (a) a real part, (b) an imaginary part.
12 is an exemplary diagram of a hologram generated using a deep neural network according to an experiment of the present invention, (a) a real part, (b) an imaginary part.
13 is an exemplary view of a hologram generation result using a deep neural network according to an experiment of the present invention, (a) a point cloud for an object, and (b) an exemplary view showing a hologram restoration result using the hologram of Fig. 8;

Hereinafter, specific contents for carrying out the present invention will be described with reference to the drawings.

In addition, in demonstrating this invention, the same part is attached|subjected with the same code|symbol, and the repetition description is abbreviate|omitted.

First, examples of the configuration of the entire system for implementing the present invention will be described with reference to FIG. 3 .

As shown in FIG. 3, the artificial intelligence and deep learning-based super-high-resolution hologram generation method for learning the fringe pattern for each area according to the present invention is a program on the computer terminal 20 that receives three-dimensional data 10 and generates a hologram. can be implemented in the system. That is, the digital hologram generating method may be configured as a program and installed in the computer terminal 20 to be executed. A program installed in the computer terminal 20 may operate as one program system 30 .

On the other hand, as another embodiment, artificial intelligence and deep learning-based ultra-high-resolution hologram generation method that learns fringe patterns for each area is composed of a program and operated on a general-purpose computer, and is composed of one electronic circuit such as an ASIC can be carried out. Alternatively, it may be developed as a dedicated computer terminal that exclusively processes only the generation of a holographic image. This will be referred to as a hologram generating device 30 . Other possible forms may also be implemented.

On the other hand, 3D data is composed of a point cloud representing a 3D object.

Next, a basic principle and process of a method for generating a digital hologram using a deep learning technique according to the present invention will be described with reference to FIGS. 4 to 5 .

A computer-generated hologram (CGH) is obtained by accumulating the fringe patterns for all light sources input from the coordinates of all hologram planes and then accumulating the patterns.

Equation 1 shows the CGH formula of the real part for generating a complete complex hologram.

[Equation 1]

Here, I(u,v) is the brightness at the coordinates (u,v) of the hologram plane, and A _j (x,y,z) is the brightness of the light source at the x,y,z coordinates. N is the number of light sources, λ is the wavelength of a reference wave used to generate the hologram, and p is the size of the pixels of the hologram and the light source. Here, they are treated as the same value for convenience.

The process of Equation 1 is separated as shown in Equations 2 and 3.

[Equation 2]

[Equation 3]

Accordingly, as shown in FIG. 4 , a fringe pattern is generated for each object point using Equation 2 .

Meanwhile, the fringe pattern is made in the form of a complex number (I _Re + iI _Im ), and in the complex number, the real part (I _Re ) and the imaginary part (I _Im ) are calculated separately. In Equation 2, the real part is calculated as a cosine, but the imaginary part is calculated as a sine function.

At this time, as shown in FIG. 5 , the method according to the present invention is generated using a deep neural network for Equation (2).

The generated fringe pattern is multiplied by a brightness value as shown in Equation 3, and cumulative addition is performed to generate a final hologram.

Next, a method for generating a super-high-resolution hologram based on artificial intelligence and deep learning for learning a fringe pattern for each region according to an embodiment of the present invention will be described with reference to FIGS. 6 to 10 .

As shown in FIG. 6 , the method for generating a digital hologram according to the present invention includes the steps of learning a fringe pattern generator (S10), and generating a digital hologram from object data using the learned fringe pattern generator (S20) is composed of At this time, in the learning stage, the 3D object points for learning are learned, and a digital hologram is generated using the actual target 3D object points.

First, the step of learning the fringe pattern generator (S10) will be described.

7 shows the structure of a model for learning a deep neural network to generate a fringe pattern. The overall structure is based on the adversarial generative model.

As shown in FIG. 7 , when region information (or region coordinates) of an object point and a high-resolution fringe pattern is input, a real fringe pattern is generated using a computer generated hologram (CGH) algorithm. In this case, a fringe pattern (hereinafter referred to as a fringe pattern for each first area) corresponding to the given area information (or area coordinates) from among the fringe patterns is extracted and generated. That is, the real fringe generated by the CGH algorithm is used as the correct answer data. In addition, the learning method described herein applies one learning method (eg, GAN learning method) for an example, and can be applied to all supervised learning methods of neural networks using correct answer data.

At the same time, for object points and region coordinates, a fake fringe pattern (hereinafter referred to as a fringe pattern for each second region) is generated using a deep neural network-based fringe pattern generator. However, here, in Real Fringe, Real means real, and in Real Data, Real means real information.

The Fringe Pattern Discriminator based on Deep Neural Network compares these two results and outputs how similar the real and the fake are as a probability value, and based on this probabilistic calculation result, the fringe pattern generator is more We get a loss value so that we can create a fake that is similar to the real thing.

Based on this probabilistic calculation result, the internal parameters of the fringe pattern generator are updated so that the fringe pattern generator can make a fake more similar to the real thing. If this process is repeatedly performed, the real and the fake become indistinguishably similar. This enables the fringe pattern generator to generate the same fringe pattern as the fringe pattern generated by CGH, and the deep neural network outputs the fringe pattern. It means you have been taught to do it.

Hereinafter, a specific procedure for training using the fringe pattern generation training model will be described in detail. Each term and method used at this time will be described.

c ₁ ,...,c _m : (x,y,z,h _U ,h _V ) spatial coordinates and hologram area coordinates of an object point in integer or real form, which are condition information. That is, x, y, z represent the spatial coordinates of the object, and h _U , h _V are the hologram area coordinates. Preferably, it is a coordinate indicating each region when the hologram is divided into a grid of the same size.

z ⁽¹⁾ ,...,z ^(m) : One-dimensional noise or noise image.

f _re ⁽¹⁾ ,...,f _re ^(m) : A Real Real Fringe Pattern _{of (f h} × f _w ) size generated using a computer-generated hologram algorithm.

f _im ⁽¹⁾ ,...,f _im ^(m) : A Real Imaginary Fringe Pattern _{of (f h} × f _w ) size generated using a computer-generated hologram algorithm.

G(z _i | c _i ): Given the coordinate condition c _i of the object point, generate a fringe pattern using the noise z _{i as an input.} This is called a fringe pattern generator, and to generate a real hologram and an imaginary hologram, G _Re (z _i |c _i ) and G _Im (z _i |c _i ) It consists of two deep neural networks.

D(x) : represents the probability that _{x came from the noise p g (z).} That is, it probabilistically indicates how similar the generated fringe pattern is to the actual fringe pattern. It is called a fringe pattern discriminator, and two deep neural networks, _{D Re} (x) and D _Im (x), are used to discriminate real holograms and imaginary holograms. network) is composed of

: 프린지 패턴 생성기 G_Im(z_i|c_i)를 통해 만들어진 허수부 프린지 패턴이다.

: An imaginary fringe pattern created by the fringe pattern generator G _Im (z _i |c _{i ).}

: 프린지 패턴 생성기 G_Re(z_i|c_i)를 통해 만들어진 실수부 프린지 패턴이다.

: A real-part fringe pattern created by the fringe pattern generator G _Re (z _i |c _{i ).}

▽ _x : It is a Del operator for the differentiation of deep neural networks using stochastic gradient ascending with respect to x.

ε: means a random number between 0 and 1.

θ _G : A parameter of the fringe pattern generator composed of a deep neural network. These parameters include the weight, bias, and optimizer parameters of the neural network. The parameters of the optimizer are determined according to the type of optimizer. For example, when using Adam, α, β ₁ , and β ₂ are included.

θ _D : A parameter of the fringe pattern discriminator composed of a deep neural network. These parameters include the weight, bias, and optimizer parameters of the neural network. The parameters of the optimizer are determined according to the type of optimizer. For example, when using Adam, α, β ₁ , and β ₂ are included.

L: It is a loss function used for deep neural network training in deep learning. Currently, this document uses WGAN-GP (Wasserstein Generative Adversarial Network - Gradient Panelty) as an example, but other loss functions may be used.

m: number of mini-batches

k : number of training (training)

First, in the learning step (S10) of the fringe pattern generator, the points of the three-dimensional object are sampled with a mini-batch size m, and a fringe pattern generator composed of a neural network using the sampled data, and a fringe pattern discriminator are trained (or learn)

That is, the _{noise z 1} ,z ₂ ...,z _{m is sampled from the predetermined noise p g} (z), and the real fringe pattern f ¹ _Re ,f ² _Re ,... ,f ^m _Re and an imaginary fringe pattern f ¹ _Im ,f ² _Im ,...,f ^m _Im for each first area are generated. _{Also, the condition c 1} ,c ₂ ...,c _m is sampled from the object points and the area parameter.

In particular, as described above, the condition c _i is the (x,y,z,h _U ,h _V ) spatial coordinates and hologram area coordinates of an integer or real number object point, which is condition information. . In this case, x, y, and z represent spatial coordinates of the object, and h _U and h _V are hologram area information (or area coordinates).

As shown in FIG. 8 , the hologram area coordinates h _u , h _v indicate a divided area in the entire hologram. In this case, preferably, the coordinates indicate each region when the hologram is divided into a grid of the same size.

In the example of FIG. 8 , the size of the entire hologram is F _h × F _w , and it can be divided into grid-shaped hologram regions of the same size f _h × f _{w .} In this case, each hologram area F(h _U ,h _V ) may be represented by area coordinates h _U ,h _{V .}

In addition, the real fringe pattern f ¹ _Re ,f ² _Re ,...,f ^m _{Re for each} first area or the imaginary fringe pattern f ¹ _Im ,f ² _Im ,...,f ^m _Im for each first area is in CGH Among the fringe patterns obtained by , it is a fringe pattern of a divided region of _{(f h} × f _{w ) size corresponding to the coordinates of the region of the condition.} Hereinafter, for convenience of description, a fringe pattern for each region and a fringe pattern are used interchangeably when there is no need for a distinction.

Next, for each sampled data, a second imaginary fringe pattern and a second real fringe pattern are generated by a (imaginary or real) fringe pattern generator. In this case, the following equation is used.

[Equation 4]

Next, a third imaginary fringe pattern is calculated by varying the weights of the first imaginary fringe pattern and the second imaginary fringe pattern with a random value ε, and a loss function L _Imi of the imaginary fringe pattern is calculated.

[Equation 5]

[Equation 6]

Next, a third real fringe pattern is calculated by varying the weights of the first real fringe pattern and the second real fringe pattern with a random value ε, and a loss function L _Rei of the real fringe pattern is calculated.

[Equation 7]

[Equation 8]

In Equations 6 and 8, λ is a penalty coefficient and serves to adjust the size of the loss function. λ is a predetermined value. In addition, D( ) and the like are values obtained by the discriminator, and indicate how similar the values are to the actual values. Also, |||||| ₂ is an L2 norm operation, and it is an operation which squares and adds each element inside, and takes the square root. That is, the normal operation is an operation that measures the length or size of a vector. ▽ _f D(f _i ) is the value obtained by differentiating D(f _i ) with f by the Del operator (Del operator to differentiation of deep neural networks using stochastic gradient ascending technique) ▽.

Next, if the loss function is obtained for all the sampling data of each mini-batch as described above, the parameters of the imaginary fringe pattern discriminator and the real fringe pattern discriminator are optimized as shown in the following equation to learn.

[Equation 9]

θ _D represents the parameters of each discriminator. The parameters of the discriminator are parameters of the neural network, and represent weights, biases, and the like. Also, Optimizer(x,parameters) represents a function that optimizes by changing the parameter parameters so that the value of x is minimized. That is, Equation 9 indicates that _{the average value of the loss function L i} of each sampling data is differentiated by the del operator (▽), and the values of parameters optimizing the differentiated values are obtained and optimization is performed.

In other words, the point with the smallest loss in the loss function is the point where the slope of the loss function is zero. In order to find the point where the slope of the loss function, which is a multidimensional function, is zero, the slope of the loss function is obtained by taking the del operator. The stochastic gradient descent method is to construct a loss function with the difference between the result obtained through forward propagation of the neural network and the correct answer, and then adjust the parameters constituting the neural network so that the gradient of the loss function becomes zero. However, Equation 9 is an optimization process. After obtaining the slope by differentiating the loss function, the problem of how to change the parameters of the neural network so that the slope of the loss function becomes zero using this slope remains. Here, the process of changing the parameters of the neural network using the differential value of the loss function is optimization, and performing this is the optimizer.

Next, the previous learning process (or training process) is repeated as many times as the number of training k. That is, in the previous process, each discriminator was trained once using the sampling data. Each discriminator is learned by repeating this learning process k times.

Next, for training of the fringe pattern generator, the points of the object and the noise are sampled. And, as shown in the following equation, the result (or probability) generated by the generator G is determined by the discriminator D, and the parameters of the generator G are optimized by maximizing the discrimination result. In this case, the previously trained discriminator is used.

[Equation 10]

Here, θ _GIm represents the parameters of the imaginary fringe pattern generator, and θ _GRe represents the parameters of the real fringe pattern generator.

In summary, we use the method of training generator 1 after training discriminator k. Randomly extract noise, actual fringe patterns and conditions as many as m mini-batch sizes, and use them to forward the model. After calculating the loss function using the forward propagation result and finding the derivative of the loss function, the parameters of the discriminator are updated through the backpropagation operation. Next, the model is forward propagated again using the randomly extracted noise and conditions, and the parameters of the generator are updated while backpropagating using the results. Here, forward propagation is the process of making an output by putting an input into the neural network, and it is the same as inference or testing. Backpropagation is a process of differentiating the loss function, which is the difference between the forward propagation result and the correct answer, and changing the parameters of the neural network while moving the differentiation result from the output part to the input part of the neural network using an optimization algorithm.

Next, a step (S20) of generating a digital hologram will be described.

9 shows a procedure for generating a digital hologram using a fringe pattern generator created (learned) through the fringe pattern generation training step S10.

In the process described above, a real fringe pattern and an imaginary fringe pattern are generated. Since the purpose is to make a high-resolution hologram, when receiving the coordinates necessary for generating the _{hologram, patch holograms (f w} , f _h ) from the fringe generator representing the parts of the high-resolution fringe expressed by _{F w} and F _{h are generated.} They are all stitched together to make one super-resolution fringe pattern (F _w , F _{h ).}

The subsequently generated high-resolution fringe pattern is multiplied by the brightness value, and cumulative addition is performed with the previously generated high-resolution fringe pattern. When high-resolution fringe patterns for all object points are accumulated and added, a final high-resolution hologram is created.

A detailed procedure for generating a hologram using the fringe pattern generation model will be described. Each term and method used at this time will be described.

f _h : The height value of the fringe pattern of one piece (or divided area) output from the fringe pattern generator

f _w : the width value of the fringe pattern of one piece (or divided area) output from the fringe pattern generator

f ^~ : fringe pattern by area (or patch of fringe pattern) output from the fringe pattern generator with size _{f h} × f _w

F _h : Height value of high-resolution fringe pattern

F _w : Width value of high-resolution fringe pattern

F : One high-resolution fringe pattern with size _{F h} × F _w

condition(object points, region parameter(h _U ,h _V )): The process of calling 3D coordinates and region coordinates of a high-resolution hologram from object point information

intensity(object points): The process of calling brightness information from object point information

One digital hologram from a fringe pattern generator with size H: F _h × F _w

A _i : Brightness value of object point

Call coordinate information and brightness information to be used as conditions for deep neural network from object point information. A single fringe pattern is generated through a generator using the coordinate information and the zone information of the zone super-resolution fringe pattern, and a brightness weight is given to the generated patterns, and then accumulated to generate a hologram. That is, area information in the high-resolution fringe pattern is given from _{F w} /f _w and F _h /f _h. A high-resolution fringe pattern sheet is completed by arranging the patch fringe pattern from the fringe pattern generator as a condition for the given area information and the object point obtained from the object in each place for the given area in all cases.

_{Specifically, for each object point i, a fringe pattern F i} (w,h) for each hologram area (w,h) is generated for each area.

[Equation 11]

Next, a fringe pattern (or patch of fringe pattern) F _i (w,h) for each area is placed (connected) at each position to complete _{a high-resolution fringe pattern F i for each object point i.}

Next, the completed high-resolution fringe pattern of each object point i is summed to generate a corresponding high-resolution hologram.

[Equation 12]

A process of summing the fringe patterns for each region is illustrated in FIG. 10 .

Next, the effects of the present invention will be described through experiments.

11 shows a complex fringe pattern generated using a fringe pattern generator. The real number in Fig. 11(a) and the imaginary part fringe pattern in Fig. 11(b) are shown for 64 conditions, respectively.

Fig. 11 shows a hologram generated by concatenating and cumulatively adding fringe patterns generated through a fringe pattern generator. Fig. 12(a) is a real part, and Fig. 12(b) is an imaginary part.

The result of restoring the generated hologram is shown in FIG. 13 . 13(a) shows the point cloud information of an object made into an 8-bit image file. FIG. 13( b ) is a result of numerically restoring the hologram of FIG. 12 . As can be seen from the results, it was confirmed that a digital hologram very similar to that of CGH was generated with a difference of 1 to 5 dB.

In the above, the invention made by the present inventors has been described in detail according to the above embodiments, but the present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention.

10: three-dimensional data 20: computer terminal
30: program system

Claims (10)

In the artificial intelligence and deep learning-based super-resolution hologram generation method for learning fringe patterns by area,
(a) Optimize the parameters of the fringe pattern generator that receives the object point and area information and outputs the fringe pattern for each area, but the fringe pattern for each area for the object point generated by the computer generated hologram (CGH) algorithm (hereinafter the first area) star fringe pattern) as the correct answer data and optimizing; and;
(b) generating a fringe pattern for each region corresponding to all regions using the optimized fringe pattern generator, connecting the fringe patterns for each region of all regions to generate a high-resolution fringe pattern, and a high-resolution hologram from the generated high-resolution fringe pattern comprising the steps of creating
The fringe pattern generator is composed of a deep neural network, and the parameters of the fringe pattern generator are parameters of the deep neural network of the fringe pattern generator,
In step (a), with respect to the sampled points and area information, a fringe pattern for each second area generated by applying each point and each area information to the fringe pattern generator, and each point and each area information to obtain the degree of similarity between the fringe patterns for each first area, which is the correct answer data generated by applying to the CGH algorithm, and optimize the parameters by setting the parameters of the fringe pattern generator so that the degree of similarity is maximized,
In step (b), for each of the points of a given 3D object, each point and all area information for each point are applied to the fringe pattern generator to generate all the area-specific fringe patterns corresponding to all area information of each point. Create a high-resolution fringe pattern by connecting the fringe patterns for each area of all the generated areas, multiply the generated high-resolution fringe pattern by the brightness value of each point, and accumulate the high-resolution fringe patterns of all points multiplied by the brightness value Artificial intelligence and deep learning-based ultra-high-resolution hologram generation method, characterized in that it generates a digital hologram.
delete According to claim 1,
The area information is information representing each area when the size of the entire hologram is divided into a grid of the same size.
4. The method of claim 3,
The area information is an artificial intelligence and deep learning-based ultra-high-resolution hologram generating method, characterized in that it is displayed in area coordinates.
According to claim 1,
In the step (a), a fringe pattern discriminator representing the degree of similarity of the fringe patterns for each first and second region as a probability value is configured, the parameters of the fringe pattern discriminator are optimized, and the output value of the optimized fringe pattern discriminator is optimizing the parameters of the fringe pattern generator so that the degree of similarity is maximized,
The fringe pattern discriminator consists of a deep neural network, and the parameters of the fringe pattern discriminator are parameters of the deep neural network of the fringe pattern discriminator,
In step (a), for the sampled points and area information, a loss function between the fringe patterns for each first and second area is obtained, and the parameters of the fringe pattern discriminator are set so that the loss function is minimized. , artificial intelligence and deep learning-based super-resolution hologram generation method characterized by optimizing parameters.
According to claim 1,
The fringe pattern generator is composed of a real part generator and an imaginary part generator, and each generator generates a real fringe pattern for each area and an imaginary fringe pattern for each area.
delete delete delete A computer-readable recording medium recording a program for performing the method of generating a super-high-resolution hologram based on artificial intelligence and deep learning of any one of claims 1, 3 to 6.

Images