KR102277100B1 - A method of generating hologram with random phase using deep learning and artificial intelligence - Google Patents

Abstract

The present invention relates to a method of generating a hologram with a random phase using deep learning and artificial intelligence to post-process a random phase by using a deep neural network, rather than adding a random phase, to display the hologram when generating the hologram, the method is configured to include: (a) optimizing parameters of a fringe pattern generator that receives object points and outputs the fringe pattern, in which the fringe pattern for the object point generated by a computer generated hologram (CGH) algorithm is determined as correct answer data; (b) optimizing a random phase inserter that receives the fringe pattern and outputs a fringe pattern to which a random phase is inserted (hereinafter referred to as a random phase fringe pattern), in which the random phase fringe pattern generated by the CGH algorithm is determined as the correct answer data; and (c) generating a digital hologram using the optimized fringe pattern generator and the random phase inserter. In the same way as above, the random phase is inserted into the fringe pattern generated by the fringe pattern generator through the random phase inserter as the post-processing process, so that a viewing angle is extended when displaying holograms.

Description

{ A method of generating hologram with random phase using deep learning and artificial intelligence }

The present invention generates a hologram using a deep neural network that uses a generative network based on deep learning technology, but does not add a random phase when generating a hologram for display of the hologram, but instead of adding a random phase using a deep neural network It relates to a method of generating a hologram having a random phase using artificial intelligence and deep learning technology.

In general, a hologram is composed of fringe patterns generated as a result of interfering a light wave (reference wave) as a reference with a light wave (object wave) reflected from an object. A digital hologram (DH) is obtained by acquiring a pattern as digital data. Therefore, although a hologram is 2D data, it is 3D data including distance information to an object due to a phase difference between an object wave and a reference wave [Non-Patent Document 1]. When the reference wave is irradiated to this hologram, the original image of the object at the original position is reproduced. Therefore, although the holographic image has a viewing angle limitation, since it displays a three-dimensional image, it is considered as the ultimate 3D image [Non-Patent Document 2].

From the perspective of broad classification, a method for generating a traditional digital hologram can be divided into a method using optical imaging and a computer generated hologram (CGH) method using a computer. 1 shows a method using optical imaging. Countless studies on optical imaging and CGH methods have been conducted so far. Each of these methods will have different advantages and disadvantages.

In the case of the CGH method, the biggest disadvantage is that the operation processing speed is very slow. In order to improve this speed, methods such as improving an algorithm, using a GPU, making a field programmable gate array (FPGA), or making a dedicated chipset have been suggested.

In addition, as shown in FIG. 2 , a method for generating a hologram using a deep learning technology that has been rapidly developed in recent years has been proposed [Patent Document 1].

Korean Patent Publication No. 10-2018-0064884 (published on June 15, 2018)

S. A. Benton and V. M. Bove Jr., Holographic imaging, John Wiley and Sons Inc., 2008. Ki-Pyung Han, "Giga Korea," TTL Journal, Vol. 146, pp. 16-21, March 2013 (March 2013).

It is an object of the present invention to solve the above-described problems, and to generate a hologram using a deep neural network using a generative network based on deep learning technology, but to generate a hologram for display of a hologram, a random phase It is to provide a method of generating a hologram having a random phase using artificial intelligence and deep learning technology, in which the random phase is post-processed using a deep neural network rather than added.

In addition, an object of the present invention is to learn a fringe pattern generator, obtain a fringe pattern including a random phase with the learned fringe pattern generator, and learn a random phase inserter using this, and randomize it with a fringe pattern generator and a random phase inserter An object of the present invention is to provide a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, which generates a fringe pattern including a phase.

In order to achieve the above object, the present invention relates to a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, (a) optimizing the parameters of a fringe pattern generator that receives an object point and outputs a fringe pattern, Optimizing a fringe pattern for an object point generated by a computer generated hologram (CGH) algorithm as correct answer data; (b) optimizing a random phase inserter that receives a fringe pattern and outputs a fringe pattern into which a random phase is inserted (hereinafter, a random phase fringe pattern), and optimizes the random phase fringe pattern generated by the CGH algorithm by determining the correct answer data; and, (c) generating a digital hologram using the optimized fringe pattern generator and the random phase inserter.

In addition, the present invention is a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, for the points sampled in step (a), a first fringe pattern for each point is generated by the CGH algorithm. And, by applying each point to the fringe pattern generator to generate a second fringe pattern, it is characterized in that the parameters are optimized so that the degree of similarity between the generated first and second fringe patterns is maximized.

In addition, the present invention provides a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, in the step (a), a fringe pattern discriminator indicating the degree of similarity between the first and second fringe patterns as a probability value; A phase discriminator representing the similarity of the first and second phases of the first and second fringe patterns as a probability value is configured, parameters of the fringe pattern discriminator and the phase discriminator are optimized, and the optimized fringe pattern discriminator and phase discriminator It is characterized in that the parameters of the fringe pattern generator are optimized so that the output value of the device is maximized.

In addition, the present invention provides a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, wherein the fringe pattern generator is composed of a real part generator and an imaginary part generator, and each generator is a real fringe pattern and an imaginary fringe pattern. characterized in that it creates

In addition, the present invention is a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, characterized in that the fringe pattern generator and the random phase inserter are composed of a deep neural network.

In addition, the present invention is a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, wherein the fringe pattern discriminator and the phase discriminator are composed of a deep neural network.

In addition, the present invention provides a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, for the points sampled in step (b), a first random phase fringe pattern for each point with a CGH algorithm and applying each point to the fringe pattern generator to generate a second fringe pattern, and applying the generated second fringe pattern to the random phase inserter to generate a second random phase fringe pattern. and optimizing the parameters of the random phase inserter so that the similarity of the second random phase fringe pattern is maximized.

In addition, the present invention is a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, in the step (b), for each object point, a random value between -π and π is generated, and the CGH algorithm It is characterized in that the first random phase fringe pattern is generated by adding the generated random value to the phase of the cosine or sine function.

In addition, in the method for generating a hologram having a random phase using artificial intelligence and deep learning technology, in the step (b), a random phase discriminator indicating the degree of similarity between the first and second random phase fringe patterns as a probability value and optimizing the parameters of the random phase discriminator, and optimizing the parameters of the fringe pattern generator so that the output value of the optimized random phase discriminator is maximized.

In addition, the present invention is a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, the random phase discriminator is characterized in that it is composed of a deep neural network.

In addition, the present invention is a method for generating a hologram having a random phase using artificial intelligence and deep learning technology, in the step (c), for the points of a given three-dimensional object, each point is applied to the fringe pattern generator, A fringe pattern is generated, the generated fringe pattern is applied to the random phase inserter to generate a random phase fringe pattern, the generated random phase fringe pattern is multiplied by the brightness value of each point, and the brightness value is multiplied by all points. It is characterized in that a digital hologram is generated by accumulating a random phase fringe pattern.

In addition, the present invention relates to a computer-readable recording medium in which a program for performing a method for generating a hologram having a random phase using artificial intelligence and deep learning technology is recorded.

As described above, according to the method for generating a hologram having a random phase using artificial intelligence and deep learning technology according to the present invention, the random phase is inserted into the fringe pattern generated by the fringe pattern generator as a post-processing process by a random phase inserter, When displaying a hologram, an effect of extending a viewing angle is obtained.

In addition, according to the method for generating a hologram having a random phase using artificial intelligence and deep learning technology according to the present invention, the process of generating a fringe pattern and inserting a random phase is all processed with a neural network, so that iterative calculation is required through a formula CGH method Compared to the prior art of , the operation can be processed at a very high speed, and the effect of saving the memory capacity is obtained.

That is, in the case of the software method, compared to the method using the formula, the method using the formula requires repeated calculations as much as the size of the hologram to generate the fringe pattern for one light source, whereas the method of the neural network method according to the present invention is one Since a single fringe pattern is output with one repetition, theoretically, the speed increases in proportion to the size of the hologram.

Also, that is, the memory capacity used can be very small compared to the lookup table method according to the related art. Specifically, in the case of the lookup table method, a memory capacity of several giga to several tera is required, whereas the method of the neural network method according to the present invention uses only tens of megabytes to a maximum of several hundred megabytes.

1 is an exemplary view showing a method of generating and restoring a digital hologram by an optical acquisition method according to the prior art.
2 is an exemplary view showing a digital hologram generation method using artificial intelligence according to the prior art.
3 is a diagram showing the configuration of an overall system for implementing the present invention.
4 is a diagram illustrating a process of generating a fringe pattern based on object points according to the method for generating a hologram of the present invention.
5 is a view showing a process of generating a fringe pattern (diffraction pattern for a single object pointer) using a deep neural network (deep learning model) according to the hologram generation method of the present invention.
6 is a flowchart illustrating a method for generating a hologram having a random phase using artificial intelligence and deep learning technology according to an embodiment of the present invention.
7 is a diagram illustrating a process of training a deep neural network (deep learning model) for generating a diffraction pattern (part of a diffraction pattern for a single object point) according to an embodiment of the present invention.
8 is a diagram illustrating a deep learning-based step of inserting a random phase according to an embodiment of the present invention.
9 is a flowchart illustrating a hologram generation step using a fringe pattern generation deep neural network according to an embodiment of the present invention.
10 is an exemplary view of a fringe pattern generated using a deep neural network according to an experiment of the present invention, (a) a real part, (b) an imaginary part.
11 is an exemplary diagram of a hologram generated using a deep neural network according to an experiment of the present invention, (a) a real part, (b) an imaginary part.
12 is an exemplary view of a hologram generation result using a deep neural network according to an experiment of the present invention, (a) a point cloud for an object, and (b) an exemplary view showing a hologram restoration result using the hologram of Fig. 8;

Hereinafter, specific contents for carrying out the present invention will be described with reference to the drawings.

In addition, in demonstrating this invention, the same part is attached|subjected with the same code|symbol, and the repetition description is abbreviate|omitted.

First, examples of the configuration of the entire system for implementing the present invention will be described with reference to FIG. 3 .

As shown in FIG. 3 , the method for generating a hologram having a random phase using artificial intelligence and deep learning technology according to the present invention is implemented as a program system on a computer terminal 20 that receives three-dimensional data 10 and generates a hologram. can be That is, the digital hologram generating method may be configured as a program and installed in the computer terminal 20 to be executed. A program installed in the computer terminal 20 may operate as one program system 30 .

On the other hand, as another embodiment, the method for generating a hologram having a random phase using artificial intelligence and deep learning technology is composed of a single electronic circuit such as an ASIC (application-specific semiconductor) in addition to being composed of a program and operating in a general-purpose computer. have. Alternatively, it may be developed as a dedicated computer terminal that exclusively processes only the generation of a holographic image. This will be referred to as a digital hologram generating device 30 . Other possible forms may also be implemented.

On the other hand, 3D data is composed of a point cloud representing a 3D object.

Next, a basic principle and process of a method for generating a digital hologram using a deep learning technique according to the present invention will be described with reference to FIGS. 4 to 5 .

A computer-generated hologram (CGH) is obtained by accumulating the fringe patterns for all light sources input from the coordinates of all hologram planes and then accumulating the patterns.

Equation 1 shows the CGH formula of the real part for generating a complete complex hologram.

[Equation 1]

Here, I(u,v) is the brightness at the coordinates (u,v) of the hologram plane, and A _j (x,y,z) is the brightness of the light source at the x,y,z coordinates. N is the number of light sources, λ is the wavelength of a reference wave used to generate the hologram, and p is the size of the pixels of the hologram and the light source. Here, they are treated as the same value for convenience.

The process of Equation 1 is separated as shown in Equations 2 and 3.

[Equation 2]

[Equation 3]

Accordingly, as shown in FIG. 4 , a fringe pattern is generated for each object point using Equation 2 .

Meanwhile, the fringe pattern is made in the form of a complex number (I _Re + iI _Im ), and in the complex number, the real part (I _Re ) and the imaginary part (I _Im ) are calculated separately. In Equation 2, the real part is calculated as a cosine, but the imaginary part is calculated as a sine function.

At this time, as shown in FIG. 5 , the method according to the present invention is generated using a deep neural network for Equation (2).

The generated fringe pattern is multiplied by a brightness value as shown in Equation 3, and cumulative addition is performed to generate a final hologram.

Next, a method for generating a hologram having a random phase using artificial intelligence and deep learning technology according to an embodiment of the present invention will be described with reference to FIGS. 6 to 9 .

As shown in FIG. 6 , the method for generating a digital hologram according to the present invention includes the steps of learning a fringe pattern generator (S10), learning a random phase inserter (S20), and inserting the learned fringe pattern generator and random phase It consists of a step (S30) of generating a digital hologram from the object data by using the At this time, in the learning stage, the 3D object points for learning are learned, and a digital hologram is generated using the actual target 3D object points.

First, the step of learning the fringe pattern generator (S10) will be described.

7 shows the structure of a model for learning a deep neural network to generate a fringe pattern. The overall structure is based on the adversarial generative model.

As shown in FIG. 7 , when object point information is input, a real fringe pattern (hereinafter, a first fringe pattern) is generated using a computer generated hologram (CGH) algorithm. That is, the real fringe generated by the CGH algorithm is used as the correct answer data. In addition, the learning method described herein applies one learning method (eg, GAN learning method) for an example, and can be applied to all supervised learning methods of neural networks using correct answer data.

At the same time, a fake fringe pattern (hereinafter referred to as a second fringe pattern) is generated using a deep neural network-based fringe pattern generator. However, here, in Real Fringe, Real means real, and in Real Data, Real means real information.

The Fringe Pattern Discriminator based on Deep Neural Network compares these two results and outputs how similar the real and the fake are as a probability value, and based on this probabilistic calculation result, the fringe pattern generator is more We get a loss value so that we can create a fake that is similar to the real thing.

In addition, in order to prevent the loss of the phase information containing the depth information of the hologram, a fake phase obtained from a fake real fringe and a fake imaginary fringe, and a real phase obtained from a real real fringe and a real imaginary fringe A phase discriminator based on deep neural network compares and outputs a probability value, and a loss value is calculated based on this comparison.

Based on this probabilistic calculation result, the internal parameters of the fringe pattern generator are updated so that the fringe pattern generator can make a fake more similar to the real thing. If this fixing is repeatedly performed, the real and the fake become indistinguishably similar. This enables the fringe pattern generator to generate the same fringe pattern as the fringe pattern generated by CGH, and the deep neural network outputs the fringe pattern. It means you have been taught to do it.

Hereinafter, each term and method in the description will be described.

c ₁ ,...,c _m : (x,y,z) spatial coordinates of an object point in the form of an integer or real number, which is condition information. That is, x, y, and z represent the spatial coordinates of the object.

z ⁽¹⁾ ,...,z ^(m) : One-dimensional noise or noise image.

f _re ⁽¹⁾ ,..., f _re ^(m) : A Real Real Fringe Pattern generated using a computer-generated hologram algorithm.

f _im ⁽¹⁾ ,...,f _im ^(m) : A Real Imaginary Fringe Pattern generated using a computer-generated hologram algorithm.

G _fringe (z _i | c _i ): Given the coordinate condition c _i of the object point, generate a fringe pattern using noise z _{i as an input.} This is called a fringe pattern generator, and to generate a real hologram and an imaginary hologram, G _Re (z _i |c _i ) and G _Im (z _i |c _i ) It consists of two deep neural networks.

D _fringe (x): represents the probability that x comes from the noise p _{g (z).} That is, it probabilistically indicates how similar the generated fringe pattern is to the actual fringe pattern. It is called a fringe pattern discriminator, and two deep neural networks, _{D Re} (x) and D _Im (x), are used to discriminate real holograms and imaginary holograms. network) is composed of

D _phase (x): represents the probability that _{x came from the noise p g (z).} That is, it probabilistically indicates how similar the phase component from the generated fringe pattern and the phase component from the actual fringe pattern are. This is called a phase discriminator.

: 프린지 패턴 생성기 G_Im(z_i|c_i)를 통해 만들어진 허수부 프린지 패턴이다.

: An imaginary fringe pattern created by the fringe pattern generator G _Im (z _i |c _{i ).}

: 프린지 패턴 생성기 G_Re(z_i|c_i)를 통해 만들어진 실수부 프린지 패턴이다.

: A real-part fringe pattern created by the fringe pattern generator G _Re (z _i |c _{i ).}

▽ _x : It is a Del operator for the differentiation of deep neural networks using stochastic gradient ascending with respect to x.

ε: means a random number between 0 and 1.

θ _G : A parameter of the fringe pattern generator composed of a deep neural network. These parameters include the weight, bias, and optimizer parameters of the neural network. The parameters of the optimizer are determined according to the type of optimizer. For example, when using Adam, α, β ₁ , and β ₂ are included.

θ _D : The parameter of a discriminator composed of a deep neural network. These parameters include the weight, bias, and optimizer parameters of the neural network. The parameters of the optimizer are determined according to the type of optimizer. For example, when using Adam, α, β ₁ , and β ₂ are included.

L: It is a loss function used for deep neural network training in deep learning. Currently, this document uses WGAN-GP (Wasserstein Generative Adversarial Network - Gradient Panelty) as an example, but other loss functions may be used.

m: number of mini-batches

k : number of training (training)

First, in the learning step (S10) of the fringe pattern generator, the points of the three-dimensional object are sampled with a mini-batch size m, and a fringe pattern generator composed of a neural network using the sampled data, a fringe pattern discriminator, and, Train (or train) the phase discriminator.

First, the _{noise z 1} ,z ₂ ...,z _{m is sampled from the predetermined noise p g} (z), and the first real fringe pattern f ¹ _Re ,f ² _Re ,...,f by the CGH algorithm. ^m _Re and the first imaginary fringe pattern f ¹ _Im ,f ² _Im ,...,f ^m _Im are generated. Also, the condition c ₁ ,c ₂ ...,c _m is sampled from the object points.

Next, for each sampled data, a second imaginary fringe pattern and a second real fringe pattern are generated by a (imaginary or real) fringe pattern generator. In this case, the following equation is used.

[Equation 4]

Next, the first phase f _phase is calculated using the first imaginary fringe pattern and the first real fringe pattern, and the second phase f ^to _phase is calculated using the second imaginary fringe pattern and the second real fringe pattern.

[Equation 5]

^{Next, a third phase f ^} _{phase is calculated by varying the} weights of the first phase and the second phase with a random value ε, and a phase loss function L phasei is calculated.

[Equation 6]

[Equation 7]

Here, λ is a penalty coefficient and serves to adjust the size of the loss function. λ is a predetermined value. In addition, D( ) and the like are values obtained by the discriminator, and indicate how similar the values are to the actual values. Also, |||||| ₂ is an L2 norm operation, and it is an operation which squares and adds each element inside, and takes the square root. That is, the normal operation is an operation that measures the length or size of a vector. ▽ _f D(f _i ) is the value obtained by differentiating D(f _i ) with f by the Del operator (Del operator to differentiation of deep neural networks using stochastic gradient ascending technique) ▽.

Next, a third imaginary fringe pattern is calculated by varying the weights of the first imaginary fringe pattern and the second imaginary fringe pattern with a random value ε, and a loss function L _Imi of the imaginary fringe pattern is calculated.

[Equation 8-1]

[Equation 8-2]

Next, a third real fringe pattern is calculated by varying the weights of the first real fringe pattern and the second real fringe pattern with a random value ε, and a loss function L _Rei of the real fringe pattern is calculated.

[Equation 9-1]

[Equation 9-2]

Next, if the loss function is obtained as above for all sampling data of each mini-batch, it is learned by optimizing the parameters of the phase discriminator, the imaginary fringe pattern discriminator, and the real fringe pattern discriminator as shown in the following equation.

[Equation 10]

θ _D represents the parameters of each discriminator. The parameters of the discriminator are parameters of the neural network, and represent weights, biases, and the like. Also, Optimizer(x,parameters) represents a function that optimizes by changing the parameter parameters so that the value of x is minimized. That is, Equation 10 indicates that _{the average value of the loss function L i} of each sampling data is differentiated by the del operator (▽), and the optimization is performed by obtaining the values of parameters optimizing the differentiated value.

In other words, the point with the smallest loss in the loss function is the point where the slope of the loss function is zero. In order to find the point where the slope of the loss function, which is a multidimensional function, is zero, the slope of the loss function is obtained by taking the del operator. The stochastic gradient descent method is to construct a loss function with the difference between the result obtained through forward propagation of the neural network and the correct answer, and then adjust the parameters constituting the neural network so that the gradient of the loss function becomes zero. However, Equation 10 is an optimization process. After obtaining the slope by differentiating the loss function, the problem of how to change the parameters of the neural network so that the slope of the loss function becomes zero using this slope remains. Here, the process of changing the parameters of the neural network using the differential value of the loss function is optimization, and performing this is the optimizer.

Next, the previous learning process (or training process) is repeated as many times as the number of training k. That is, in the previous process, each discriminator was trained once using the sampling data. Each discriminator is learned by repeating this learning process k times.

Next, for training of the fringe pattern generator, the points of the object and the noise are sampled. And, as shown in the following equation, the result (or probability) generated by the generator G is determined by the discriminator D, and the parameters of the generator G are optimized by maximizing the discrimination result. In this case, the previously trained discriminator is used.

[Equation 11]

Here, θ _GIm denotes an imaginary fringe pattern generator, and θ _GRe denotes a real fringe pattern generator.

In summary, we use the method of training generator 1 after training discriminator k. Randomly extract noise, actual fringe patterns and conditions as many as m mini-batch sizes, and use them to forward the model. After calculating the loss function using the forward propagation result and finding the derivative of the loss function, the parameters of the discriminator are updated through the backpropagation operation. Next, the model is forward propagated again using the randomly extracted noise and conditions, and the parameters of the generator are updated while backpropagating using the results. Here, forward propagation is the process of making an output by putting an input into the neural network, and it is the same as inference or testing. Backpropagation is a process of differentiating the loss function, which is the difference between the forward propagation result and the correct answer, and changing the parameters of the neural network while moving the differentiation result from the output part to the input part of the neural network using an optimization algorithm.

Next, the step of learning the random phase inserter (S20) will be described.

As shown in FIG. 8 , a random phase is inserted as a post-processing process into the fringe pattern generated using the fringe pattern generator created through the fringe pattern generation step S10 . In order to improve the problem of a hologram having a narrow viewing angle during display, a random phase is inserted, which is the method so far that it should be inserted when generating a hologram. However, using deep learning, post-processing creates a fringe pattern with a random phase added with an extension of the viewing angle. This also uses the adversarial generative model that was used to generate the fringe pattern.

Hereinafter, a process of applying a random phase to a fringe pattern created through a fringe pattern generator as a post-processing will be described in detail. In this case, each term and method used will be first described.

c ₁ ,...,c _m : (x,y,z) spatial coordinates of an object point in the form of an integer or real number, which is condition information.

f ⁽¹⁾ ,...,f ^(m) : A Real Fringe Pattern with a random phase corresponding to a condition generated using a computer-generated hologram algorithm.

: 랜덤 페이즈 삽입기를 통해 만들어진 가짜 랜덤 페이즈 프린지 패턴이다.

: This is a fake random phase fringe pattern created through a random phase inserter.

f _fake ⁽¹⁾ ,...,f _fake ^(m) : A fake fringe pattern created by the fringe pattern generator.

G _rand (f _fake ): Generates a random phase fringe pattern with a random phase inserted by using the fake fringe pattern generated from the fringe pattern generator as an input. This is called a Fringe Pattern Transfer with Random Phase.

D _rand (f _fake ,f) : represents the probability that _{x came from f fake .} That is, it probabilistically indicates how similar the generated random phase fringe pattern is to the actual random phase fringe pattern. This is called a random phase fringe pattern discriminator.

▽ _x : It is a Del operator for the differentiation of deep neural networks using stochastic gradient ascending with respect to x.

ε: means a random number between 0 and 1.

θ _G : The parameter of a random phase inserter composed of a deep neural network. These parameters include the weight, bias, and optimizer parameters of the neural network. The parameters of the optimizer are determined according to the type of optimizer. For example, when using Adam, α, β ₁ , and β ₂ are included.

θ _D : A parameter of a random phase discriminator composed of a deep neural network. These parameters include the weight, bias, and optimizer parameters of the neural network. The parameters of the optimizer are determined according to the type of optimizer. For example, when using Adam, α, β ₁ , and β ₂ are included.

L: It is a loss function used for deep neural network training in deep learning. Currently, in this document, the use of L1 loss and that of WGAN-GP (Wasserstein Generative Adversarial Network - Gradient Panelty) is used as an example, but other loss functions may be used.

m: number of mini-batches

k : number of training (training)

First, after training the k discriminator, the method of training the generator 1 is used. Fringe patterns with actual random phases inserted as much as m mini-batch sizes and fringe patterns from the fringe pattern generator are extracted, and the model is propagated using them. After calculating the loss function using the forward propagation result and finding the derivative of the loss function, the parameters of the discriminator are updated through the backpropagation operation. Next, the model is forward propagated again using the randomly extracted noise image and conditions, and the parameters of the generator are updated while backpropagating using the results. One such random phase inserter may be attached to a real and imaginary fringe pattern generator, respectively, or only one inserter for the two generators may be used.

Specifically, first, the condition c ₁ ,c ₂ ...,c _m is sampled from the object points, and the actual random phase fringe pattern (or the first random phase fringe) by the CGH algorithm to which the condition (mini-batch of conditions) is applied. pattern) f ₁ ,f ₂ ...,f _m are sampled. That is, the first random phase fringe pattern generated by the CGH algorithm is used as correct answer data. In addition, the learning method described herein applies one learning method (eg, GAN learning method) for an example, and can be applied to all supervised learning methods of neural networks using correct answer data.

The equation for inserting a random phase into the hologram of the real part when generating the hologram is as follows.

[Equation 21]

When inserting a random phase into the imaginary part hologram, you can change the cosine to a sine function. It consists of a random value between -π and π. These random values are all randomly sampled for each object point.

Therefore, as shown in the following equation, a random phase fringe pattern is generated by adding a random phase to the phase of the equation for generating a fringe pattern in the CGH algorithm for each object point.

[Equation 21]

_{Next, the fake fringe pattern (or the second fringe pattern) f fake} ⁽¹⁾ ,...,f _fake ^(m) by the fringe pattern generator to which the mini-batch of conditions is applied is sampled.

Then, for each sampling data i, a second random phase fringe pattern (or false random phase fringe pattern) f ^~ _i is generated from the false fringe pattern (second fringe pattern) using a random phase inserter. In this case, the following equation is used.

[Equation 12]

^{Next, a third random phase fringe pattern f ^} _i is calculated by varying the weights of the first random phase fringe pattern f _i and the second random phase fringe pattern with a random value ε, and a phase loss function L _i is calculated.

[Equation 13]

Here, f _i , f ^~ _i , and f ^{^} _i are first, second, and third random phase fringe patterns, respectively, ε is a preset weight, and Li is a loss function. In addition, D _rand () is a random phase discriminator, λ is a penalty coefficient, and ▽ _f^ D _rand (f ^{^} _i ) converts D _rand (f ^{^} _i ) to f ^{^} with the del operator (stochastic gradient ascending technique). It is the value differentiated by the del operator) ▽ to the differentiation of the deep neural network used.

Next, if the loss function is obtained for all the sampling data of each mini-batch as above, the parameters of the random phase discriminator are optimized as shown in the following equation to learn.

[Equation 14]

Here, L _i is the loss function, ▽ _θD represents the del operator to the differentiation of the deep neural network using the stochastic gradient ascending technique by θ _D , and the Optimizer(x,parameter) sets the parameter parameters so that the value of x is minimized. Represents a function that changes and optimizes.

Next, for training the random phase inserter, the fake fringe patterns f _fake ⁽¹⁾ ,...,f _fake ^(m) by the fringe pattern generator are sampled.

Next, a loss function representing the L1 loss between the _{fake image G rand} (f ⁽ⁱ⁾ _fake ) created from the random phase inserter and the corresponding real random phase insert image f _{i is obtained.} L1 Loss ≫ ₁ is the one-dimensional Euclidean distance for each pixel of the image.

[Equation 15]

And, as shown in the following equation, the result (or probability) generated by the inserter G is determined by the discriminator D, and parameters θ _G of the inserter G are optimized by maximizing the discrimination result. In this case, the previously trained discriminator is used.

[Equation 16]

Here, ▽θ _G represents the del operator by θ _{G , and G rand} (f ⁽ⁱ⁾ _fake ) is a random phase that inserts and outputs a random phase when a fringe pattern from the network, which is the result from the fringe pattern generator, is given. Indicates the value of the inserter.

Next, the step of generating a digital hologram (S30) will be described.

9 shows a procedure for generating a digital hologram using a fringe pattern generator including a random phase created through the fringe pattern generation training step S10 and the random phase fringe pattern generation training step S20.

In the process described above, a real fringe pattern and an imaginary fringe pattern including a random phase are generated. The subsequently generated fringe pattern is multiplied by the brightness value, and cumulative addition is performed with the previously generated fringe pattern. When fringe patterns for all object points are accumulated and added, a final hologram is created.

Hereinafter, a detailed procedure for generating a hologram using the fringe pattern generation model will be described. Each term and method used at this time will be described.

H: One digital hologram

A _i : Brightness value of object point

That is, coordinate information and brightness information to be used as conditions for deep neural networks are called from object point information. Using the coordinate information, a fringe pattern f ^~ _i is generated _{through a fringe pattern generator (G fringe} ) and a random phase inserter ( _{Grand ).} In this case, the following equation is used.

[Equation 17]

In addition, a hologram is generated by accumulating after giving brightness weights to the generated fringe patterns.

In this case, the following equation is used.

[Equation 18]

Next, the effects of the present invention will be described through experiments.

10 shows a complex fringe pattern generated using a fringe pattern generator. The real number in Fig. 10(a) and the imaginary part fringe pattern in Fig. 10(b) are shown for 64 conditions, respectively.

A hologram generated by cumulatively adding fringe patterns generated through a fringe pattern generator is shown in FIG. 10 . Fig. 11(a) is a real part, and Fig. 11(b) is an imaginary part.

The result of restoring the generated hologram is shown in FIG. 12 . 12( a ) shows the point cloud information of an object made into an 8-bit image file. FIG. 12( b ) is a result of numerically restoring the hologram of FIG. 11 . As can be seen from the results, it was confirmed that a digital hologram very similar to that of CGH was generated with a difference of 1 to 5 dB.

In the above, the invention made by the present inventors has been described in detail according to the above embodiments, but the present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention.

10: three-dimensional data 20: computer terminal
30: program system

Claims (12)

In the method for generating a hologram having a random phase using artificial intelligence and deep learning technology,
(a) Optimize the parameters of the fringe pattern generator that receives the object points and outputs the fringe pattern, but determines the fringe pattern (hereinafter the first fringe pattern) for the object point generated by the CGH (computer generated hologram) algorithm as the correct answer data optimizing;
(b) optimizing a random phase inserter that receives a fringe pattern and outputs a fringe pattern (hereinafter referred to as a random phase fringe pattern) in which a random phase is inserted, but a random phase fringe pattern generated by the CGH algorithm (hereinafter, the first random phase fringe pattern) optimizing by setting as the correct answer data; and;
(c) generating a digital hologram using the optimized fringe pattern generator and the random phase inserter,
The fringe pattern generator is composed of a deep neural network, and the parameters of the fringe pattern generator are parameters of the deep neural network of the fringe pattern generator,
In step (a), for the sampled points, a second fringe pattern generated by applying each point to the fringe pattern generator, and a first fringe that is correct answer data generated by applying each point to the CGH algorithm optimizing the parameters by obtaining the degree of similarity between the patterns and setting the parameters of the fringe pattern generator so that the degree of similarity is maximized,
The random phase inserter consists of a deep neural network, and the parameters of the random phase inserter are parameters of the deep neural network of the random phase inserter,
In step (b), with respect to the sampled points, the second random phase fringe pattern is generated by applying the second fringe pattern generated by the fringe pattern generator to the respective points to the random phase inserter, and the generated The degree of similarity between the second random phase fringe pattern and the first random phase fringe pattern, which is correct data generated by inserting a random phase by the CGH algorithm at each point, is obtained, and the degree of similarity is maximized. A method of generating a hologram having a random phase using artificial intelligence and deep learning technology, characterized in that by setting the parameters of the deep neural network, the random phase inserter is optimized.
delete According to claim 1,
In step (a), a fringe pattern discriminator representing the degree of similarity between the first and second fringe patterns as a probability value, and a phase discriminator representing the degree of similarity between the first and second phases of the first and second fringe patterns as a probability value. configure, optimize the parameters of the fringe pattern discriminator and the phase discriminator, and optimize the parameters of the fringe pattern generator so that the degree of similarity that is the output value of the optimized fringe pattern discriminator and the phase discriminator is maximized,
The fringe pattern discriminator consists of a deep neural network, and the parameters of the fringe pattern discriminator are parameters of the deep neural network of the fringe pattern discriminator,
In step (a), for the sampled points, a loss function between the first and second fringe patterns of each point is obtained, and the parameters of the fringe pattern discriminator are set so that the loss function is minimized. to optimize,
The phase discriminator is composed of a deep neural network, and the parameters of the phase discriminator are parameters of the deep neural network of the phase discriminator,
In the step (a), for the sampled points, the loss function between the first and second phases of each point is obtained, and the parameters of the fringe pattern discriminator are set so that the loss function is minimized. A method for generating a hologram having a random phase using artificial intelligence and deep learning technology, characterized in that optimization.
According to claim 1,
The fringe pattern generator is composed of a real part generator and an imaginary part generator, and each generator generates a real fringe pattern and an imaginary fringe pattern. A method for generating a hologram having a random phase using artificial intelligence and deep learning technology.
delete delete delete According to claim 1,
In step (b), for each object point, a random value between -π and π is generated, and a first random phase fringe pattern is generated by adding the generated random value to the phase of the cosine or sine function in the CGH algorithm. A method of generating a hologram having a random phase using artificial intelligence and deep learning technology, characterized in that
According to claim 1,
In step (b), a random phase discriminator representing the degree of similarity between the first and second random phase fringe patterns as a probability value is constructed, the parameters of the random phase discriminator are optimized, and the output value of the optimized random phase discriminator is Optimize the parameters of the random phase inserter so that the degree of similarity is maximized,
The random phase discriminator consists of a deep neural network, and the parameters of the random phase discriminator are parameters of the deep neural network of the random phase discriminator,
In step (b), with respect to the sampled points, by obtaining a loss function between the first and second random phase fringe patterns of each point, and setting the parameters of the random phase discriminator so that the loss function is minimized, A method for generating a hologram having a random phase using artificial intelligence and deep learning technology, characterized in that the parameter is optimized.
delete According to claim 1,
In step (c), for the points of a given three-dimensional object, each point is applied to the fringe pattern generator to generate a fringe pattern, and the generated fringe pattern is applied to the random phase inserter to generate a random phase fringe pattern. Artificial intelligence and deep learning technology, characterized in that the generated random phase fringe pattern is multiplied by the brightness value of each point, and the random phase fringe pattern of all points multiplied by the brightness value is accumulated to generate a digital hologram. A method of generating a hologram having a random phase using
Claims 1, 3, 4, 8, 9, 11 as a computer recording a program for performing the method of generating a hologram having a random phase using artificial intelligence and deep learning technology of any one of claims 1, 3, 4, 8, 9, 11 readable recording medium.

Images