KR102188732B1 - System and Method for Data Processing using Sphere Generative Adversarial Network Based on Geometric Moment Matching - Google Patents

Abstract

The present invention relates to a data processing apparatus and method using an adversarial generating network on a sphere through geometric moment matching to solve the problem of gradient constraints by assuming that the feature space of the discriminant network is a sphere and minimize multi-order moments thereon. As a result, a generator that receives a value from random noise z to generate fake data and transmits the generated fake data to a discriminator; a discriminator that receives and trains the real data and the fake data; and n-dimensional Euclid receiving final data from the discriminator (Euclidean) Mapping to the feature space, re-mapping the feature points of fake data and real data into n-dimensional hyperspheres by geometric transformation, respectively, and using the mapped points to map the north of the hypersphere to a spherical GAN. pole); a geometric moment matching unit that calculates a geometric moment centered on the pole); wherein the discriminator attempts to maximize a moment difference in probability measurement between real and fake data, and the generator attempts to interfere with the discriminator by minimizing the moment difference. It is characterized by trying to train.

Description

System and Method for Data Processing using Sphere Generative Adversarial Network Based on Geometric Moment Matching

The present invention relates to a generative model that is a subfield of machine learning and computer vision, and specifically, assuming that a feature space of a discriminant network is a sphere, geometric moment matching that minimizes multi-order moments on it to solve the problem of gradient constraints It relates to a data processing apparatus and method using a hostile generation network on the sphere through the.

Generative Adversarial Networks (GANs) are a wide range of computer vision to provide a variety of functions such as image generation, super resolution, video prediction, style transfer, visual tracking, 3D reconstruction, segmentation, object detection, reinforcement learning, and medical imaging. It is being used to achieve high performance in applications.

This hostile generation network has been receiving a lot of attention since its first introduction by Ian J. Goodfellow in 2014.

Unlike existing deep learning algorithms such as CNN and RNN, GAN generates image and voice data using a non-transfer learning method.

The principle of operation is that different subjects composed of generators and discriminators compete against each other and maximize their performance. This is the principle of creating fake data close to real data through this process.

Using GAN, you can restore a low-quality image or create a finished image with just a simple sketch. In reality, the GAN model can be used for automating disaster accident inspection, where the data necessary for learning are absolutely insufficient.

GAN of the prior art attempts to minimize the distribution difference between fake data and real data.To this end, a generator tries to generate a desired sample that looks like real data, and a discriminator tries to distinguish it from the real data. do.

GAN like this has been successfully applied to various tasks, but it is very difficult to train it. Therefore, it is difficult to solve more complex problems using GAN.

As such, Generative Adversarial Networks (GANs) are networks for generative models, which are subfields of machine learning and computer vision, and several studies are being conducted to increase the quality of generated images similar to real images.

In order to solve the problem of the first GAN, various technologies such as DCGAN, LSGAN, and WGAN were proposed. GAN, which is developing in this way, is used not only in academia, but also in the actual IT industry as a whole, and is creating various values, especially in the image processing field.

However, in order to reduce the difference between the probability measure representing the actual data and the probability measure created by the generating network, the Wesserstein distance is used, and for this purpose, a constraint is placed on the gradient of the discriminant network.

This can be helpful for training, but it limits the capacity of the network, which results in poor performance due to not fully utilizing the characteristics of the neural network.

Therefore, development of a data processing technology using a hostile generation network of a new technology to solve the problem of the gradient constraint is required.

Korean Patent Registration No. 10-1843066 Republic of Korea Patent Publication No. 10-2018-0120478

The present invention is to solve the problem of the data processing technology using the hostile generation network of the prior art, assuming that the feature space of the discrimination network is a sphere, the geometrical structure that minimizes the multi-order moment thereon to solve the problem of the gradient constraint An object thereof is to provide a data processing apparatus and method using a hostile generation network on a sphere through moment matching.

The present invention is a new concept sphere GAN (sphere GAN) that provides several advantages over general IPM-based GANs, so that a metric between mathematically well-defined measures is newly defined to predict the behavior of a trained network. An object of the present invention is to provide a data processing apparatus and method using a hostile generation network on a sphere through one geometric moment matching.

The present invention uses Riemannian manifolds to define IPM (integral probability metrics) in the GAN objective function, and uses geometric moment matching to enable stable training without using a gradient penalty or virtual data sampling technique. An object thereof is to provide a data processing apparatus and method using the hostile generation network described above.

The present invention uses an algebraic function in Sphere GAN to enable stable training by binding IPM in the objective function, and geometric moment matching to provide more accurate results by utilizing high-dimensional data statistical information. An object thereof is to provide a data processing apparatus and method using a hostile generation network on a sphere through matching.

Sphere GAN according to the present invention can efficiently match higher-order moments in a feature space to significantly improve accuracy, and generate images that are almost similar to real life through a network that creates images from image sets for real objects such as CIFAR-10 and STL-10. An object of the present invention is to provide a data processing apparatus and method using a hostile generation network on a sphere through geometric moment matching.

Other objects of the present invention are not limited to the objects mentioned above, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

A data processing apparatus using a hostile generation network on a sphere through geometric moment matching according to the present invention to achieve the above object generates fake data by receiving a value from random noise z, and uses the generated fake data as a discriminator. A generator that delivers; A discriminator that receives and trains real data (x) and fake data; receives final data from the discriminator and maps it to an n-dimensional Euclidean feature space, and geometrically transforms feature points of fake data and real data, respectively Including; a geometric moment matching unit for re-mapping to an n-dimensional hypersphere by using the mapped point, and calculating a geometric moment centered on the north pole of the hypersphere with a spherical GAN using the mapped point, The discriminator is characterized in that the discriminator attempts to maximize the moment difference of the probability measurement between the real and the fake data, and the generator attempts to interfere with the discriminator by minimizing the moment difference.

The data processing method using the hostile generation network on the sphere through geometric moment matching according to the present invention to achieve another object is assuming that the feature space of the discriminant network is a sphere, minimizing the multi-order moment on it, and between mathematically defined measures. Generating fake data similar to the real thing by receiving a value from the random noise z in a generator using a spherical GAN that allows predicting the behavior of the network to be trained by newly defining a metric; discriminating fake data generated in the generator Transmitting to a Qi; The discriminator receives real data and fake data, trains them, and maps them to an n-dimensional Euclidean feature space; Each feature point of the fake data and real data is converted back into an n-dimensional hypersphere by geometric transformation. Mapping; Using the mapped points, the spherical GAN calculates a geometric moment about a north pole of a hypersphere; the discriminator of the spherical GAN calculates the moment difference of the probability measurement between real and fake samples. And training by attempting to maximize, and the generator attempting to interfere with the discriminator by minimizing the moment difference.

The data processing apparatus and method using a hostile generation network on a sphere through geometric moment matching according to the present invention as described above has the following effects.

First, assuming that the feature space of the discriminant network is a sphere, the problem of the gradient constraint can be solved by minimizing the multi-order moment on it.

Second, by using a new concept of sphere GAN (sphere GAN) that provides several advantages over general IPM-based GANs, a metric between mathematically well-defined measures is newly defined so that the behavior of the trained network can be predicted. do.

Third, Riemannian manifolds are used to define IPM (integral probability metrics) in the GAN objective function, so that stable training is possible without using a gradient penalty or virtual data sampling technique.

Fourth, Sphere GAN uses an algebraic function to bind the IPM in the objective function to ensure stable training, and to provide more accurate results by utilizing high-dimensional data statistics information using geometric moment matching.

Fifth, Sphere GAN can efficiently match higher-order moments in a feature space to greatly improve accuracy, so that images that are almost similar to real life can be generated through a network that creates images from image sets for real objects such as CIFAR-10 and STL-10. do.

1 is an overall configuration diagram of a data processing apparatus using a hostile generation network on a sphere through geometric moment matching according to the present invention
FIG. 2 is a diagram illustrating a stereographic projection, which is an example of a geometric transformation function according to the present invention.
3 is a flow chart showing a data processing method using a hostile generation network on a sphere through geometric moment matching according to the present invention
4 is a graph of evaluation results according to the dimension and matching mode of a hypersphere using IS (Inception scores)
5 is a graph of gradient characteristics of the Sphere GAN and WGAN-GP discriminator networks
6 is a graph comparing average calculation time repeated 100 times for different GAN variants
7A and 7B are qualitative results of the old GAN of the LSUN-bedroom data set and the configuration diagram of the qualitative results of the old GAN of the STL-10 data set.

Hereinafter, an exemplary embodiment of a data processing apparatus and method using a hostile generation network on a sphere through geometric moment matching according to the present invention will be described in detail as follows.

Features and advantages of a data processing apparatus and method using a hostile generation network on a sphere through geometric moment matching according to the present invention will become apparent through detailed description of each embodiment below.

1 is an overall configuration diagram of a data processing apparatus using a hostile generation network on a sphere through geometric moment matching according to the present invention.

The present invention assumes that the feature space of the discriminant network is a sphere and minimizes the multi-order moment on it to newly define the metric between mathematically well-defined measures to predict the behavior of the network to be trained, CIFAR-10, STL- It is possible to create an image that is almost similar to the real life through the creation network from the image set of the 10th real object.

The present invention relates to a generative adversarial network, which is one of the generative models that are sub-fields of machine learning and computer vision, and discrimination of GANs that distinguish real data from fake data. The discriminator includes a configuration that maps the final data to a sphere and a configuration that calculates the distance by mapping the final data to a sphere, and uses the presence of a bound of the distance. This is to enable stable GAN learning.

The data processing apparatus using the hostile generation network on the sphere through geometric moment matching according to the present invention receives a value from random noise z and generates an image similar to the real one, as shown in FIG. A generator 100 that transmits data to the discriminator 200, a discriminator 200 that receives and trains real and fake data, and maps to an n-dimensional Euclidean feature space, respectively, and a fake sample and a real sample A geometric moment matching unit that calculates the geometric moment centered on the north pole of the hypersphere using the mapped point and re-maps the feature points of the n-dimensional hypersphere by geometric transformation. Including 300, the discriminator 200 of the spherical GAN attempts to maximize the moment difference of the probability measurement between the real and the fake sample, and the generator 100 trains by attempting to interfere with the discriminator by minimizing the moment difference. will be.

First, when z enters the generator 100, a fake image is created and transmitted as an input of the discriminator 200.

Thereafter, the sphere 34 is mapped to the sphere 34 as shown in the yellow above through Inver stereographic projection (ISP) through LReLu (31), Avg Pooling (32), and Dense (33).

At this time, the ISP plays a role of mapping the points on the plane, which are the output of Dense, to the points on the sphere.

Fig. 1 shows the pipeline structure of an old GAN, where fake data is generated from noise input by a generator.

The real and spurious data are then sent to a discriminator to map its output to an n-dimensional Euclidean feature space (ie, a yellow plane).

The green and purple circles on the plane represent the feature points of the fake and real samples, respectively. By geometric transformation, these feature points are mapped back to n-dimensional hyperspheres (ie, yellow spheres).

Using these mapped points, the spherical GAN calculates the geometric moment about the north pole of the hypersphere.

The discriminator of the old GAN attempts to maximize the moment difference of the probabilistic measure between the real and the fake sample, while the generator attempts to interfere with the discriminator by minimizing the moment difference.

Using the geometric moments defined in the hypersphere, the generator and discriminator are to improve performance through a two-player minimax game.

The hostile generation network on a sphere through geometric moment matching according to the present invention will be described in detail as follows.

In the objective function based on the Westerstein metric, the first moment in the one-dimensional feature space is defined as follows.

Here, G and D represent the generator and the discriminator, respectively, and P and N represent the actual data and the potential code distribution, respectively, and the discriminator (D) in Equation 1 maps the data (x) to the real number (R), this is

Here, D must satisfy the 1-Lipschitz condition D ∈ Lip1, and X ⊂ Rn is an n-dimensional Euclidean image space.

As in the existing IPM-based GAN, the objective function of the old GAN is based on Equation 1.

Unlike the existing GAN, which directly matches the first moment of the one-dimensional feature space, the spherical GAN matches the high-dimensional and multiple moments defined in a hypersphere that extends to a dimension larger than three-dimensional.

To this end, the output of the discriminator according to the present invention is defined as follows.

Equation 1 shows the shape of the objective function of the existing WGAN (baseline of Sphere GAN).

Where x is the actual image data and z is the random noise.

Generator G receives a value from random noise z and generates an image similar to the real one, and Discriminator D receives real data x and trains.

The generator works in a way that minimizes the difference value expressed as above (min), and tries to minimize the'discrimination' trained by the actual discriminator, and the discriminator is trained to try to maximize it to determine it.

The objective function of the spherical GAN is summarized as follows.

는 각 샘플과 hypersphere N의 north pole사이의 r 번째 모멘트 거리를 측정하는 것을 나타내고, 아래 첨자 s는

가

상에서 정의되어 있음을 나타낸다.Function for r = 1, ..., R

Represents the measurement of the r-th moment distance between each sample and the north pole of hypersphere N, and the subscript s is

end

Indicates that it is defined in phase.

If the new objective function of Equation 4 is used, the old GAN defines the IPM on the hypersphere, and some restrictions imposed on the discriminator can be relaxed.

However, the condition to be satisfied in the objective function described above is that D must be a 1-Lipschitz function.

To satisfy this condition, the WGAN-GP, CT, and LP technologies based on WGAN use the following gradient penalties.

Here, G* represents a fixed generator and C represents an additional constraint defined in Table 1.

In Equation 5, all gradient norms must be calculated each iteration, which increases the computational complexity.

Since such a gradient penalty limits the capacity of a discriminator, it is necessary to take a performance reduction in order to satisfy the Lipschitz condition, and the present invention can solve such a problem.

Unlike the existing approach, the old GAN does not require additional constraints to ensure that the discriminator is located in the desired function space. Using geometric transformations, the spherical GAN ensures that the distance function is in the desired function space.

The new objective function of the discriminator is:

The algorithm in Table 2 shows the pseudo-code of the old GAN according to the present invention.

The hypersphere expanded to a dimension larger than 3D will be described as follows.

에서 정의된 특징 공간을 통해 여러 모멘트를 매칭한다.As in Equation 4, the spherical GAN is hypersphere

Multiple moments are matched through the feature space defined in.

The older GAN uses a hypersphere instead of any Riemann manifold M, providing the following advantages:

는 바운드를 형성하고 구현하기가 매우 용이하다.First, the hypersphere's distance function

Is very easy to form and implement bounds.

Second, the gradient norm works well as a distance function, which is important for stable learning.

Third, the Riemannian structure of hypersphere is suitable for defining GAN objects.

을 전형적으로 고려한다. 이러한 GAN은 임의의 리만 매니폴드를 모델링하여 확장할 수 있다. 그러나 이러한 매니폴드는 컴팩트하지 않고 거리 함수가 바운드되지 않고, 그레디언트 폭발(gradient explosion)과 불안정한 학습을 유발할 수 있다.In contrast, a general GAN is a Euclidean space with a Euclidean distance.

Is typically considered. These GANs can be extended by modeling any Riemann manifold. However, these manifolds are not compact, the distance function is not bound, and can lead to gradient explosions and unstable learning.

을 초구(hypersphere)

으로 변환하는 기하학 인식 변환 함수를 사용한다.To solve this problem, the old GAN is a Euclidean space

Hypersphere

Use a geometry recognition transform function to transform into.

This function is implemented by the last convolutional layer of the discriminator.

에서

에 이르는 미분동형사상(diffeomorphism)에 의해 설계된다. 따라서, 변환 함수는 미분 가능하고 특징 공간의 모든 점에서 차원을 보존할 수 있다.Such a conversion function according to the present invention is

in

It is designed by diffeomorphism to reach. Thus, the transform function is differentiable and can preserve dimensions at every point in the feature space.

The following describes stereographic projection as a geometric transformation function.

2 is a block diagram illustrating a stereographic projection, which is an example of a geometric transformation function according to the present invention.

을 초구(hypersphere)

으로의 미분동형사상(diffeomorphism)이다.The inverse function of stereographic projection is Euclidean space

Hypersphere

It is a diffeomorphism.

Intuitively, the inverse function of stereographic projection can be considered a method of projecting onto a hyperplane.

의 좌표계를 p = (p1, ..., pn)이라고 하고, 초구(hypersphere)의 중심점(north pole)을 N = (0, ..., 1)이라고 하면,

If the coordinate system of is p = (p1, ..., pn) and the north pole of the hypersphere is N = (0, ..., 1),

은 다음과 같이 정의된다.Inverse function of stereographic projection

Is defined as

을 투영한 후, 두 점 사이의 거리를 초구(hypersphere) 메트릭 측면에서 측정한다.Two points through the inverse function of stereographic projection

After projecting, the distance between the two points is measured in terms of a hypersphere metric.

는

에 정의되는 거리함수이다. 기하학적으로

는 측지 거리로 고려될 수 있다.here,

Is

It is a distance function defined in. Geometrically

Can be considered as a geodetic distance.

As shown in Fig. 2, since the geodesic distance between two points on the hypersphere is much shorter than the Euclidean distance and is bound to the hyperplane (i.e., yellow sphere), a sphere having an objective function in Equation 4 Stable learning is possible when using GAN.

(Body adjustment 1) The distance function of Equation 8 is differentiable and bound.

Since the distance function of Equation 8 satisfies a non-negative value, a symmetric value, and a triangular inequality, it can be differentiated.

Since the hypersphere is a compact manifold, the distance between the two points is bound.

)For example, the Euclidean distance between two points 0 = (0, ..., 0) and q = (t, ..., t) diverges.

)

In contrast, the distance defined in the hypersphere in Equation 8 converges.

)(

)

The geometry-recognition transformation function of the spherical GAN limits the divergence of the boundary of the discriminator output to enhance a stable training power, preserves the dimension of the feature space, and maintains differentiation.

The mathematical analysis of the spherical GAN is as follows.

First, the connection to the IPM will be described as follows.

It is proved that minimizing the objective function of Equation 4 minimizes IPM.

To this end, we define the geometric center moment about the Riemannian manifold.

M is called a small, connected, geodetically complete Riemann manifold with Borel σ algebra, Σ.

와

는 모두 측정 가능한 공간 (M, Σ)에 정의된 확률 측정치이다.

Wow

Are all probability measures defined in the measurable space (M, Σ).

IPM is defined as follows.

와

사이의 거리 측정값이다.(Definition 1) IPM is two measures of probability

Wow

Is a measure of the distance between.

Here, F is a class of a function capable of measuring the boundary of the actual value for M.

The geometric moment about M is defined as follows.

의 r 번째 중심 모멘트는 다음과 같다.(Definition 2) on (M, Σ) for a given point p ₀

The r-th center moment of is

및

이다.here,

And

to be.

은 M에서의 리만 거리 함수(Riemannian distance function)이다.

Is the Riemannian distance function in M.

와

사이의 새로운 IPM을 다음과 같이 정의한다.In the old GAN

Wow

The new IPM between is defined as follows.

(Definition 3)

The IPM based on the moment difference is as follows.

은 주어진 점 p₀에서 다른 점으로부터의 유한 거리 함수(bounded distance functions)의 클래스이다.here,

Is a class of bounded distance functions from another point at a given point p ₀ .

When comparing (Definition 1) and (Definition 3), consider the relationship between the previous IPM and the IPM of the old GAN.

는 수학식 4의

에 해당하지만, M은

으로 대체될 수 있고,

는 중심점(north pole) N으로 설정할 수 있다.Of Equation 11

Is of Equation 4

Corresponds to, but M is

Can be replaced with

Can be set as the north pole N.

Then, an equation such as Equation 4 is obtained. This means that minimizing the objective function of Equation 4 is to minimize the IPM in Equation 11.

However, there are some differences between the previous IPM and the IPM of the old GAN.

을 중심으로 한 M상의 한정된 거리 함수의 집합이다.The function space of IPM according to the present invention is

It is a set of finite distance functions of the M phase centered on.

Therefore, the spherical GAN is as follows when the distance function is parameterized.

Here, {xi} is an image set. In contrast, the function space of WGAN's IPM is a set of 1-Lipschitz discriminants.

Therefore, if the discriminator is parameterized, it is as follows.

이다.here,

to be.

And the Westerstein distance link is as follows.

은 수학식 11에서 정의된 구형 GAN의 IPM이며, 여기서

이다.

Is the IPM of the spherical GAN defined in Equation 11, where

to be.

를 감소시키는 것을 목적로 하는데, 이는

에 정의된 두 가지 확률 측정

와

사이의 고차 중심 모멘트를 매칭시키는 것과 상응한다.The generator of the old GAN is

Is aimed at reducing

The two probability measures defined in

Wow

Corresponds to matching high-order central moments between.

가 약하게

에 수렴한다.(Proposition 1)

Weakly

Converge to

를

에 정의된 확률 측정의 r-웨서스테인 거리라 한다.

To

It is called the r-westerstein distance of the probability measure defined in.

을 최소화하는 것은 모든 r에 대해 r-웨서스테인 거리의 합계를 최소화하는 것과 같다.after that

Minimizing r is equivalent to minimizing the sum of the r-westerstein distances for all r.

이 0으로 수렴한다.(Proposition 2)

Converges to zero.

In the conventional GAN based on the Westerstein distance, the objective function is designed in a double form by the Kantorovich-Rubinstein duality theorem. In the dual form, it can be implemented only with 1-Wesserstein distance for efficient learning of GAN.

Unlike the conventional GAN, the spherical GAN area has a much wider function space because a more general r-Wesserstein distance can be used.

And the gradient analysis is as follows.

을 사용하면, 구형 GAN은

의 다른 모멘트를 선택하여 손실 함수(loss functions)의 그레디언트를 계산할 수 있다.Than other IPMs

Using, the old GAN is

You can calculate the gradient of the loss functions by selecting a different moment of.

If you choose different moments, the gradient leads to different learning behaviors. Therefore, it is possible to learn stable about any moment by using the old GAN.

(Supplementary Theorem 2)

(Supplementary Theorem 2) means that using a hypersphere is a reasonable option for stably learning GAN.

In the above description, W of WGAN is'Wasserstein', and is used as a metric for comparing the distance of the probability measure of the actual image and the probability measure of the generated image by using the Wasserstein distance.

In the present invention, a slightly larger range of metrics is defined in order to solve the problems of these technologies.

The metric used in the present invention is IPM (Integral Probability Metrics) and is defined in the same format as in Equation 9 above.

If f is a 1-Lipschitz function in Equation 9, the Wasserstein distance can be considered a type of IPM.

In the present invention, f is a distance function defined as in Equation 12. At this time, the distance function is the distance function between two points on the sphere.

This means that the 1-Lipschitz condition no longer needs to be satisfied, meaning that the gradient penalty does not need to be used.

Looking at Equation 12, in the previous WGAN-based techniques, when training a discriminator, the term of C(x) in Equation 5 is added to limit capacity, whereas in the present invention, Equation As in 6, since there is no additional term, there are fewer constraints on the discriminator network than the existing method, and accordingly, the performance is better even with the same neural network shape.

A data processing method using a hostile generation network on a sphere through geometric moment matching according to the present invention will be described in detail as follows.

3 is a flow chart showing a data processing method using a hostile generation network on a sphere through geometric moment matching according to the present invention.

First, the generator receives a value from the random noise z and generates an image (fake data) similar to the actual one. (S301)

Then, the fake data generated by the generator is transmitted to the discriminator (S302).

Then, the discriminator receives real data and fake data, trains it, and maps it to an n-dimensional Euclidean feature space (S303).

Subsequently, the feature points of the fake sample and the real sample are mapped back to an n-dimensional hypersphere by geometric transformation (S304).

And using the mapped point, the spherical GAN calculates the geometric moment centered on the north pole of the hypersphere (S305).

Subsequently, the discriminator of the spherical GAN attempts to maximize the moment difference of the probability measurement between the real and the fake sample, and the generator trains by attempting to interfere with the discriminator by minimizing the moment difference (S306).

Performance evaluation results of the data processing apparatus and method using the hostile generation network on a sphere through geometric moment matching according to the present invention described above are as follows.

4 is a graph of evaluation results according to different dimensions and moment matching modes of hypersphere using IS (Inception scores).

,

,

모멘트 매칭모드를 나타낸 것이고, 가로축은 초구(hypersphere)

의 차원을 나타낸 것으로,n = 16, 64, 256, 1024이다.The red, yellow and blue bars are respectively

,

,

Moment matching mode is shown, and the horizontal axis is hypersphere.

It represents the dimension of n = 16, 64, 256, 1024.

5 is a graph of gradient characteristics of the Sphere GAN and WGAN-GP discriminator networks.

Table 3 below shows the results of unsupervised image generation of CIFAR-10, and the higher the IS (Inception Scores) is, the lower the FID (Frechet Inception Distance) is, the better results are interpreted.

As shown in Table 3, which shows the quantitative results of CIFAR-10, the old GAN-ResNet has the latest records, and it can be seen that there is a big difference between both IS and FID, and the old GAN-Conv is WGAN-GP and MMD. It shows much better results than GAN.

Table 4 below shows the results of unsupervised image generation in STL-10.

In the STL-10 experiment, about half of the number of network parameters compared to the original network was used, and despite the small number of network parameters, the old GAN-ResNet was shown in Table 4, SN-GAN and other IPM-based GANs. It shows better performance.

Table 5 below shows the results of unsupervised image creation in LSUN bedroom.

In the LSUN bedroom experiment, IS is meaningless, so only FID is shown, showing that the old GAN-ResNet has better performance than the state-of-the-art GAN.

And Figure 6 is a graph comparing the average calculation time repeated 100 times for different GAN variants.

The yellow and red bars represent the average calculation time when the generator and discriminator update rates are 1: 1 and 1: 5, respectively.

7A and 7B are diagrams showing qualitative results of the old GAN of the LSUN-bedroom data set and the qualitative results of the old GAN of the STL-10 data set.

As can be seen from such evaluation results, the network proposed in the present invention is a hostile generation network that is a part of the generation model, which is a subfield of machine learning and computer vision, and solves the problems of the prior art through mathematical modeling and provides fast training speed. And extensibility.

Accordingly, it is possible to generate data suitable for the field through the technology presented in the present invention in the industrial field, and by having a fast execution speed, especially when used for purposes other than image data such as 3D and audio processing, excellent expansion by a simple structure Have a surname

The data processing apparatus and method using the hostile generation network on the sphere through geometric moment matching according to the present invention described above relates to a generative adversarial network, which is one of the generation models that are subfields of machine learning and computer vision. The GAN's discriminator, which distinguishes between real data and fake data, maps the final data to a sphere and maps the final data to a sphere. Including the configuration to calculate the distance (distance) is to enable stable GAN learning by using the existence of a bound.

As described above, it will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention.

Therefore, the specified embodiments should be considered from a descriptive point of view rather than a limiting point of view, and the scope of the present invention is shown in the claims rather than the above description, and all differences within the scope equivalent thereto are included in the present invention. It will have to be interpreted.

100. Generator
200. Discriminator
300. Geometric Moment Matching Unit

Claims (15)

A generator that receives a value from the random noise z, generates fake data, and transmits the generated fake data to a discriminator;
A discriminator that receives and trains real data (x) and fake data;
Receive the final data from the discriminator and map it to an n-dimensional Euclidean feature space, remap the feature points of fake data and real data to n-dimensional hyperspheres by geometric transformation, and use the mapped points. Including; a geometric moment matching unit for calculating a geometric moment centered on a central point (north pole) of a hypersphere with a spherical GAN,
The discriminator attempts to maximize the moment difference in probability measurement between real and fake data, and the generator tries to interfere with the discriminator by minimizing the moment difference and trains a hostile generating network on a sphere through geometric moment matching. Data processing device to be used. The method of claim 1, wherein the spherical GAN assumes that the feature space of the discriminant network is a sphere and minimizes the multi-order moment thereon to newly define a metric between mathematically defined measures to predict the behavior of the network to be trained. A data processing device that uses a hostile generation network on a sphere through geometric moment matching. The method of claim 1, wherein the geometric moment matching unit,
The first moment in the one-dimensional feature space is

Is defined as,
Here, G and D represent the generator and discriminator, respectively, and P and N represent the actual data and potential code distribution, respectively,
Discriminator (D) maps data (x) to real number (R),

Is defined as,
Here, D must satisfy the 1-Lipschitz condition D ∈ Lip1, and X ⊂ Rn is an n-dimensional Euclidean image space.Based on the fact that the spherical GAN is a high-dimensional and multi-dimensional hypersphere that has been expanded to a dimension larger than 3D A data processing apparatus using a hostile generation network on a sphere through geometrical moment matching, characterized in that matching the moment. The method of claim 3, wherein the output of the discriminator is

Is defined as,
Here, x is the actual image data, z is a data processing device using a hostile generation network on a sphere through geometric moment matching, characterized in that the random noise. The method of claim 4, wherein the objective function of a sphere GAN is

Is defined as,
Function for r = 1, ..., R

Represents the measurement of the r-th moment distance between each sample and the north pole of the hypersphere N, and the subscript s is

end

A data processing apparatus using a hostile generation network on a sphere through geometric moment matching, which indicates that the phase is defined. The method of claim 5, wherein the spherical GAN does not require additional constraints that allow the discriminator to be located in the desired function space,
Using geometric transformation, the spherical GAN is to ensure that the distance function is in the desired function space,
The new objective function of the discriminator

A data processing apparatus using a hostile generation network on a sphere through geometric moment matching, characterized in that it is defined as. The method of claim 6, wherein the spherical GAN is hypersphere

Matching several moments through the feature space defined in
The old GAN uses a hypersphere instead of any Riemann manifold M,
Hypersphere distance function

Is to form a bound,
The gradient norm works as a function of distance,
A data processing apparatus using a hostile generation network on a sphere through geometric moment matching, characterized in that it uses defining a GAN object through a Riemannian structure of a hypersphere. The method of claim 6, wherein the spherical GAN is a Euclidean space

Hypersphere

Use a geometric transformation function that converts to,
The conversion function is

in

A data processing device that uses a hostile generation network on a sphere through geometric moment matching, characterized in that it is designed according to diffeomorphism to be differentiated and preserves dimensions at all points in the feature space. The method of claim 8, using a stereographic projection as a geometric transformation function,
The inverse function of stereographic projection is Euclidean space

Hypersphere

Is the differential weighted morphism of

If the coordinate system of is p = (p1, ..., pn) and the north pole of the hypersphere is N = (0, ..., 1),
Inverse function of stereographic projection

silver,

Data processing apparatus using a hostile generation network on a sphere through geometric moment matching, characterized in that defined as. The method of claim 9, wherein the two points through the inverse function of stereographic projection (stereographic projection)

After projecting, the distance between the two points is measured in terms of the hypersphere metric,

Is defined as,
here,

Is

It is a distance function defined in. Geometrically

A data processing apparatus using a hostile generation network on a sphere through geometric moment matching, characterized in that is considered as a geodetic distance. The method of claim 10, wherein in order to minimize the metric IPM (Integral Probability Metrics) used in the old GAN,
Define the geometric centroid moment about the Riemannian manifold, where M is the Borel σ algebra, a small, connected, geodetically complete Riemann manifold with Σ,

Wow

Are all probabilistic measures defined in the measurable space (M, Σ), and both probabilistic measures

Wow

IPM, a measure of the distance between,

Is defined as,
F is a data processing device using a hostile generation network on a sphere through geometric moment matching, characterized in that F is a class of a function capable of measuring the boundary of an actual value with respect to M. The method of claim 11, defining the geometric moment about M,
On (M, Σ) for a given point p ₀

The r th central moment of is,

Is defined as,
here,

And

ego,

Is a Riemannian distance function in M. A data processing device using a hostile generation network on a sphere through geometric moment matching. The method of claim 12, wherein in the old GAN

Wow

If you define a new IPM between,
IPM based on the moment difference,

Is defined as,
here,

Is a class of bounded distance functions from another point at a given point p _{0. A} data processing apparatus using a hostile generation network on a sphere through geometric moment matching. The method of claim 13, wherein the function space of IPM is

Is a set of finite distance functions of the M phase centered on, and the spherical GAN parameterizes the distance function,

ego,
Where {xi} is an image set,
Parameterizing the discriminator,

ego,
here,

Data processing apparatus using a hostile generation network on a sphere through geometric moment matching, characterized in that. Generator that receives a value from random noise z, generates fake data, and transfers the generated fake data to a discriminator; Discriminator that receives and trains real data (x) and fake data; n-dimensional by receiving final data from the discriminator Mapping to the Euclidean feature space, respectively, the feature points of the fake data and the real data are re-mapped to an n-dimensional hypersphere by geometric transformation, and the center point of the hypersphere with a spherical GAN using the mapped points ( In the data processing method of a data processing device using a hostile generation network on a sphere through geometric moment matching, including; a geometric moment matching unit that calculates a geometric moment centered on a north pole),
By assuming that the feature space of the discriminant network is a sphere, using a spherical GAN that minimizes the multi-order moment on it and newly defines the metric between mathematically defined measures to predict the behavior of the network being trained,
Generating fake data similar to real by receiving a value from the random noise z in a generator;
Passing the fake data generated by the generator to the discriminator;
The discriminator receives real data and fake data, trains them, and maps them to an n-dimensional Euclidean feature space in a geometric moment matching unit;
Re-mapping the feature points of the fake data and the real data into an n-dimensional hypersphere by geometric transformation in the geometric moment matching unit;
Calculating a geometric moment about a north pole of a hypersphere in the spherical GAN by using the mapped point in the geometric moment matching unit;
The discriminator of the spherical GAN attempts to maximize the moment difference in the probability measurement between the real and the fake sample, and the generator attempts to interfere with the discriminator by minimizing the moment difference and trains it; through geometric moment matching, comprising: Data processing method using hostile generation networks on the sphere.

Images