KR102276369B1 - 3D Point Cloud Reliability Determining System and Method - Google Patents

Abstract

The present invention provides a fake 3D point cloud generated based on a fully connected layer, a 3D point cloud input unit that receives a real 3D point cloud, and the input A feature information extraction unit for extracting feature information based on statistical estimation from each of the received fake three-dimensional point cloud and the real three-dimensional point cloud, and the feature information of the fake three-dimensional point cloud and Based on statistical estimation, including a comparison performing unit that compares the feature information of the real 3D point cloud, and a reliability determination unit that determines the reliability of the received fake 3D point cloud according to the comparison result. It has the effect of being able to judge the accuracy and reliability of a more objective 3D point cloud generation technology through comparison of feature information.

Description

3D Point Cloud Reliability Determining System and Method {3D Point Cloud Reliability Determining System and Method}

The present invention relates to a technology for determining the reliability of a three-dimensional point cloud, and more particularly, to a technology for determining the accuracy and reliability of a three-dimensional point cloud generation technology more objectively through comparison of characteristic information based on statistical estimation. It relates to a system and method for determining dimensional point cloud reliability.

Recently, the problem of 3D data generation based on deep neural networks has attracted considerable attention, and image-to-point cloud, image-to-voxel, image-to-mesh, point cloud-to-voxel, and point cloud-to- It is being solved through various approaches including point cloud. On the other hand, little effort has been devoted to the development of generative adversarial networks (GANs) that can generate 3D point clouds in an unsupervised manner.

There are only two tasks regarding GANs for transforming random latent codes (i.e. z vectors) into 3D point clouds, one of which is to generate the point cloud using only fully connected layers, and the other is to generate the point cloud using only fully connected layers. It is a method using a local topology to generate geometrically accurate point clouds using the nearest neighbor technique.

However, this has a problem in that it suffers from high computational complexity as the number of dynamic graph updates increases. Also, the use of point clouds has a limitation in that it can only create a limited number of object categories (eg chairs, airplanes, sofas).

For this reason, conventionally, three methods such as mesh, point cloud, and volume were used to represent 3D objects.

However, although the mesh model has the advantage of being able to express a geometric structure, there is a problem in that different results (Difficult to implement permutation invariant in deep learning) are derived depending on the permutation of the object.

In the case of the point cloud model, it has the advantage that it can be easily acquired from raw sensor data and uses less memory, but it is difficult to express geometric relationships (fully connected layers), and post processing algorithms are used to smooth the surface. The downside is that you need this.

Volume model needs a lot of memory (High memory usage) because it has to be expressed as a set of small boxes, many parameters are required for 3D convolution, and post processing algorithms are used to smooth the surface. This is necessary, and a large number of small boxes are required, so there is a problem that more and more memory is required.

Korean Patent Publication No. 10-2050995

Accordingly, the present invention has been proposed in consideration of the above matters, and the purpose of the present invention is to more objectively determine the reliability of the 3D point cloud generation technology through comparison of feature information based on statistical estimation. do.

Another object of the present invention is to enable more specific reliability determination through comparison of feature information extracted from a network enriched by an increase in the number of parameters, that is, a fully connected layer.

Another object of the present invention is to best express the characteristics of a fake data distribution or a real data distribution using a mean vector and a covariance matrix among statistical measurements.

In addition, an object of the present invention is to accurately measure the quality of a 3D point cloud by proposing a new evaluation metric called FPD.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the three-dimensional point cloud reliability determination system according to the technical idea of the present invention includes a fake three-dimensional point cloud generated based on a fully connected layer and , 3D point cloud input unit receiving a real 3D point cloud, feature information extraction for extracting characteristic information based on statistical inference from each of the received fake 3D point cloud and real 3D point cloud unit, a comparison performing unit for performing comparison between the feature information of the fake three-dimensional point cloud and the feature information of the real three-dimensional point cloud based on the extracted feature information; It may include a reliability determination unit for determining the reliability of the point cloud.

The feature information extraction unit may extract feature information from an intermediate layer including a fully connected layer of the received fake 3D point cloud and the real 3D point cloud, respectively.

The feature information may include a mean vector and a covariance matrix of the received fake 3D point cloud and the real 3D point cloud.

The comparison performing unit may perform a comparison between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information based on a Gaussian 2-Wasserstein technique.

The comparison performing unit may perform a comparison between the characteristic information of the fake 3D point cloud and the characteristic information of the real 3D point cloud according to a distance value calculation formula between the characteristic information expressed by Equation 9 below.

<Equation 9>

: 본 발명의 FPD(Frechet Point Cloud Distance),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 평균 벡터(mean vector,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 평균 벡터(mean vector),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 공분산(covariance) 행렬,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 공분산(covariance) 행렬)(

: FPD (Frechet Point Cloud Distance) of the present invention,

: the mean vector of points calculated from the real point cloud {x};

: the mean vector of points calculated from the fake point cloud {x'},

: the covariance matrix of points calculated from the real point cloud {x},

: Covariance matrix of points calculated from fake point cloud {x'})

The reliability determining unit may determine the reliability as the distance between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information is smaller.

In order to achieve the above object, the 3D point cloud reliability determination method according to the technical idea of the present invention is a fake 3D point cloud generated based on a fully connected layer in the 3D point cloud input unit. (3D Point cloud), a 3D point cloud input step of receiving a real 3D point cloud, and statistical inference from each of the fake 3D point cloud and the real 3D point cloud received from the feature information extraction unit a feature information extraction step of extracting feature information based on estimation), a comparison in which a comparison performing unit compares the feature information of the fake three-dimensional point cloud and the feature information of the real three-dimensional point cloud based on the extracted feature information The performing step, the reliability determination unit may include a reliability determination step of determining the reliability of the received fake 3D point cloud according to the comparison result performed by the reliability determination unit.

The feature information extraction step may extract feature information from an intermediate layer including a fully connected layer of the received fake 3D point cloud and the real 3D point cloud, respectively.

The feature information may include a mean vector and a covariance matrix of the received fake 3D point cloud and the real 3D point cloud.

The comparison performing step may perform a comparison between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information based on a Gaussian 2-Wasserstein technique.

The comparison performing step may perform a comparison between the feature information of the fake three-dimensional point cloud and the feature information of the real three-dimensional point cloud according to a distance value calculation formula between the feature information expressed by Equation 9 below.

<Equation 9>

: 본 발명의 FPD(Frechet Point Cloud Distance),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 평균 벡터(mean vector,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 평균 벡터(mean vector),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 공분산(covariance) 행렬,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 공분산(covariance) 행렬)(

: FPD (Frechet Point Cloud Distance) of the present invention,

: the mean vector of points calculated from the real point cloud {x};

: the mean vector of points calculated from the fake point cloud {x'},

: the covariance matrix of points calculated from the real point cloud {x},

: Covariance matrix of points calculated from fake point cloud {x'})

In the reliability determination step, as the distance between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information is smaller, the reliability may be determined to be higher.

According to the three-dimensional point cloud reliability determination system and method as described above,

First, the present invention has the effect of more objectively determining the reliability of the 3D point cloud generation technology through comparison of feature information based on statistical estimation.

Second, the present invention has the effect that a more specific reliability determination is possible through comparison of feature information extracted from a network enriched by an increase in the number of parameters, that is, a fully connected layer.

Third, the present invention has the effect of being able to best express the characteristics of a fake data distribution or a real data distribution using a mean vector and a covariance matrix among statistical measurements.

Fourth, the present invention has the effect of accurately measuring the quality of a 3D point cloud by proposing a new evaluation metric called FPD.

1 is a block diagram showing a three-dimensional point cloud reliability determination system according to an embodiment of the present invention.
2 is a flowchart illustrating a three-dimensional point cloud reliability determination method according to an embodiment of the present invention.
3 is a diagram illustrating a pipeline of a tree-structured generative adversatial networks (GAN) according to an embodiment of the present invention;
4 is a diagram illustrating a loop term with K-supports according to an embodiment of the present invention.
Figure 5 is a view showing an ancestor term (Ancestor term) according to an embodiment of the present invention.
6 is a view showing the semantic part generation and interpolation results of a tree-GAN according to an embodiment of the present invention.
7 is a view showing the results of unsupervised 3D point cloud generation of a baseline (eg, r-GAN) and a Tree-GAN according to an embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings illustrating preferred embodiments of the present invention and the contents described in the accompanying drawings. The features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Prior to this, the terms or words used in the present specification and claims are based on the technical idea of the present invention based on the principle that the inventor can appropriately define the concept of the term in order to describe his invention in the best way. It should be interpreted with the corresponding meaning and concept. In addition, it should be noted that, when it is determined that the detailed description of known functions and configurations related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof is omitted.

1 is a block diagram illustrating a three-dimensional point cloud reliability determination system according to an embodiment of the present invention.

Referring to FIG. 1 , the three-dimensional point cloud reliability determination system according to an embodiment of the present invention is largely a three-dimensional point cloud input unit (!00), a feature information extraction unit 200, a comparison performing unit 300, and a reliability determination unit. (400).

The 3D point cloud input unit 100 may receive a fake 3D point cloud and a real 3D point cloud generated based on a fully connected layer. .

The feature information extraction unit 200 may extract feature information based on statistical estimation from each of the fake 3D point cloud and the real 3D point cloud received from the 3D point cloud input unit 100 .

At this time, the feature information extraction unit 200 receives from the intermediate layer including the fully connected layer of the fake 3D point cloud and the real 3D point cloud input from the 3D point cloud input unit 100 . Each feature information can be extracted.

The characteristic information may be referred to as characteristic information including a mean vector and a covariance matrix of the received fake 3D point cloud and the real 3D point cloud.

The comparison performing unit 300 may perform a comparison between the characteristic information of the fake 3D point cloud and the characteristic information of the real 3D point cloud based on the characteristic information extracted from the characteristic information extraction unit 200 .

The comparison performing unit 300 may perform a comparison between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information based on a Gaussian 2-Wasserstein technique.

The comparison performing unit 300 may perform a comparison between the characteristic information of the fake 3D point cloud and the characteristic information of the real 3D point cloud according to the formula for calculating the distance value between the characteristic information expressed in Equation 9 below.

<Equation 9>

: 본 발명의 FPD(Frechet Point Cloud Distance),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 평균 벡터(mean vector,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 평균 벡터(mean vector),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 공분산(covariance) 행렬,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 공분산(covariance) 행렬)(

: FPD (Frechet Point Cloud Distance) of the present invention,

: the mean vector of points calculated from the real point cloud {x};

: the mean vector of points calculated from the fake point cloud {x'},

: the covariance matrix of points calculated from the real point cloud {x},

: Covariance matrix of points calculated from fake point cloud {x'})

The reliability determining unit 400 may determine the reliability of the received fake 3D point cloud according to the comparison result performed by the comparison performing unit 300 .

The reliability determining unit 400 may determine the reliability as the distance between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information is smaller.

2 is a flowchart illustrating a method for determining reliability of a three-dimensional point cloud according to an embodiment of the present invention.

Referring to FIG. 2 , the three-dimensional point cloud reliability determination method according to an embodiment of the present invention largely includes a three-dimensional point cloud input step (S!00), a feature information extraction step (S200), a comparison execution step (S300), and reliability It may include a determination step (S400).

The 3D point cloud input step includes a fake 3D point cloud generated based on a fully connected layer in the 3D point cloud input unit 100 and a real 3D A point cloud may be input (S100).

In the feature information extraction step, the feature information extraction unit 200 extracts feature information based on statistical estimation from each of the fake three-dimensional point cloud and the real three-dimensional point cloud received from the three-dimensional point cloud input step S100. can be extracted (S200).

At this time, the feature information extraction step (S200) is an intermediate layer including the fully connected layer of the fake 3D point cloud and the real 3D point cloud received from the 3D point cloud input step (S100). From each feature information can be extracted.

The characteristic information may be referred to as characteristic information including a mean vector and a covariance matrix of the received fake 3D point cloud and the real 3D point cloud.

In the comparison performing step, the comparison performing unit 300 performs a comparison between the characteristic information of the fake 3D point cloud and the characteristic information of the real 3D point cloud based on the characteristic information extracted from the characteristic information extraction step (S200). Can be (S300).

The comparison performing step S300 may perform a comparison between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information based on a Gaussian 2-Wasserstein technique.

The comparison performing step S300 may perform a comparison between the characteristic information of the fake 3D point cloud and the characteristic information of the real 3D point cloud according to the distance value calculation formula between the characteristic information expressed by Equation 9 below.

<Equation 9>

: 본 발명의 FPD(Frechet Point Cloud Distance),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 평균 벡터(mean vector,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 평균 벡터(mean vector),

: 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 공분산(covariance) 행렬,

: 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 공분산(covariance) 행렬)(

: FPD (Frechet Point Cloud Distance) of the present invention,

: the mean vector of points calculated from the real point cloud {x};

: the mean vector of points calculated from the fake point cloud {x'},

: the covariance matrix of points calculated from the real point cloud {x},

: Covariance matrix of points calculated from fake point cloud {x'})

In the reliability determination step, the reliability determination unit 400 may determine the reliability of the received fake 3D point cloud according to the comparison result performed from the comparison performing step S300 ( S400 ).

In the reliability determination step ( S400 ), the smaller the distance value between the fake 3D point cloud characteristic information and the real 3D point cloud characteristic information, the higher the reliability may be determined.

3 is a diagram illustrating a pipeline of a tree-structured generative adversarial networks (GAN) according to an embodiment of the present invention, and FIG. 4 is a diagram showing a loop term with K-supports according to an embodiment of the present invention , and FIG. 5 is a diagram showing an ancestor term according to an embodiment of the present invention, and FIG. 6 is a diagram showing a semantic part generation and interpolation result of a tree-GAN according to an embodiment of the present invention , and FIG. 7 is a diagram illustrating a result of unsupervised 3D point cloud generation of a baseline (eg, r-GAN) and a Tree-GAN according to an embodiment of the present invention.

Referring first to FIG. 3 , a tree-structured generative adversarial networks (GANs) according to an embodiment of the present invention may include two networks. Here, the two networks can be referred to as a generator and a discriminator.

로부터 입력 데이터로서 하나의 포인트를 입력받을 수 있다. 생성기(Generator)의 각 레이어에서 그래프 합성곱(GraphConv)과 분기(Branching)는 포인트

의 I번째 세트를 생성할 수 있다. 이전 레이어들에 의해 생성된 모든 포인트들은 현재 레이어의 트리로 저장되고, 어펜드(append)될 수 있다. 해당 트리는 노드(node) z부터 시작하고, 분기 작업을 통해 하위노드인 자식노드(Child node)로 분할하며, 그래프 합성곱(GraphConv) 작업에 의해 노드를 수정할 수 있다. 생성기(Generator)는 출력 데이터로서 3D 포인트 클라우드

를 생성할 수 있다. 여기서

은 마지막 레이어 L에서 포인트들의 세트이고, n은 포인트들의 전체 개수라 할 수 있다.Generator is a Gaussian distribution

One point may be input as input data from . In each layer of the Generator, GraphConv and Branching are points

can create the I-th set of All points generated by previous layers can be stored and appended to the tree of the current layer. The tree starts from node z, and is divided into child nodes that are sub-nodes through branching operations, and nodes can be modified by graph convolution (GraphConv) operations. Generator is a 3D point cloud as output data

can create here

is the set of points in the last layer L, and n may be the total number of points.

A discriminator can distinguish between a real point cloud and a generated fake point cloud so that the generator generates a more realistic point. Here, Fig. 3 can be said to use a discriminator similar to that in the r-GAN.

(1) 3D point cloud GAN

3 is a diagram illustrating a pipeline of a tree-structured generative adversarial networks (GAN) according to an embodiment of the present invention. In order to generate a 3D point cloud x' from the latent code z, the objective function of Wasserstein GAN can be used in the present invention.

은 아래 수학식 2로 정의할 수 있다.Loss function of Generator

can be defined by Equation 2 below.

<Equation 2>

은 잠재 코드(latent code) 분포를 나타낼 수 있다. 본 발명에서는 정규 분포(Normal distribution)

를 가진

을 설계할 수 있다. 구분자(Discriminator)의 손실 함수(Loss function)

은 아래 수학식 3으로 정의할 수 있다.Here, G and D may represent a generator and a discriminator, respectively,

may represent a latent code distribution. In the present invention, the normal distribution (Normal distribution)

with

can be designed. Loss function of Discriminator

can be defined by Equation 3 below.

<Equation 3>

은 리얼(Real) 포인트 클라우드와 페이크(Fake) 포인트 클라우드 사이의 선분으로부터 샘플링될 수 있고,

와 x은 각각 생성된 페이크(fake) 포인트 클라우드와, 리얼(real) 포인트 클라우드를 나타낼 수 있으며,

은 리얼(real) 데이터 분포를 나타낼 수 있다. 상기 수학식 3은 1-립시츠 조건(1-Lipschitz condition)을 만족하는 gradient penalty를 사용한 것이라 할 수 있다.

은 가중계수(weighting parameter)라 할 수 있다.here,

can be sampled from a line segment between a real point cloud and a fake point cloud,

and x may represent a generated fake point cloud and a real point cloud, respectively,

may represent a real data distribution. Equation 3 can be said to use a gradient penalty that satisfies the 1-Lipschitz condition.

can be referred to as a weighting parameter.

(2) Tree structure GCN proposed in the present invention

In order to implement the generator of Equation 2, the multilayer graph convolution introduced in the present invention can be considered as Equation 4 below.

<Equation 4>

은 활성 유닛(activation unit),

은 l번째 레이어의 그래프(즉, 포인트 클라우드의 3D 좌표)에서 i번째 노드,

은

의 j번째 이웃이며,

은

의 모든 이웃들의 세트라 할 수 있다. 그런 다음, 상기 수학식 3에서 z 및 x’은 각각

및

에 의해 표현될 수 있다. 여기서, L은 마지막 레이어이며, n은 L에서 포인트들의 개수라 할 수 있다.here,

is an activation unit,

is the i-th node in the graph of the l-th layer (i.e. the 3D coordinates of the point cloud),

silver

is the j-th neighbor of

silver

is the set of all neighbors of . Then, in Equation 3, z and x' are each

and

can be expressed by Here, L is the last layer, and n is the number of points in L.

및

, 베스트 바이어스

을 찾을 수 있다. 그런 다음, 포인트 클라우드에 대한 3D 생성은 리얼(Real) 포인트 클라우드의 유사성을 보장하기 위해 상기 파라미터를 사용하여 조정할 수 있다. 상기 수학식 3에서 첫 번째 텀(term)과 두 번째 텀(term)은 각각 루프(loop)와 이웃(neighbors)이라 할 수 있다.During the learning process, GCNs are best weighted at each layer.

and

, best bias

can be found Then, the 3D generation for the point cloud can be adjusted using the above parameters to ensure the similarity of the real point cloud. In Equation 3, the first term and the second term may be referred to as loops and neighbors, respectively.

In order to improve the general GCN, the GCN of the present invention can be said to be a new GCN increased to a tree structure (ie, TreeGCN). The TreeGCN of the present invention can be said to introduce a tree structure for hierarchical GCNs by transferring information from ancestors (Ancestors) to descendants (Descendants) of vertices. The most unique feature of the TreeGCN of the present invention is that each vertex refers to the value of the ancestors in the tree instead of the vertices of the neighbor, so that the value of the corresponding ancestor can be updated.

General GCNs can be considered as a method of referencing only neighbors at a single depth, as in Equation 4 defined above. The graph convolution proposed in the present invention can be defined by Equation 1 below.

<Equation 1>

: l+1번째 레이어의 그래프(즉, 포인트 클라우드의 3D 좌표)에서 i번째 노드,

: l번째 레이어의 그래프(즉, 포인트 클라우드의 3D 좌표)에서 i번째 노드, K: support의 갯수,

: 활성함수(activation function),

: 기하학적 관계 특성,

: 계층적 특성,

: j번째 조상 노드,

:

노드의 가능한 모든 조상 노드들의 집합,

:

노드를 위한 wight matrix,

: l번째 레이어의 모든 노드들을 위한 bias term)(

: i-th node in the graph of the l+1th layer (that is, the 3D coordinates of the point cloud),

: i-th node in the graph of the l-th layer (ie, the 3D coordinates of the point cloud), K: the number of supports,

: activation function,

: geometric relationship characteristics,

: hierarchical characteristics,

: the jth ancestor node,

:

the set of all possible ancestors of a node,

:

weight matrix for nodes,

: bias term for all nodes of the lth layer)

There are two main differences in Equation 1 that are distinguished from Equation 4 above.

을 사용하여 일반적인 루프텀을 향상시키는 것이다. 여기서,

은

의 K supports에 의해 생성되는 것이라 할 수 있다. 이 때

은 K-supports을 가진 루프텀이라 할 수 있다. The first difference is the subnetwork

is used to improve the general loop term. here,

silver

It can be said that it is created by the K supports of . At this time

is a loop term with K-supports.

은

의 모든 조상(Ancestors)의 세트를 나타낸 것이라 할 수 있다. 이를 조상 텀(term ancestors)라 할 수 있다.The second difference is the consideration of values received from all Ancestors in the tree to update the value of the current point. here,

silver

It can be said to represent the set of all the Ancestors of These can be referred to as term ancestors.

(2-1) Graph convolution of the present invention (Advanced Graph convolution)

The tree-structured graph convolution (ie, GraphConv) proposed in the present invention can aim to correct the adjustment of points by using a loop with K-supports and ancestor terms.

부터

까지 매핑을 학습하기 위해 상기 수학식 4의 단일 파라미터

을 사용할 수 있다. 도 4의 오른쪽(Right)에 해당하는 본 발명의 루프텀은

부터

까지 더욱 복잡한 매핑을 학습하기 위한 것으로, K 노드(즉,

)을 가진 완전연결층(Fully connected layer)라 할 수 있다.Loop term with K-supports: FIG. 4 is a diagram illustrating a loop term with K-supports according to an embodiment of the present invention. A typical loop term corresponding to the left of FIG. 4 is

from

single parameter of Equation 4 above to learn the mapping up to

can be used The loop term of the present invention corresponding to the right of FIG. 4 is

from

For learning more complex mappings up to K nodes (i.e.,

) can be called a fully connected layer.

만을 사용하는 대신에 K-supports을 기초로 다음 포인트를 제안하는 것이라 할 수 있다. 이는 아래 수학식 5로 표현할 수 있다.When the Loop term with K-supports is described with reference to FIG. 4 , the target of the new loop term in Equation 1 is a single parameter as shown in Equation 4 above.

It can be said that the next point is suggested based on K-supports instead of using only the K-supports. This can be expressed by Equation 5 below.

<Equation 5>

은 K 노드들을 포함하는 완전연결층(Fully connected layer)이라 할 수 있다. 1차 근사치를 사용하는 기존의 GCN은 현재 포인트에서 다음 포인트를 생성하기 위해, 루프텀에서 단일 파라미터를 채택할 수 있다.here,

can be called a fully connected layer including K nodes. Existing GCN using first-order approximation can adopt a single parameter in the loop term to generate the next point from the current point.

However, the representative capacity of a single parameter for a large graph may be insufficient to express a complex point distribution. Therefore, the loop term of the present invention can utilize K supports to express a more complex distribution of points.

을 생성하기 위해

의 이웃(neighbor)을 사용할 수 있다. 도 5의 오른쪽(Right)에 해당하는 본 발명에서 생성된 조상 텀(Ancestor term)은

을 생성하기 위해

의 조상(Ancestor)을 사용할 수 있다.Ancestor term: Figure 5 can be said to be a diagram showing an ancestor term (Ancestor term) according to an embodiment of the present invention. A general neighbor term corresponding to the left of FIG. 5 is

to create

can be used as a neighbor of Ancestor term generated in the present invention corresponding to the right (Right) of Figure 5 is

to create

You can use the Ancestor of .

When the Ancestor term is described with reference to FIG. 5 , it can be said that knowing the connectivity of the graph for graph convolution is very important because the corresponding information allows the GCN to propagate useful information from vertices to other connected vertices. However, in the point cloud generation environment of the present invention, it is impossible to use prior knowledge about connection because various types of point clouds must be created in the present invention.

을 통해 조상(Ancestor)

으로부터 모든 정보를 결합할 수 있다. 그 이유는 각 조상(Ancestor)이 다른 레이어의 다른 특징공간에 속하는데, 이 때 본 발명의 조상 텀(Ancestor term)은 이전 레이어들과 다른 특징공간으로부터 모든 정보를 융합할 수 있기 때문이다. 다음 포인트를 생성하기 위해, 현재 포인트는 조상(ancestor) 정보를 효과적으로 결합할 수 있는 최상의 매핑

을 찾기 위해 다양한 특징공간에서 상기 현재 포인트에 해당하는 조상(ancestors)을 참조할 수 있다. 이 새로운 조상 텀(ancestor term)을 사용함으로써, 본 발명의 tree-GAN은 하기 (3)에서 설명한 바와 같이 몇 가지 바람직한 수학적 특성(Mathematical Properties)을 얻을 수 있다. 도 5는 조상 텀(ancestor term)을 사용한 그래프 합성곱 과정을 보여주는 도면이라 할 수 있다.Therefore, the problem of dynamic 3D point generation cannot be addressed using existing GCNs because these networks assume that the connection of the graph is given. As an alternative to the neighboring term in Equation 4, in the present invention, an ancestor may be defined in the second term with respect to Equation 1 above. This second term is a linear mapping

Ancestors through

You can combine all information from The reason is that each ancestor belongs to a different feature space of a different layer, and in this case, the Ancestor term of the present invention can fuse all information from the feature space different from the previous layers. To create the next point, the current point is the best mapping that can effectively combine ancestor information.

Ancestors corresponding to the current point may be referred to in various feature spaces to find . By using this new ancestor term, the tree-GAN of the present invention can obtain some desirable mathematical properties as described in (3) below. 5 is a diagram showing a graph convolution process using an ancestor term.

(2-2) Branching

은 단일 포인트

를 자식노드

로 변형시킨 것이라 할 수 있다. 여기서,

이며, 아래 수학식 6을 참조할 수 있다.Branching can be said to be a process for increasing the total number of points, which can be said to be similar to up-sampling in 2D convolution. in the branch,

is a single point

is a child node

It can be said that it has been transformed into here,

, and can be referred to Equation 6 below.

<Equation 6>

은 행렬 A의 j번째 열(Column)을 나타낼 수 있다. 그런 다음, (l+1)번째 레이어에서 포인트들의 전체 개수는

라 할 수 있다. 여기서,

은 l번째 레이어에서 포인트들의 개수라 할 수 있다.here,

may indicate the j-th column of the matrix A. Then, the total number of points in the (l+1)th layer is

can be said here,

is the number of points in the l-th layer.

)에 대한 다른 분기점을 사용할 수 있다.In an embodiment of the present invention, the present invention provides another layer (eg,

) can be used for other branching points.

라 할 수 있다.The number of points in the final layer is

can be said

(3) Mathematical Properties

On the other hand, it is possible to mathematically analyze the geometrical relationship between the points generated through the embodiment of the present invention, and prove how this relationship is formed in the output Euclidean space through tree-structured graph convolution.

및

은 각각 상기 수학식 1에서 생성된 포인트로서, 마지막 레이어 L에서

을 가진 동일한 부모(parents)와 다른 부모(parents)를 공유하는 포인트라 할 수 있다.

은 상기 수학식 5에서 l번째 레이어에서 포인트

의 루프라 할 수 있다. 이후, 본 발명에서는 첨자 l이 마지막 레이어 L를 나타낸다면,

및

의 첨자를 생략할 수 있다.Proposition 1:

and

are points generated in Equation 1, respectively, in the last layer L

It can be said to be a point that shares the same parents with different parents.

is a point in the l-th layer in Equation 5

It can be said that the loop of Then, in the present invention, if the subscript l represents the last layer L,

and

can be omitted.

<Equation 7>

<Equation 8>

은 l번째 레이어에서 포인트 p의 모든 조상(ancestors)라 할 수 있다. 단순하게, 본 발명의 실시예에서는 상기 수학식 5에서 분기(Branch) 과정 및

을 무시할 수 있다.here,

can be said to be all ancestors of point p in the lth layer. Simply, in the embodiment of the present invention, the branch process and

can be ignored.

Based on Proposition 1, the embodiment of the present invention can prove that the following ① and ② are true.

및

이 다른 조상(ancestors)을 가진다면, 해당 두 개의 포인트들 사이의 기하학적 거리는 각 레이어 I (즉, 상기 수학식 8에서

)에서 해당 두 개의 포인트들 조상들(ancestors) 사이와, 이의 루프들 (즉, 상기 수학식 8에서

) 사이의 차이의 합으로 계산될 수 있다. 따라서, 해당 두 개의 포인트들 조상들(ancestors)이 점점 달라짐에 따라, 해당 두 개의 포인트들 사이의 기하학적 거리는 증가할 수 있다.① The geometric distance between two points can be determined by the number of shared ancestors. : If two points

and

If we have these different ancestors, the geometric distance between the two points is each layer I (i.e., in Equation 8 above)

) between the corresponding two points ancestors and its loops (that is, in Equation 8 above

) can be calculated as the sum of the differences between Accordingly, as the ancestors of the two points gradually change, the geometric distance between the two points may increase.

및

은 같은 조상들(ancestors)을 공유한다면, 해당 두 개의 포인트들 사이의 기하학적 거리는 상기 수학식 7과 같이, 해당 포인트들의 루프들에 의해서만 영향을 받을 수 있다. 따라서, 상기 수학식 7에서 같은 조상들(ancestors)을 가진 두 개의 포인트들 사이의 기하학적 거리는,

=0 와 같은 상기 수학식 8에서 다른 조상들(ancestors)을 가진 포인트들 사이와 비교한다면 감소할 수 있다.② Geometrically related points can share the same ancestors. : If two points

and

If n shares the same ancestors, the geometric distance between the two points can be affected only by the loops of the corresponding points as in Equation 7 above. Therefore, in Equation 7 above, the geometric distance between two points with the same ancestors is,

It can be reduced if compared between points with different ancestors in Equation 8, such as =0.

Based on the characteristics of ① and ②, the tree-GAN of the present invention can generate a semantic part of an object as shown in FIG. 6 . Here, it can be assumed that points with the same ancestors belong to a part of the same object. In an embodiment of the present invention, a partial generation problem for the Tree-GAN can be confirmed in (5-1) below.

At this time, FIG. 6 can be said to be a diagram showing the semantic part generation and interpolation results of the tree-GAN according to an embodiment of the present invention. More specifically, the red and blue point clouds in FIG. 6 may be generated from different ancestors of the tree forming families of geometrically different points. The leftmost and rightmost point clouds of the planes can be generated from different noise inputs. Intermediate planes are obtained by interpolating between the leftmost and rightmost point clouds based on the latent spatial representation.

(4) Frechet Point Cloud Distance

In an embodiment of the present invention, for quantitative comparison between GANs, an evaluation index capable of accurately measuring the quality of the 3D point cloud generated by the GAN may be required. In the case of 2D data generation problems, FID is the most common metric. FID can utilize the functional space for evaluation by adopting a pre-trained inception V3 model. Although a traditional metric can be used to evaluate the quality of the generated points by directly measuring the coincidence distance between the real point cloud and the generated fake point cloud, the goal of the GAN is similar to the sample ( It can be considered as sub-optimal metrics because it is not generating an actual synthetic probability measure as close as possible to the actual synthetic probability measure, e.g. MMD or CD. This perspective can be said to have been explored in the work of generating unsupervised 2D images (or videos) using GAN. Accordingly, in an embodiment of the present invention, a new evaluation metric for generating a 3D point cloud called FPD may be proposed. Here, FPD (Frechet Point Cloud Distance) means a distance between a fake 3D point cloud and a real 3D point cloud characteristic information.

Similar to FID, the FPD proposed in the present invention can calculate a 2-Wasserstein distance between a real Gaussian measurement value and a fake Gaussian measurement value in the feature space as shown in Equation 9 below.

<Equation 9>

및

은 각각 리얼(real) 포인트 클라우드 {x}로부터 계산된 포인트들의 평균 벡터(mean vector)와 공분산(covariance) 행렬이라 할 수 있다. 그리고,

및

은 각각 생성된 페이크(fake) 포인트 클라우드 {x’}로부터 계산된 포인트들의 평균 벡터(mean vector)와 공분산(covariance) 행렬이라 할 수 있다. 여기서,

및

라 할 수 있다. 상기 수학식 9에서, Tr(A)은 행렬 A의 메인 대각선(diagonal)을 따른 원소의 합계라 할 수 있다. 본 (4)에서의 목적을 위해, 본 발명의 실시예에서는 합성곱 평가 지표들과, 제안된 FPD 모두 사용할 수 있다.here,

and

may be referred to as a mean vector and a covariance matrix of points calculated from a real point cloud {x}, respectively. And,

and

may be referred to as a mean vector and a covariance matrix of points calculated from each generated fake point cloud {x'}. here,

and

can be said In Equation 9, Tr(A) may be the sum of elements along the main diagonal of the matrix A. For the purpose of this (4), both the convolution evaluation indices and the proposed FPD can be used in the embodiment of the present invention.

(5) Experimental Results

인 학습속도와 기타 계수

=0 및

을 가진 생성기(Generator)와 구분자(Discriminator) 네트워크 모두에 Adam Optimizer를 사용하였다. 생성기(Generator)에서는 배치 정상화(Batch normalization)가 없는 비선형 함수로서 LeakyReLU을 사용할 수 있다. 구분자(Discriminator)의 네트워크 아키텍처(architecture)는 r-GAN의 아키텍처와 동일하였다. 경사도형 패널티 계수(gradient penalty coefficient)는 10으로 설정되었고, 구분자(Discriminator)는 반복 당 5회로 업데이트되었으며, 생성기(Generator)는 반복 당 1회로 업데이트되었다. 도 3에 도시된 바와 같이, 잠재 벡터(latent vector)

은 정상 분포

로부터 샘플링되어 입력 역할을 하였다. 7개의 레이어들 (L=7)은 TreeGCN을 위해 사용되었다. 상기 수학식 5에서 TreeGCN의 루프텀은 K=10 supports을 가지고 있었다. 마지막 레이어에서 포인트들의 전체 개수는 n=2048로 설정되었다.Implementation details: In an embodiment of the present invention,

learning rate and other coefficients

=0 and

Adam Optimizer was used for both generator and discriminator networks with The Generator can use LeakyReLU as a non-linear function without batch normalization. The network architecture of Discriminator was the same as that of r-GAN. The gradient penalty coefficient was set to 10, the Discriminator updated 5 times per iteration, and the Generator updated once per iteration. As shown in Figure 3, the latent vector (latent vector)

is the normal distribution

was sampled from and served as input. Seven layers (L=7) were used for TreeGCN. In Equation 5, the loop term of TreeGCN had K=10 supports. The total number of points in the last layer was set to n=2048.

Comparison: There are only two existing GANs for creating 3D point clouds: r-GAN and GAN. Therefore, the tree-GAN proposed in the present invention was compared with the two existing GANs. Existing GANs train a separate network for each class, whereas the Tree-GAN proposed in the present invention trains only a single network.

Evaluation metrics: As an embodiment in the present invention, tree-GAN can be evaluated using ShapeNet, which is a large-scale data set of 3D shapes. At this time, the evaluation was performed in terms of the FPD and matrix proposed in the present invention. As a reference model for FPD, the classification module of PointNet can be used in the present invention because it can process partial input of objects. This property is suitable for FPD because the generated point cloud gradually takes shape, which may mean that the point cloud can be partially completed while learning. For the implementation of FPD, in the present invention, the classification module of 40 generations (epochs) may be first learned in order to achieve 98% accuracy of the classification task. Then, in order to calculate the mean and covariance in Equation 9, a 1808-dimensional feature vector may be extracted from the output of the density layer.

(5-1) Ablation Study

It is possible to analyze the Tree-GAN proposed in the present invention, and examine its useful properties, namely, unsupervised semantic part generation through interpolation, latent spatial expression, branching, and the like.

Unsupervised semantic part generation: The Tree-GAN of the present invention can generate point clouds for other semantic parts without prior knowledge about the corresponding part during the learning process. The Tree-GAN of the present invention can perform this semantic generation thanks to tree-structured graph convolution, which is a unique feature in the GAN-based 3D point cloud method. As mentioned in Proposition 1 above, the geometric distance between points may be determined by ancestors of the corresponding tree. Different ancestors may refer to families of geometrically different points.

은 잠재 코드(latent code)의 샘플로 추출될 수 있다.

=

및

은 생성된 포인트 클라우드에 상응하는 것이라 할 수 있다.

은 2048 포인트 색인(indices)의 특정 부분집합이라 할 수 있다. 그런 다음,

색인(indices)

에서

의 서브셋을 나타낼 수 있다. 도 6에 도시된 바와 같이, 만약 본 발명에서 포인트 클라우드의 색인(indices)의 동일한 서브셋을 선택한다면, 이는 비록 잠재 코드(latent code) 입력이 서로 다르다 하더라도 같은 의미적(semantic) 부분의 결과가 도출될 수 있다.Thus, by selecting different ancestors, the tree-GAN of the present invention can generate semantically different parts of the point cloud. It should be noted here that the geometric families of the points are the same between different latent code inputs. For example,

can be extracted as a sample of the latent code.

=

and

can be said to correspond to the generated point cloud.

can be said to be a specific subset of the 2048 point indices. after that,

indices

in

can represent a subset of As shown in FIG. 6, if the same subset of indices of the point cloud is selected in the present invention, the same semantic result is derived even if the latent code inputs are different. can be

(예를 들어,

및

)에 의해 인덱스된 모든 붉은색 포인트들은 비행기의 조종석(cockpits)을 나타내는 반면,

(예를 들어,

및

)에 의해 인덱스된 모든 파란색 포인트들이 비행기의 꼬리를 나타낼 수 있다. 이러한 ablation study로부터, 본 발명에서는 조상(ancestors) 간의 차이가 포인트들 사이의 의미적(semantic) 차이를 결정한다는 것과, 동일한 조상(ancestor)(예를 들어, 도 9에서 왼쪽 날개를 나타내는 녹색 및 보라색 포인트들)을 가진 두 개의 포인트가 서로 다른 잠재 코드(latent codes)에 대한 상대적 거리를 유지한다는 것을 확인할 수 있다.For example,

(For example,

and

All red points indexed by ) represent the cockpits of the plane, while

(For example,

and

), all blue points indexed by ) may represent the tail of the airplane. From this ablation study, in the present invention, the difference between ancestors (ancestors) determines the semantic difference between points, and the same ancestor (eg, green and purple representing the left wing in FIG. points) maintain relative distances for different latent codes.

에 기초하여, 입력 잠재 코드(latent code)를

로 설정하여 3D 포인트 클라우드를 보간(interpolate)할 수 있다. 도 6에서의 비행기의 가장 왼쪽 및 가장 오른쪽 포인트 클라우드는 각각

및

에 의해 생성될 수 있다. 또한, 본 발명의 tree-GAN은 두 개의 포인트 클라우드 간의 실제 보간(interpolations)을 생성할 수 있다.Interpolation: In the present invention, six alphas

Based on the input latent code

can be set to interpolate the 3D point cloud. The leftmost and rightmost point clouds of the plane in FIG. 6 are respectively

and

can be created by In addition, the tree-GAN of the present invention can generate actual interpolations between two point clouds.

,

). Branching strategy: In an embodiment of the present invention, an experiment may be conducted to show that the convergence dynamics of the proposed metric are not sensitive to different branching strategies. As in other experiments, the total number of generated points is 2048, but branching degrees can be set differently. (For example,

,

).

(5-2) Comparisons with Other GANs

The tree-GAN proposed in the present invention can be quantitatively and qualitatively compared with other state-of-the-art GANs for point cloud generation in terms of both accuracy and computational efficiency.

Comparisons: Tables 1 and 2 below may include quantitative comparisons in terms of metrics by FPD proposed in the present invention, respectively (i.e., JSD, MMD-CD, MMD-EMD, COV-CD and COV). -Etc).

<Table 1>

<Table 2>

At this time, Table 1 is a table showing quantitative comparison in terms of metrics used in Achlioptas. In more detail, the red value and the blue value may represent best results and second-best results, respectively.

Table 2 is a table showing a quantitative comparison in terms of the FPD proposed in the present invention. More specifically, the FPD proposed in the present invention relates to a real point cloud, which can be said to be close to zero. This value can serve as the lower bound of the generated point cloud.

The tree-GAN proposed in the present invention continuously shows a larger margin than other GANs in terms of all metrics, showing the effect of the tree-GCN proposed in the present invention.

For qualitative comparison, in the present invention, the set overall index is divided into four subsets, and points are drawn with the same color in each subset. 3 to 7, although real 3D point clouds are not ordered, the tree-GAN of the present invention successfully generates 3D point clouds with intuitive semantics without any prior knowledge. can do. However, r-GAN failed to generate semantically ordered point clouds. Additionally, the tree-GAN of the present invention could generate detailed and complex parts of an object, whereas the r-GAN could generate a more distributed point distribution.

7 is a diagram illustrating a result of unsupervised 3D point cloud generation of a baseline (eg, r-GAN) and a Tree-GAN according to an embodiment of the present invention. In more detail, FIG. 7 shows that the tree-GAN proposed in the present invention generates a point cloud of more accurate and detailed objects compared to the r-GAN, and points for each part of the object even if there is no prior knowledge of the corresponding part. You can create a cloud. The point cloud generated by this tree-GAN can represent different geometric types for each class. The first, second, and third columns can show the ground truth, baseline and tree-GAN respectively.

7 can show the qualitative result of the tree-GAN of the present invention. This Tree-GAN has created realistic point clouds for multi-object categories, and can create very different types of point clouds for each class.

Computational cost: In the method of using static links for graph convolution, adjacency matrices can be used for the convolution of vertices in general. These methods are known to yield good results on graph data, but require prior knowledge about connection. In another method using dynamic links for graph convolution, adjacency matrices have to be constructed from vertices to derive connectivity information for all convolutional layers instead of using prior knowledge. do.

은 각각 l번째 레이어에서 출력 그래프의 레이어의 개수, 배치 크기 및 유도 정점(vertex) 크기를 나타낼 수 있다. 이와 같이 설명한 방법에는 연결 정보를 활용하기 위한 추가적인 계산이 필요하다. 이러한 연산에는

의 순서에 따라 시간과 메모리 리소스가 필요하다. 다만, 본 발명의 TreeGCN은 정적 링크(static link) 방식과 같은 사전 연결 정보를 요구하지 않으며, 동적 링크(dynamic link) 방식과 같은 추가적인 연산을 요구하지 않는다. 따라서, 본 발명의 네트워크는 시간과 메모리 리소스를 훨씬 더 효율적으로 사용할 수 있고,

의 순서에 따라 더 적은 자원을 필요로 할 수 있다.For example,

may represent the number of layers of the output graph, the batch size, and the derived vertex size in the l-th layer, respectively. The method described above requires additional calculations to utilize the connection information. In these calculations

time and memory resources are required according to the order of However, the TreeGCN of the present invention does not require pre-linking information such as a static link method, and does not require an additional operation such as a dynamic link method. Thus, the network of the present invention can use time and memory resources much more efficiently,

may require fewer resources depending on the order of

(6) Conclusion

In the present invention, we can propose a generational adversarial network called tree-GAN that can generate 3D point clouds in an unsupervised manner. The proposed generator for Tree-GAN, called tree-GCN, can preform a graph convolution based on the tree structure. This tree-GCN utilizes the tree's ancestor information and can use several supports to represent the 3D point cloud.

Therefore, the tree-GAN proposed in the present invention can outperform other GAN-based point cloud generation methods in terms of accuracy and computational efficiency. Through various experiments according to an embodiment of the present invention, tree-GAN can generate a semantic part of an object without any prior knowledge, and can represent a 3D point cloud in a latent space through interpolation. can prove that there is

Although described and illustrated in connection with a preferred embodiment for illustrating the technical idea of the present invention above, the present invention is not limited to the configuration and operation as shown and described as such, and deviates from the scope of the technical idea. It will be apparent to those skilled in the art that many changes and modifications can be made to the invention without reference to the invention. Accordingly, all such suitable changes and modifications are to be considered as falling within the scope of the present invention.

100: 3D point cloud input unit
200: feature information extraction unit
300: comparison execution unit
400: reliability determination unit

Claims (12)

a 3D point cloud input unit that receives a fake 3D point cloud generated based on a fully connected layer and a real 3D point cloud;
a feature information extraction unit for extracting feature information based on statistical estimation from each of the received fake three-dimensional point cloud and real three-dimensional point cloud;
a comparison performing unit that compares the feature information of the fake three-dimensional point cloud and the feature information of the real three-dimensional point cloud based on the extracted feature information; and
and a reliability determination unit that determines the reliability of the received fake 3D point cloud according to the comparison result performed;
The 3D point cloud reliability determination system, characterized in that the fully connected layer is based on a 3D point cloud based on a tree structure neural network. According to claim 1, wherein the feature information extraction unit,
A three-dimensional point cloud reliability determination system that extracts feature information from each of the received fake three-dimensional point cloud and the density layer including the fully connected layer of the real three-dimensional point cloud. According to claim 1, wherein the characteristic information,
A three-dimensional point cloud reliability determination system that is characteristic information including a mean vector and a covariance matrix of the received fake three-dimensional point cloud and the real three-dimensional point cloud. The method of claim 1, wherein the comparison performing unit,
A three-dimensional point cloud reliability determination system that compares the fake three-dimensional point cloud feature information and the real three-dimensional point cloud feature information based on a Gaussian 2-Wasserstein technique. The method of claim 4, wherein the comparison performing unit,
A three-dimensional point cloud reliability determination system that compares the feature information of the fake three-dimensional point cloud and the feature information of the real three-dimensional point cloud according to a distance value calculation formula between the feature information expressed by Equation 9 below.
<Equation 9>

(

: FPD (Frechet Point Cloud Distance) proposed in the present invention,

: the mean vector of points calculated from the real point cloud {x};

: the mean vector of points calculated from the fake point cloud {x'},

: the covariance matrix of points calculated from the real point cloud {x},

: Covariance matrix of points calculated from fake point cloud {x'}) The method of claim 5, wherein the reliability determining unit,
A three-dimensional point cloud reliability determination system for judging reliability as the distance value between the fake three-dimensional point cloud feature information and the feature information of the real three-dimensional point cloud is smaller. A 3D point cloud input that receives a fake 3D point cloud generated based on a fully connected layer and a real 3D point cloud from the 3D point cloud input unit step;
a feature information extraction step of extracting feature information based on statistical estimation from each of the received fake three-dimensional point cloud and the real three-dimensional point cloud by the feature information extraction unit;
a comparison performing step of performing a comparison between the feature information of the fake three-dimensional point cloud and the feature information of the real three-dimensional point cloud based on the extracted feature information in the comparison performing unit; and
a reliability determination step of determining the reliability of the received fake 3D point cloud according to the comparison result performed by the reliability determination unit;
The 3D point cloud reliability determination method, characterized in that the fully connected layer is based on a 3D point cloud based on a tree-structured neural network. The method of claim 7, wherein the extracting of the feature information comprises:
A 3D point cloud reliability determination method for extracting feature information from each of the received fake 3D point clouds and density layers including fully connected layers of the real 3D point cloud. The method of claim 7, wherein the characteristic information,
A three-dimensional point cloud reliability determination method which is feature information including a mean vector and a covariance matrix of the received fake three-dimensional point cloud and the real three-dimensional point cloud. The method according to claim 7, wherein the comparing step comprises:
A 3D point cloud reliability determination method for comparing the fake 3D point cloud feature information and the real 3D point cloud feature information based on a Gaussian 2-Wasserstein technique. 11. The method of claim 10, wherein the performing the comparison step,
A three-dimensional point cloud reliability determination method for performing a comparison between the feature information of the fake three-dimensional point cloud and the feature information of the real three-dimensional point cloud according to a distance value calculation formula between the feature information expressed by Equation 9 below.
<Equation 9>

(

: FPD (Frechet Point Cloud Distance) proposed in the present invention,

: the mean vector of points calculated from the real point cloud {x};

: the mean vector of points calculated from the fake point cloud {x'},

: the covariance matrix of points calculated from the real point cloud {x},

: Covariance matrix of points calculated from fake point cloud {x'}) The method of claim 11, wherein the reliability determination step comprises:
A three-dimensional point cloud reliability determination method for judging reliability as the distance value between the fake three-dimensional point cloud feature information and the real three-dimensional point cloud feature information is smaller.

Images