KR20190054899A - Method and apparatus for processing a plurlity of nondirected graphs - Google Patents

Abstract

A method and an apparatus for processing multiple undirected graphs are provided. Lattices for each of multiple undirected graphs representing feature on edges and nodes of the undirected graph are generated and matched, thereby matching the multiple undirected graphs and processing the multiple matched undirected graphs.

Description

[0001] METHOD AND APPARATUS FOR PROCESSING A PLURITY OF NONDIRECTED GRAPHS [0002]

The following embodiments relate to a method and apparatus for processing a plurality of non-directional graphs, and more particularly to a method and apparatus for processing a plurality of non-directional graphs by generating lattices corresponding to each of a plurality of non-directional graphs, And more particularly,

A graph that has no direction of edges between nodes (or vertices) in a graph composed of a plurality of nodes is called an undirected graph. The non-directional graph model is widely used in various fields, and the non-directional graph can represent various relations. For example, a technique of matching a plurality of non-directional graphs can be used for image synthesis techniques, posture analysis techniques for objects in an image, and the like.

One embodiment may provide an apparatus and method for processing a plurality of non-directed graphs.

One embodiment may provide an apparatus and method for matching a plurality of non-directed graphs.

According to one aspect, a method for processing a plurality of non-directional graphs includes the steps of obtaining a first undirected graph and a second non-directional graph, Generating a first lattice for the first non-directional graph and a second lattice for the second non-directional graph, the coordinates being indicative of characteristics of the nodes and edges, Matching the first lattice and the second lattice based on a first global structure of a first lattice and a second global structure of the second lattice, And wherein the second global structure corresponds to nodes of the second non-directional graph, and wherein the first non-directional graph and the second non-directional graph based on the matched first lattice and the second lattice, And processing the graph.

Wherein generating the first lattice and the second lattice comprises generating the first lattice for the first non-directional graph through nonmetric multidimensional scaling (NMDS), and generating the first lattice through the NMDS And generating each of the second lattices for the second non-directional graph.

Wherein generating the first lattice through the NMDS further comprises generating randomly a target lattice based on the natures of the first non-directional graph and the features of the edges of the first non- And generating the first lattice by optimizing the lattice.

Wherein the step of generating the first lattice by optimizing the target lattice comprises: calculating a distance between a first point and a second point in the target lattice, the first point and the second point being the first non- Characterized by using a monotone function that does not include a parameter to define a feature of an edge for the first node and the second node to the first point and the second node of the second node, Calculating a Kruskal's Stress-1 for the target lattice, and calculating the Kruskal's Stress-1 for the target lattice when the calculated Kruskal's pressure-1 converges, And determining the first lattice as the first lattice.

The monotone function is determined by minimizing [Equation 1] below,

[Equation 1]

는 상기 단조함수이고,

는 상기 제1 노드 및 상기 제2 노드에 대한 엣지의 특징이고,

는 상기 제1 포인트 및 상기 제2 포인트 간의 거리이고, i 및 j는 상기 제1 무방향 그래프의 노드들의 총 개수인 N 이내의 숫자일 수 있다.In Equation (1)

Is the above monotonic function,

Is characteristic of an edge for the first node and the second node,

Is a distance between the first point and the second point, and i and j may be a number within N, which is the total number of nodes of the first non-directional graph.

The crusher pressure-1 can be calculated by the following equation (2).

&Quot; (2) "

The matching of the first lattice and the second lattice may include matching the first lattice and the second lattice through PR-GLS (Point Set Registration by Preserving Global and Local Structures) algorithm .

Wherein obtaining the first non-directional graph and the second non-directional graph includes generating the first non-directional graph from the first image and generating the second non-directional graph from the second image .

The processing of the first non-directional graph and the second non-directional graph may comprise compositing the first image and the second image based on the matched first lattice and the second lattice have.

Wherein the first non-directional graph and the second non-directional graph relate to the same object, and wherein processing the first non-directional graph and the second non-directional graph includes analyzing the attitude of the object .

According to another aspect, an apparatus for processing a plurality of non-directional graphs includes a memory in which a program for processing a plurality of non-directional graphs is recorded, and a processor for executing the program, Obtaining an undirected graph and a second non-directional graph based on the first non-directional graph and the second non-directional graph; Generating a first lattice for the first lattice and a second lattice for the second non-directional graph based on the first global structure of the first lattice and the second global structure of the second lattice; Matching the first lattice and the second lattice, wherein the first global structure corresponds to nodes of the first non-directional graph, and the second global structure corresponds to an upper You may perform the step of processing the first non-directed graph, and the second non-directed graph on the basis of, and the matching of the first lattice and the second lattice-second non-corresponding to the nodes of the directed graph.

Wherein generating the first lattice and the second lattice comprises generating the first lattice for the first non-directional graph through nonmetric multidimensional scaling (NMDS), and generating the first lattice through the NMDS And generating each of the second lattices for the second non-directional graph.

The matching of the first lattice and the second lattice may include matching the first lattice and the second lattice through PR-GLS (Point Set Registration by Preserving Global and Local Structures) algorithm .

Wherein obtaining the first non-directional graph and the second non-directional graph includes generating the first non-directional graph from the first image and generating the second non-directional graph from the second image .

The processing of the first non-directional graph and the second non-directional graph may comprise compositing the first image and the second image based on the matched first lattice and the second lattice have.

Wherein the first non-directional graph and the second non-directional graph relate to the same object, and wherein processing the first non-directional graph and the second non-directional graph includes analyzing the attitude of the object .

1 is an image including a repeated pattern according to an example.
FIG. 2 is an image of a scene with little change in contrast according to an example.
3 is a configuration diagram of a non-directional graph processing apparatus according to an embodiment.
4 is a flow diagram of a method for processing a plurality of non-directional graphs according to an embodiment.
5 is a flowchart of a method for generating a first non-directional graph and a second non-directional graph according to an example.
6 is a flow diagram of a method for generating a first lattice and a second lattice through an NMDS according to an example.
7 is a flow diagram of a method for generating a first lattice based on a target lattice according to an example.
8 is a flow diagram of a method for generating a first lattice by optimizing target lattice according to an example.
9 is a flowchart of a method for processing a first non-directional graph and a second non-directional graph according to an example.
Each of Figs. 10 to 12 shows the matched first non-directional graph and the second non-directional graph according to an example.
13 is a flowchart of a method of matching a first non-directional graph and a second non-directional graph according to an example.
FIG. 14 is a result of matching CMU house and CMU hotel, which are public data sets, through various algorithms according to an example.
FIG. 15 is a result of matching Pscal 2007, which is an open data set, through various algorithms according to an example.

In the following, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

Various modifications may be made to the embodiments described below. It is to be understood that the embodiments described below are not intended to limit the embodiments, but include all modifications, equivalents, and alternatives to them.

The terms used in the examples are used only to illustrate specific embodiments and are not intended to limit the embodiments. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this embodiment belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. In the following description of the embodiments, a detailed description of related arts will be omitted if it is determined that the gist of the embodiments may be unnecessarily blurred.

FIG. 1 is an image including a repeated pattern according to an example, and FIG. 2 is an image for a scene having a small change in contrast according to an example.

Matching non-directional graphs affects the efficiency and accuracy of the results. According to an aspect of the present invention, for a scene having a small gray scale change such as an image including a repeated pattern such as the image shown in FIG. 1 and an image shown in FIG. 2, A matching algorithm (e.g., Scale-invariant feature transform (SIFT)) can not be used. For the image of FIG. 1 and the image of FIG. 2, a matching algorithm using structure information between nodes should be used.

The quadratic assignment problem is a model of operations according to an example, and finding the answer to the above problem is a difficult task. One way to match non-directional graphs is to represent the problem of matching non-directional graphs as a quadratic assignment problem and to obtain an approximate solution to the quadratic assignment problem. For example, a non-directional graph can be viewed as a graph model G, and a graph model G includes a node V and an edge E having no direction. The node V of the non-directional graph has a corresponding feature F and the edge E has a corresponding corresponding feature W. The problem of matching two non-directional graphs can be transformed into a problem of matching the nodes and edges of each non-directional graph. The second assignment problem is to maximize the similarity of features matching the nodes and edges of two non-directional graphs through optimization of a target function.

<Optimization of secondary assignment problem>

Optimization of the secondary assignment problem for matching between the template non-directed graph and the scene non-directed graph is described below.

로 표현된다.

는 템플릿 무방향 그래프를 구성하는 M개의 노드들의 집합이고,

는 M개의 노드들 중 임의의 두 개의 노드들 사이의 엣지들의 집합이고,

는 노드(

)의 특징(

)들의 집합이고,

는 엣지(

)의 특징(

)들의 집합니다. 예를 들어, 노드의 특징은 형태 컨텍스트(形上下文)이고, 엣지의 특징은 엣지의 길이일 수 있다.Template non-directional graph

Lt; / RTI &gt;

Is a set of M nodes constituting a template non-directional graph,

Is a set of edges between any two of the M nodes,

Is a node (

Features of

), &Lt; / RTI &gt;

Edge

Features of

) Houses. For example, a feature of a node may be a shape context, and the feature of an edge may be the length of an edge.

로 표현된다. 장면 무방향 그래프는 N개의 노드들의 집합일 수 있다.Similar to the template non-directional graph, the scene non-

Lt; / RTI &gt; The scene non-directional graph may be a set of N nodes.

에 추가될 수 있다. According to one aspect, N may be greater than or equal to M. In this case, a node (external node) that does not correspond to the node of the template non-directional graph among the nodes of the scene non-directional graph occurs. This external node makes the matching algorithm unstable and reduces the accuracy of the matching result. Thus, in order to improve the robustness of the external nodes of the scene direction graph in the matching process,

Lt; / RTI &gt;

와 장면 무방향 그래프의

사이의 대응관계를 최적화한다. 일반적으로, 매칭 알고리즘은 아래의 2차 할당 문제를 최적화한다.The matching algorithm is based on the

And Scene of a Directional Graph

Lt; / RTI &gt; In general, the matching algorithm optimizes the following quadratic assignment problem.

는 템플릿 무향 그래프의 제i 번째 노드가 장면 무향 그래프의 제j 번째 노드와 매칭되는 것을 의미한다. 1은 모든 원소가 1인 열벡터이다.

는 매트릭스 K의 제((i-1)N+j)행 제((k-1)N+l)열의 원소이고, 특징

와 특징

사이의 유사하지 않은 정도를 의미한다.

는 0과 1사이의 값이고, 특징의 종류 또는 성질에 따라

가 나타내는 값의 형식도 달라진다. 예를 들어,

는 0 또는 1인 이산된 값을 가질 수 있다. 다른 예로,

는 0과 1사이의 실수의 값을 가질 수 있다.

는 2차 할당 문제가 뒤에 열거한 조건을 만족해야 함을 나타낸다.In Equation (1), x is a vector obtained by combining the column vectors of the matrix X,

Means that the i-th node of the template omnipresent graph is matched with the j-th node of the scene de-orientation graph. 1 is a column vector with all elements 1.

(K-1) N + 1) columns of the matrix K, and the characteristic

And features

And the like.

Is a value between 0 and 1, depending on the type or nature of the feature

The format of the value represented by E.g,

Can have a discrete value of zero or one. As another example,

Can have a real number value between 0 and 1.

Indicates that the secondary assignment problem must satisfy the conditions listed below.

Since it is a difficult task to find a solution to the secondary allocation problem, there is no algorithm that can find the optimal solution of the objective function in a short time. The existing algorithm used is a similar solution to the second order assignment problem, but the accuracy of the matching result through these existing algorithms is relatively low, and the complexity with respect to existing algorithms is relatively high. Also, for matching situations for non-directional graphs with significant noise, contamination of external points, and matching situations for large non-directional graphs, existing algorithms are difficult to obtain the optimal solution of the objective function.

A method for efficiently matching a plurality of non-directional graphs and processing a plurality of matched non-directional graphs will now be described in detail with reference to Figs. 3 to 15 below.

3 is a configuration diagram of a non-directional graph processing apparatus according to an embodiment.

The non-directional graph processing apparatus 300 includes a communication unit 310, a processor 320, and a memory 330. For example, the non-directional graph processing device 300 is an electronic device, and the electronic device may be a server, a personal computer (PC), a mobile device, or the like, and is not limited to the described embodiments.

The communication unit 310 is connected to the processor 320 and the memory 330 to transmit and receive data. The communication unit 310 may be connected to another external device to transmit / receive data.

The communication unit 310 may be implemented as a circuitry in the non-directional graph processing device 300. For example, the communication unit 310 may include an internal bus and an external bus. As another example, the communication unit 310 may be an element that connects the non-directional graph processing device 300 and an external device. The communication unit 310 may be an interface. The communication unit 310 can receive data from an external device and transmit data to the processor 320 and the memory 330. [

The processor 320 processes the data received by the communication unit 310 and the data stored in the memory 330. [ &Quot; Processor " may be a data processing device embodied in hardware having circuitry having a physical structure for performing desired operations. For example, the desired actions may include code or instructions included in the program. For example, a data processing apparatus embodied in hardware may be a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, , An application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA).

Processor 320 executes computer readable code (e.g., software) stored in a memory (e.g., memory 330) and instructions initiated by processor 320.

The memory 330 stores data received by the communication unit 310 and data processed by the processor 320. [ For example, the memory 330 may store a program. The stored program may be a set of syntaxes executable by the processor 320 that are coded to process a plurality of non-directional graphs.

According to an aspect, the memory 330 may include one or more volatile memory, non-volatile memory and random access memory (RAM), flash memory, hard disk drive, and optical disk drive.

The memory 330 stores a set of instructions (e.g., software) for operating the non-directional graph processing device 300. The instruction set that operates the non-directional graph processing device 300 is executed by the processor 320. [

The communication unit 310, the processor 320, and the memory 330 will be described in detail with reference to Figs. 4 to 13 below.

4 is a flow diagram of a method for processing a plurality of non-directional graphs according to an embodiment.

The non-directional graph processing apparatus 300 described above with reference to FIG. 3 performs the following steps 410 to 440. A plurality of non-directional graphs may be matched through steps 410 to 440, and matched non-directional graphs may be processed. The non-directional graph can be represented by lattices, and the matching problem of a plurality of non-directional graphs can be converted into a problem of matching a plurality of lattices. When the matching problem of a plurality of non-directional graphs is converted into a matching problem of a plurality of lattices, since the optimal solution is not required for the secondary allocation problem, the complexity of the calculation is reduced and the accuracy of the result is improved.

In step 410, the non-directional graph processing device 300 acquires the first non-directional graph and the second non-directional graph. The first non-directional graph may be the template non-directional graph described above, and the second non-directional graph may be the above-described scene non-directional graph.

For example, the first non-directional graph and the second non-directional graph may be generated from the first image and the second image, respectively. A method of acquiring a non-directional graph will be described in detail below with reference to FIG.

In step 420, the non-directional graph processing device 300 generates a first lattice for the first non-directional graph and a second lattice for the second non-directional graph. The lattice represents the features of the nodes and edges of the non-directional graph in coordinates.

에 대한 제1 래티스는

로 표현된다.

는 제1 래티스 내의 포인트(

)들의 집합이고, 포인트(

)는 제1 무방향 그래프의 노드(

)에 대응한다. 포인트(

)는 노드(

)의 특징(

)과 서로 연관이 있다.

는 제1 래티스 내의 포인트들 사이의 엣지의 특징들의 집합이다. 제1 래티스 내의 포인트들(

,

) 사이의 엣지의 특징은 포인트들(

,

) 사이의 거리로 표현된다. 유사하게, 제2 무방향 그래프의

에 대한 제2 래티스는

로 표현된다.In the first non-directional graph

Lt; RTI ID = 0.0 &gt; lattice

Lt; / RTI &gt;

Lt; RTI ID = 0.0 &gt; lattice

), And a point (

) Is a node of the first non-directional graph (

). point(

) Is a node

Features of

).

Is a set of features of the edge between points in the first lattice. The points in the first lattice (

,

Is characterized by the fact that the points &lt; RTI ID = 0.0 &gt;

,

). &Lt; / RTI &gt; Similarly, the second non-directional graph

The second lattice for

Lt; / RTI &gt;

A method of generating the first lattice and the second lattice will be described in detail below with reference to FIGS.

일 수 있다.In step 430, the non-directional graph processing device 300 matches the first lattice and the second lattice based on the first lattice and the second lattice. For example, the non-directional graph processing device 300 may match the first lattice and the second lattice based on a first global structure of the first lattice and a second global structure of the second lattice. For example, the first global structure of the first lattice may be

Lt; / RTI &gt;

)는 가우시안 혼합 모델(Gaussian Mixture Model)로 표현될 수 있다. 가우시안 혼합 모델에서, 가우시안 성분(分量)의 중심은

이다.

는

의 변형할 수 있다. 변형은 제1 래티스의 회전 및 제1 래티스의 거울대칭일 수 있다.According to one aspect, a first global structure (

) Can be expressed by a Gaussian Mixture Model. In the Gaussian mixture model, the center of the Gaussian component (quantity)

to be.

The

. The deformation may be the rotation of the first lattice and the mirror symmetry of the first lattice.

)는 제1 글로벌 구조(

)의 가우시안 혼합 모델로부터 샘플링된 것으로 간주된다. 가우시안 혼합 모델의 모든 가우시안 성분에 등방성(isotropy)의 공분산(convariance)

이 있다고 가정하면, 제1 글로벌 구조(

)의 가우시안 혼합 모델을 샘플링함으로써 제2 래티스의 제2 글로벌 구조(

)를 획득할 확률은 아래의 [수학식 3]과 같다.The second global structure of the second lattice (

) Is the first global structure (

) &Lt; / RTI &gt; The isotropic covariance of all the Gaussian components of the Gaussian mixture model,

, The first global structure (

) By sampling the Gaussian blend model of the second lattice

) Is obtained by Equation (3) below.

은 최적화가 필요한 파라미터이고,

는 외부 포인트의 백분율이다. 외부 포인트의 분포는 균일분포이고, 확률밀도함수가

라 가정한다. N은 제1 래티스의 포인트들의 개수이다.

는 매트릭스의 프로베니우스 놈(Frobenius norm)이다. [수학식 2]가 나타내는 것은 기존의 모델을 이용하여

를 획득할 수 있는 확률이고, 우측의 첫번째 항이 나타내는 것은

의 포인트가 외부 포인트일 확률이고, 우측의 두번째 항이 나타내는 것은 가우시안 혼합 모델로부터

를 샘플링할 확률이고,

는 가우시안 성분의 구성원 확률이고,

를 만족한다.In Equation 2,

Is a parameter that needs optimization,

Is the percentage of external points. The distribution of external points is a uniform distribution, and the probability density function is

. N is the number of points of the first lattice.

Is the Frobenius norm of the Matrix. [Equation (2)] indicates that using the existing model

, And the first term on the right side

Is the probability of an external point, and the second term on the right represents the probability from the Gaussian mixture model

, &Lt; / RTI &gt;

Is the member probability of the Gaussian component,

.

를 결정한다.The local characteristic is expressed by Equation (2)

.

및

의 로컬 특징이 서로 다른 정도를 나타내는 행렬이고, 매트릭스 A의 제i행 제j열의 원소

는 특징

및

사이에 유사하지 않은 정도를 나타낸다고 가정한다. 매트릭스 A에 기초하여 하나의 할당 문제의 최적해를 구하는 것을 통하여 두 개의 래티스들

및

이 매칭될 수 있다.The matrix A consists of two lattices

And

Is the matrix indicating the degrees of local characteristics of the matrix A,

Features

And

And &lt; RTI ID = 0.0 &gt; a &lt; / RTI &gt; By finding the optimal solution of one assignment problem based on the matrix A, two lattices

And

Can be matched.

는 제1 래티스 내의 포인트

와 제2 래티스 내의 포인트

의 매칭(또는 일치함)을 나타낸다. 할당 문제는 헝가리 알고리즘을 통해 시간 복잡도 레벨 내에서

에 대해 글로벌 최적해를 구할 수 있다. In Equation (3)

Lt; RTI ID = 0.0 &gt; lattice

And points in the second lattice

(Or &lt; / RTI &gt; matching). The assignment problem is solved by the Hungarian algorithm within the time complexity level

The global optimal solution can be obtained.

가 주어진 경우, 아래의 [수학식 4]에 따라

를 설정한다.

Given by the following equation (4)

.

은

이고,

의 값은 사용자에 의해 설정될 수 있다.

의 값은 매칭 결과에 대한 사용자의 신뢰를 반영한다. 예를 들어, 사용자가 [수학식 4]의 매칭 결과가 비교적 정확하다고 생각하면

의 값을 1에 근접하게 설정할 수 있고, 그렇지 않으면

의 값을 0에 가깝게 설정할 수 있다. 제2 래티스의 포인트

와 제1 래티스의 포인트

가 매칭되면,

를 설정한다.

silver

ego,

May be set by the user.

&Lt; / RTI &gt; reflects the user's confidence in the matching result. For example, if the user thinks that the matching result of [Equation 4] is relatively correct

May be set to be close to 1, otherwise

Can be set close to zero. Point of the second lattice

And the point of the first lattice

Is matched,

.

에 대한 사전 확률을 이용하여 획득한 변형

을 보다 부드럽게 만든다. 변형

에 대한 사전 확률 밀도 함수는 아래의 [수학식 5]로 나타난다.The right side of Equation (2) represents the global structure and local characteristics. The PR-GLS (Point Set Registration by Preserving Global and Local Structures)

The variance obtained using the prior probability for

. transform

Is expressed by the following equation (5). &Quot; (5) &quot;

는 정수이고, 예를 들어, 3으로 설정될 수 있다.

는 하나의 변형 에 대한 정규화 함수(regularization function)이고, 형식은

이다.

는 하나의 벡터의 reproducing kernel Hilbert 공간이다. reproducing kernel Hilbert 공간은 하나의 대각 가우시안 매트릭스 핵함수

로 정의된다.

는 가우시안 핵함수의 표준 차이이고, 예를 들어, 2로 설정될 수 있다.In Equation (5)

Is an integer, and may be set to 3, for example.

One variant Is a regularization function for &lt; RTI ID = 0.0 &gt;

to be.

Is the reproducing kernel Hilbert space of one vector. The reconstruction kernel Hilbert space is a diagonal Gaussian matrix nuclear function

.

Is the standard difference of the Gaussian nuclear function, and can be set, for example, to 2.

의 최대 사후확률(maximum posterior probability: MAP)은 아래의 [수학식 6]과 같다.Combining Equations (5) and (2) minimizes the minus logarithm posterior probability,

The maximum posterior probability (MAP) of the MAP is expressed by Equation (6) below.

에 대한 최적해는 아래의 [수학식 7]로 나타낼 수 있다.In Equation (6), it is assumed that the sampling points are independent from each other. The above problem is solved using an expectation conditional maximization (ECM) method. transform

Can be expressed by the following equation (7). &Quot; (7) &quot;

은 매트릭스 C의 제n 번째 열벡터이다. 매트릭스 C는 아래의 [수학식 8]로 결정된다.In Equation (7)

Is the n-th column vector of the matrix C. The matrix C is determined by the following equation (8).

의 제i행 제j열은

이고,

는 새로운 대각선 매트릭스를 생성하기 위해 입력 매트릭스의 대각선 요소를 취하는 것이다.

는 사후확률 매트릭스이고, 제i행 제j열의 원소는 아래의 [수학식 9]와 같다.In Equation (8)

The first column of the ith row of

ego,

Is to take the diagonal elements of the input matrix to create a new diagonal matrix.

Is the posterior probability matrix, and the elements of the i &lt; th &gt; row and j &lt; th &gt;

가 예측될 수 있고, [수학식 9]에서 정의한 사후확률이 계산될 수 있다.A variant that optimally matches the first and second lattices through the PR-GLS algorithm

Can be predicted, and the posterior probability defined in equation (9) can be calculated.

이 비교적 높으면 제1 래티스 및 제2 래티스에서 대응하는 두개의 포인트들(

및

)은 서로 매칭된다. 예를 들어, 복수의 제2 래티스의 포인트들이 하나의 제1 래티스의 포인트에 매칭될 수 있고, 그 반대의 경우일 수도 있다.Posterior probability

Is relatively high, the first lattice and the second lattice correspond to two points (

And

Are matched with each other. For example, points of a plurality of second lattices can be matched to points of a first lattice, and vice versa.

와 제2 래티스

이 포인트의 형상 컨텍스트를 로컬 특징으로 사용하여 설명될 수 있다.

테스트 통계(test statistics)를 이용하여, 불일치 매트릭스

가 계산된다. 불일치 매트릭스

의 원소

는 포인트

와 포인트

에 대한 형상 컨텍스트의 비 유사성을 나타낸다. 변형된 제1 래티스

와 제2 래티스

의 1대1 대응관계는 아래의 [수학식 10]의 할당 문제를 최소화함으로써 얻을 수 있다.It is possible to solve the allocation problem such that the matching between the first lattice and the second lattice is one to one. To this end, the modified first lattice

And the second lattice

Can be described using the shape context of this point as a local feature.

Using test statistics, the discrepancy matrix &lt; RTI ID = 0.0 &gt;

Is calculated. Mismatch matrix

Element of

Point

And point

And the like. The modified first lattice

And the second lattice

1 can be obtained by minimizing the assignment problem expressed by the following equation (10).

은 포인트

와 포인트

의 매칭을 나타낸다. 대응하게,

이면 포인트

와 포인트

는 서로 매칭된다.In Equation (10)

Point

And point

Lt; / RTI &gt; Correspondingly,

If the point

And point

Are matched with each other.

&Lt; Concurrent processing of algorithms >

및 제2 래티스

사이의 초기 정렬에 의존하는 것을 알 수 있다. 래티스들의 초기 정렬은

및

의 좌표에 의해 결정된다.

및

의 좌표는 회전하거나 반전할 수 있다.Combining Equations (2) and (5), the solution to Equation (10)

And a second lattice

Lt; RTI ID = 0.0 &gt; alignment. &Lt; / RTI &gt; The initial alignment of the lattices

And

As shown in FIG.

And

Can be rotated or reversed.

에 대한 반전 래티스

의 해가 먼저 구해진다. 그 후에, 아래의 [수학식 11]에 따라

및

를 회전시킨다.In order to achieve a higher matching accuracy, a plurality of PR-GLS algorithms corresponding to a plurality of modified first lattices may process a plurality of modified first lattices in parallel. Each of the PR-GLS algorithms are in different initial alignment states and produce different matching results. For example, the first lattice

Lattice for Inversion

The solution is first obtained. Thereafter, according to the following equation (11)

And

.

는 [0, 360] 사이에 균일하게 분포된 각도이다. 이에 따라, 총 2h 개의 변형 래티스

가 획득된다. 각각의 변형 래티스

가 제2 래티스와 매칭되고, 각각의 매칭 결과

가 획득된다. 매칭 결과들 중 가장 좋은 매칭 결과가 아래의 [수학식 12]를 통해 선택될 수 있다. 복수의 변형 래티스들이 동시에 처리될 수 있으므로 시간이 절약된다.In Equation (11)

Is an angle uniformly distributed between [0, 360]. Thus, a total of 2h deformed lattices

Is obtained. Each modified lattice

Is matched with the second lattice, and each matching result

Is obtained. The best matching result among the matching results can be selected through Equation (12) below. Time can be saved since a plurality of modified lattices can be processed simultaneously.

In step 440, the non-directional graph processing device 300 processes the first non-directional graph and the second non-directional graph based on the matched first lattice and the second lattice. For example, when a first non-directed graph is generated from a first image and a second non-directed graph is generated from a second image, a first image and a second image are generated based on the matched first and second lattices, Can be synthesized. As another example, if the first non-directional graph and the second non-directional graph are about the same object, the attitude of the object can be analyzed. Various processes other than the described embodiments are possible.

5 is a flowchart of a method for generating a first non-directional graph and a second non-directional graph according to an example.

The following step 510 may be performed before step 410 described above with reference to FIG. 4 is performed. Step 510 includes steps 511 and 512, and steps 511 and 512 may be performed in parallel.

In step 511, the non-directional graph processing device 300 generates a first non-directional graph from the first image. For example, feature points of the first image may be extracted, and nodes of the first non-directional graph may be generated based on the extracted feature points.

In step 512, the non-directional graph processing device 300 generates a second non-directional graph from the second image. For example, the first image and the second image may be images taken from different viewpoints of the same scene. In another example, the first image and the second image may each include the same object with different attitude or position.

6 is a flow diagram of a method for generating a first lattice and a second lattice through an NMDS according to an example.

Step 420, described above with reference to FIG. 4, includes the following steps 610 and 620.

In step 610, the non-directional graph processing device 300 generates a first lattice for the first non-directional graph through nonmetric multidimensional scaling (NMDS). NMDS can be applied well for non-directional graphs. Since the NMDS has few assumptions about the data and the applicable range is relatively wide, the NMDS can allow any method to be used to quantify the dissimilarity between the nodes. A method of generating the first lattice using the NMDS will be described in detail below with reference to FIGS. 7 and 8. FIG.

In step 620, the non-directional graph processing device 300 generates a second lattice for the second non-directional graph through the NMDS.

7 is a flow diagram of a method for generating a first lattice based on a target lattice according to an example.

Step 610 described above with reference to FIG. 6 includes the following steps 710 and 720.

를 생성할 수 있다.In step 710, the non-directional graph processing device 300 arbitrarily generates a target lattice. For example, the non-directional graph processing device 300 may use the NMDS to transform the target lattice

Lt; / RTI &gt;

In step 720, the non-directional graph processing device 300 generates the first lattice by optimizing the target lattice based on the characteristics of the nodes and edges of the first non-directional graph. A method for optimizing the target lattice will be described in detail below with reference to FIG.

8 is a flow diagram of a method for generating a first lattice by optimizing target lattice according to an example.

Step 720 described above with reference to FIG. 7 includes the following steps 810-850.

및 제2 포인트

간의 거리

를 계산한다. 제1 포인트 및 제2 포인트는 타겟 래티스 내의 임의의 두 개의 포인트들일 수 있다. 타겟 래티스를 최적화함으로써 제1 래티스가 생성되므로, 타겟 래티스에 대한 표기법(notation)은 제1 래티스에 대한 표기법과 동일하다.In step 810, the non-directional graph processing device 300 receives the first point &lt; RTI ID = 0.0 &gt;

And the second point

Distance between

. The first point and the second point may be any two points in the target lattice. Since the first lattice is generated by optimizing the target lattice, the notation for the target lattice is the same as the notation for the first lattice.

를 이용하여 제1 무방향 그래프의 제1 노드

및 제2 노드

에 대한 엣지의 특징

을 제1 포인트

및 제2 포인트

간의 거리

로 변환한다. 제1 무방향 그래프의 제1 노드

및 제2 노드

는 타겟 래티스 내의 제1 포인트

및 제2 포인트

와 각각 대응한다. 예를 들어, 단조함수

는 파라미터를 포함하지 않고,

를 최소화함으로써 획득될 수 있다.In step 820, the non-directional graph processing device 300 generates a monotone function

Of the first non-directional graph,

And the second node

Edge Features

The first point

And the second point

Distance between

. The first node of the first non-directional graph

And the second node

0.0 &gt; lattice &lt; / RTI &gt;

And the second point

Respectively. For example,

Does not include parameters,

. &Lt; / RTI &gt;

In step 830, the non-directional graph processing device 300 calculates a Kruskal's stress-1 for the target lattice. The non-directional graph processing apparatus 300 minimizes the Kruskal's stress-1 expressed by the following equation (13).

In step 840, the non-directional graph processing apparatus 300 determines whether or not the calculated crosstalk pressure-1 converges. For example, the non-directional graph processing device 300 calculates the difference between the currently calculated crosstalk pressure-1 and the previously calculated crosstalk pressure-1, and if the calculated difference is within a preset threshold value It can be determined that the calculated crosbachal pressure-1 has converged.

If crosstalk pressure-1 has not converged, steps 810 through 830 may be performed again.

In step 850, the non-directional graph processing device 300 determines the target lattice as the first lattice when the calculated crosstalk pressure-1 converges.

Although a method of generating a first lattice through steps 810 through 850 has been described, the description of steps 810 through 850 may also be described as a method of generating a second lattice.

9 is a flowchart of a method for processing a first non-directional graph and a second non-directional graph according to an example.

Step 440, described above with reference to FIG. 4, includes the following steps 910 and 920. Depending on the embodiment, steps 910 and 920 may optionally be performed.

In step 910, the non-directional graph processing device 300 synthesizes the first image and the second image based on the matched first lattice and the second lattice. The matching of the first lattice and the second lattice means the matching of the first non-directional graph and the second non-directional graph, and the matching of the first non-directional graph and the second non-directional graph is performed between the first image and the second image Means matching between the same or corresponding parts. The non-directional graph processing device 300 can generate a synthesized image using the portion of the first image and the portion of the second image that are matched. For example, the synthesized image may be a panoramic image. As another example, the synthesized image may be a medical image.

In step 920, the non-directional graph processing device 300 analyzes the position or position of the same object in the first image and the second image based on the matched first lattice and the second lattice. For example, the first lattice may relate to an object in a first image, and the second lattice may relate to an object in a second image. The matching of the first lattice and the second lattice may be a matching of an object in the first image and an object in the second image.

According to one aspect, the analysis of an object's posture or position can be applied to autonomous navigation techniques. The posture or the position of the object included in the images can be analyzed based on the images photographed by the autonomous vehicle. For example, if posture data of an object of a first image temporally preceding is known in advance, posture data of an object of a second image temporally following can be predicted.

Each of Figs. 10 to 12 shows the matched first non-directional graph and the second non-directional graph according to an example.

10 and 11 show the first images 1010 and 1110 and the second images 1020 and 1120, respectively, which are different in photographing time. The first images 1010 and 1110 and the second images 1020 and 1120 include the same object. A first non-directional graph and a second non-directional graph are generated using the nodes extracted from the object, and the first non-directional graph and the second non-directional graph are matched using the above-described method.

12 shows a first image 1210 and a second image 1220 that include objects having different attitudes. The object 1211 in the first image 1210 and the object 1221 in the second image 1220 are the same object but different in attitude. The object 1211 and the object 1221 can be matched using the non-directional graph for the object 1211 and the non-directional graph for the object 1221 even when the attitude of the object is changed.

13 is a flowchart of a method of matching a first non-directional graph and a second non-directional graph according to an example.

)가 획득되고(1301), 제2 영상으로부터 제2 무방향 그래프(

)가 획득된다(1302).From the first image to the first non-directed graph (

Is obtained 1301, and a second non-directional graph

Is obtained (1302).

) 및 제2 무방향 그래프(

)에 NMDS를 적용함(1310)으로써 제1 래티스(I^t)를 생성하고(1311), 제2 래티스(I^s)가 생성된다(1312).The first non-directional graph (

) And the second non-directional graph (

The first lattice I ^t is generated 1311 and the second lattice I ^s is generated 1312 by applying 1310 the NMDS to the first lattice I ^t .

First lattice (I ^t), the modified by the modification of the first lattice are generated (1321).

) 및 로컬 특징(F^t)이 계산되고(1322), 제2 래티스에 대한 글로벌 구조(

) 및 로컬 특징(F^s)이 계산된다(1323). 계산된 제1 래티스에 대한 글로벌 구조(

) 및 제2 래티스에 대한 글로벌 구조(

)에 기초하여 제1 래티스 및 제2 래티스가 매칭된다(1324).Global Structure for Modified First Lattice (

) And the local feature F ^t are calculated 1322 and the global structure for the second lattice

) And the local feature F ^s are calculated 1323. The global structure for the calculated first lattice (

) And the global structure for the second lattice (

The first lattice and the second lattice are matched (1324).

The step 1320 including the steps 1321, 1322, 1323, and 1324 is repeatedly performed so that the first lattice and the second lattice that are variously modified are matched.

A plurality of matching results may be calculated 1330, and the best matching result among the calculated plurality of matching results is selected 1340.

<Algorithm Time Complexity>

이고, N은 제1 무방향 그래프의 노드 개수이다. 구체적으로, NMDS는

의 시간 복잡도를 통하여 하나의 무방향 그래프를 하나의 래티스로 전환한다. PR-GLS 알고리즘은

의 시간 복잡도를 통해 두 개의 래티스들을 매칭한다. 두 개의 무방향 그래프들이 주어진 경우, PR-GLS 알고리즘을

번 실행함으로써 최적의 매칭 정확도가 획득될 수 있다. h는 정확도와 효율 사이의 평형을 제어한다. PR-GLS 알고리즘은 병렬적으로 실행될 수 있다.The time complexity of the above-described non-directional graph matching method is

And N is the number of nodes of the first non-directional graph. Specifically, NMDS

One non-directional graph is transformed into one lattice through the time complexity of. The PR-GLS algorithm

Lt; RTI ID = 0.0 &gt; lattices. &Lt; / RTI &gt; Given two non-directional graphs, the PR-GLS algorithm

An optimal matching accuracy can be obtained. h controls the balance between accuracy and efficiency. The PR-GLS algorithm can be executed in parallel.

이고, [수학식 1]에 대한 계산을 요구한다. 기존의 무방향 그래프 매칭 방법과 비교하면 도 3 내지 도 13을 참조하여 설명된 무방향 그래프 매칭 방법의 시간 복잡도가 비교적 낮다.On the other hand, the execution complexity of the conventional non-directional graph matching method is generally

, And requires calculation for [Equation 1]. Compared with the conventional non-directional graph matching method, the time complexity of the non-directional graph matching method described with reference to FIGS. 3 to 13 is relatively low.

FIG. 14 is a result of matching CMU house and CMU hotel, which are public data sets, through various algorithms according to an example.

Sequential images of CMU (Carnegie Mellon University) house and CMU hotel, which are publicly available data sets, are the most popular experimental data for testing algorithms. The two sequential images are 111 and 110 width images, respectively, and each width graph has 30 feature nodes. [Table 1] below shows the accuracy of the non-directional graph matching method described with reference to Figs. 3 to 13.

Degree of deformation 10 20 30 40 50 60 70 80 90 100 matching
accuracy(%) 100 100 100 100 100 100 99.9 99.7 99.7 100

(RGWM) Reweighted Random Walks for Graph Matching, SM (spectrum matching), IPFP-U (integer-projected fixed point method) The accuracy obtained by undirected graphs, IPFP-S (integer projected fixed point method initialized with spectral matching), SMAC (spectral matching with affine constraints), and BPGM (binary constraint preserving graph matching) .

Here, the horizontal axis represents the degree of deformation of the non-directional graph model, and the vertical axis represents the accuracy of the result. The non-directional graph matching method described with reference to FIGS. 3 to 13 accurately matches nodes of all non-directional graphs, and the matching accuracy of existing non-directional graph matching algorithms is lower than 80%.

FIG. 15 is a result of matching Pscal 2007, which is an open data set, through various algorithms according to an example.

In public dataset Pascal 2007, non-directional graphs that need matching include a certain number of external points. The matching accuracy according to the non-directional graph matching method described with reference to FIGS. 3 to 13 is as shown in [Table 2] below.

Number of external points
0
2
4
6
8
10
12
14
16
18
20 matching
exact
Degree(%)
92.5
84.1
72.5
51.3
46.6
54.3
34.6
34.8
27.2
23.4
31.0

The accuracy obtained by the existing graph matching method algorithms is shown in FIG.

The horizontal axis in FIG. 15 represents the number of external points, and the vertical axis represents the accuracy of the result. When compared to conventional non-directional graph matching algorithms, the non-directional graph matching method described with reference to Figures 3 to 13 significantly improves the accuracy of matching.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with reference to the drawings, various technical modifications and variations may be applied to those skilled in the art. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI &gt; or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

300: Non-directional graph processing device
310:
320: Processor
330: Memory

Claims (17)

Obtaining a first undirected graph and a second non-directional graph;
A first lattice for the first non-directional graph and a second lattice for the first non-directional graph, the coordinates of the nodes of the first non-directional graph and the edges of the second non-directional graph, Creating a second lattice for the first lattice;
Matching the first lattice and the second lattice based on a first global structure of the first lattice and a second global structure of the second lattice, The second global structure corresponding to the nodes of the second non-directional graph; And
Processing the first non-directional graph and the second non-directional graph based on the matched first lattice and the second lattice
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
The method according to claim 1,
Wherein the generating the first lattice and the second lattice comprises:
Generating the first lattice for the first non-directional graph through nonmetric multidimensional scaling (NMDS); And
Generating each of the second lattices for the second non-directional graph through the NMDS;
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
3. The method of claim 2,
Wherein the step of generating the first lattice through the NMDS comprises:
Optionally generating a target lattice; And
Generating the first lattice by optimizing the target lattice based on features of edges of the first undirected graph and edges of the first undirected graph
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
The method of claim 3,
Wherein the step of generating the first lattice by optimizing the target lattice comprises:
Calculating a distance between a first point and a second point in the target lattice, the first point and the second point corresponding to a first node and a second node of the first non-directional graph, respectively;
Returning a characteristic of an edge for the first node and the second node to a distance between the first point and the second point using a monotone function that does not include a parameter;
Calculating Kruskal's Stress-1 for the target lattice; And
Determining the target lattice as the first lattice if the computed crosstalk pressure-1 converges;
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
5. The method of claim 4,
The monotone function is determined by minimizing [Equation 1] below,
[Equation 1]

In Equation (1)

Is the above monotonic function,

Is characteristic of an edge for the first node and the second node,

Where i and j are numbers within N, which is the total number of nodes of the first non-directional graph, i is a distance between the first point and the second point,
A method for processing a plurality of non-directional graphs.
6. The method of claim 5,
The crosbachal pressure-1 is calculated by the following equation (2)
&Quot; (2) &quot;

A method for processing a plurality of non-directional graphs.
The method according to claim 1,
Wherein matching the first lattice and the second lattice comprises:
Matching the first lattice and the second lattice through PR-GLS (Point Set Registration by preserving Global and Local Structures) algorithm
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
The method according to claim 1,
Wherein obtaining the first non-directional graph and the second non-directional graph comprises:
Generating the first non-directed graph from the first image; And
Generating the second non-directional graph from the second image
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
9. The method of claim 8,
Wherein the processing of the first non-directional graph and the second non-
Synthesizing the first image and the second image based on the matched first lattice and the second lattice,
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
9. The method of claim 8,
Wherein the first non-directional graph and the second non-directional graph relate to the same object,
Wherein the processing of the first non-directional graph and the second non-
Analyzing the posture of the object
/ RTI &gt;
A method for processing a plurality of non-directional graphs.
A computer-readable recording medium containing a program for performing the method of any one of claims 1 to 10.
An apparatus for processing a plurality of non-directed graphs,
A memory in which a program for processing a plurality of non-directional graphs is recorded; And
The processor
Lt; / RTI &gt;
The program includes:
Obtaining a first undirected graph and a second non-directional graph;
A first lattice for the first non-directional graph and a second lattice for the first non-directional graph, the coordinates of the nodes of the first non-directional graph and the edges of the second non-directional graph, Creating a second lattice for the first lattice;
Matching the first lattice and the second lattice based on a first global structure of the first lattice and a second global structure of the second lattice, The second global structure corresponding to the nodes of the second non-directional graph; And
Processing the first non-directional graph and the second non-directional graph based on the matched first lattice and the second lattice
Lt; / RTI &gt;
A device for processing a plurality of non-directional graphs.
13. The method of claim 12,
Wherein the generating the first lattice and the second lattice comprises:
Generating the first lattice for the first non-directional graph through nonmetric multidimensional scaling (NMDS); And
Generating each of the second lattices for the second non-directional graph through the NMDS;
/ RTI &gt;
A device for processing a plurality of non-directional graphs.
13. The method of claim 12,
Wherein matching the first lattice and the second lattice comprises:
Matching the first lattice and the second lattice through PR-GLS (Point Set Registration by preserving Global and Local Structures) algorithm
/ RTI &gt;
A device for processing a plurality of non-directional graphs.
13. The method of claim 12,
Wherein obtaining the first non-directional graph and the second non-directional graph comprises:
Generating the first non-directed graph from the first image; And
Generating the second non-directional graph from the second image
/ RTI &gt;
A device for processing a plurality of non-directional graphs.
16. The method of claim 15,
Wherein the processing of the first non-directional graph and the second non-
Synthesizing the first image and the second image based on the matched first lattice and the second lattice,
/ RTI &gt;
A device for processing a plurality of non-directional graphs.
16. The method of claim 15,
Wherein the first non-directional graph and the second non-directional graph relate to the same object,
Wherein the processing of the first non-directional graph and the second non-
Analyzing the posture of the object
/ RTI &gt;
A device for processing a plurality of non-directional graphs.

Images