KR20190105358A - System and method for community detection of partially observed networks - Google Patents

Abstract

The present invention relates to a system for detecting a community, which restores a representative node among omitted nodes even if a part of nodes or edges is omitted when collecting data in a network environment, where a plurality of nodes are interconnected to form a community, to reliably detect the community, and a method thereof. According to the present invention, the system comprises: an input unit receiving an observable node and an edge of the observable node in a network environment in which a plurality of nodes are interconnected to form a community; a restoration unit restoring an omitted node by using the observable node and the edge of the observable node; a selection unit selecting an influential node from the omitted nodes by using a preset logic; and an estimation unit estimating a community in the network environment by using the observable node and the influential node.

Description

SYSTEM AND METHOD FOR COMMUNITY DETECTION OF PARTIALLY OBSERVED NETWORKS}

The present invention relates to a community detection system and method of a partially observed network, and more particularly, to a network environment in which a plurality of nodes are connected to each other to form a community, even if some of the nodes or edges are missing during data collection. The present invention relates to a system and method for detecting a community more reliably by recovering a representative node among missing nodes.

Modern social relations are becoming increasingly complex, and people play many roles in complex social relations. This means that people are intertwined in many forms of relationships. For example, a person lives in various aspects of the community, including co-workers, co-authors, Facebook friends, fellowships, political gatherings, and family relationships.

In addition, these various aspects of the community are generally modeled through a multi-layer graph, where a graph corresponding to each layer represents a relationship of one of the aspects of the various aspects.

Referring to FIG. 1, in relation to community detection of a network environment, the prior art 1 partially and only partially nodes and edges because it assumes that the input data includes all nodes and edges of all nodes included in the network environment. In an observable network environment, degradation of community detection performance occurs.

Meanwhile, in order to solve the problem of the related art 1, a technique for recovering missing nodes has been proposed. Referring to FIG. 2, in order to solve the problem of the related art 1, the related art 2 infers missing nodes based on the visible nodes and then performs community detection based on the visible nodes and all the missing nodes. However, recovering all the missing nodes in the network environment, as in the prior art 2, could not perform better community detection than the prior art 1 due to inference errors.

Patent Publication No. 10-2017-0111268

Accordingly, the present invention has been proposed to solve the conventional problems as described above, and an object of the present invention is for a network environment in which a plurality of nodes are connected to each other to form a community, and some of the nodes or edges are collected during data collection. It is to provide a system and method that can detect a community more reliably by recovering a representative node among the missing nodes even if they are missing.

In order to achieve the above object, a community detection system of a network, which is partially observed according to the technical spirit of the present invention, includes a visible node and a visible node in a network environment in which a plurality of nodes form a network to form a community. An input unit for receiving an edge, a recovery unit for recovering the missing node using the visible node and the edge of the visible node, and a selecting unit for selecting a main node among the missing nodes using preset logic And an estimator for estimating a community in a network environment using the visible node and the main node.

In addition, the recovery unit may generate a matrix corresponding to the number of visible nodes and the missing nodes.

The preset logic may add the components of the row or column corresponding to the missing node in the matrix, and then select the missing node whose summed value is greater than or equal to a predetermined reference value as the main node. have.

The preset logic may be configured to analyze a centrality of a region corresponding to the missing node in the matrix, and then select the missing node included in the preset range as the main node. Can be.

In addition, the recovery unit may generate the matrix using a kronecker model and an expectation-maximization algorithm.

In addition, the recovery unit generates a visible matrix reflecting the connection relationship between the visible nodes using only the visible node and the edges of the visible node, and generates a generation parameter matrix and a node permutation value using the visible matrix and an expectation-maximization algorithm. And generating a probabilistic neighbor matrix by multiplying the generation parameter matrix a plurality of times, and reflecting information of the visible matrix in the probabilistic neighbor matrix, and corresponding to the missing node of the probabilistic neighbor matrix. Performing Bernoulli enforcement in the domain derives success and failure events, generates a recovery matrix by reflecting 1 in the component corresponding to the success event, and 0 in the component corresponding to the failure event. You can do

In addition, the estimator may estimate the community using non-negative matrix factorization (NMF).

On the other hand, in order to achieve the above object, the community detection method of the network partially observed by the technical idea of the present invention, a visible node in a network environment where the input unit forms a community by forming a plurality of nodes (network) and Receiving an edge of the visible node; recovering a missing node using the visible node and an edge of the visible node; and selecting a selector from among the missing nodes using preset logic; Selecting a main node, and estimating unit using the visible node and the main node to estimate a community in a network environment.

Further, when the recovery unit recovers the missing node by using the visible node and the edge of the visible node, a matrix corresponding to the number of the visible node and the missing node may be generated.

The preset logic may add the components of the row or column corresponding to the missing node in the matrix, and then select the missing node whose summed value is greater than or equal to a predetermined reference value as the main node. have.

The preset logic may be configured to analyze a centrality of a region corresponding to the missing node in the matrix, and then select the missing node included in the preset range as the main node. Can be.

In addition, the matrix may be generated by the recovery unit using a kronecker model and an expectation-maximization algorithm.

In addition, the recovering unit recovers the missing node by using the visible node and the edge of the visible node by using the edge of the visible node and the visible node to generate a visible matrix reflecting the connection relationship of the visible nodes step; Obtaining a generation parameter matrix and node permutation values using the visible matrix and the expectation-maximization algorithm; Generating a probabilistic neighbor matrix by squaring the generation parameter matrix a plurality of times; Reflecting information of the visible matrix in the probabilistic neighbor matrix; Deriving a success event and a failure event by performing a Bernoulli enforcement in a region corresponding to the missing node of the probabilistic neighbor matrix; The method may include generating a recovery matrix by reflecting 1 to a component corresponding to the success event and 0 to a component corresponding to the failure event.

In the estimating unit, the estimating community in a network environment using the visible node and the main node may include estimating the community using non-negative matrix factorization (NMF). Can be.

According to the system and method for community detection of a network in which the phase portion is observed according to the present invention,

First, the present invention can be applied to community detection in various network environments where partial personal data exposure and partial data only are partially observed. For example, it can be applied to various complex systems including biological, social and technical networks.

Second, since only nodes having a plurality of edges among the missing nodes are selectively used, community detection that is closer to a circle than the conventional one is possible.

Third, since a process of detecting a community using only the visible node and a method of detecting a community using the visible node and all the missing nodes are similar, a wide application is possible by replacing the conventional system.

Fourth, the present invention can be utilized by applying the present invention to an environment that infers necessary information using only partial data.

1 is a diagram illustrating a prior art process for detecting a community using only visible nodes.
FIG. 2 shows a process of prior art 2 for detecting a community using a visible node and all missing missing nodes. FIG.
3 is a block diagram of a community detection system of a partially observed network according to an embodiment of the present invention.
4 is a diagram illustrating a process of detecting a community after deriving a main node by a system and method for community detection of a partially observed network according to an embodiment of the present invention.
5 is an exemplary view illustrating a process of a recovery unit generating a recovery matrix according to an embodiment of the present invention.
6 is an exemplary diagram illustrating a process of selecting a main node using a recovery matrix according to an embodiment of the present invention.
7 is a diagram illustrating a relationship between a community and a node through a estimator according to an embodiment of the present invention.
8 is a flow chart of a community detection method of a partially observed network according to an embodiment of the present invention.
9 is a flowchart illustrating a detailed process of step S120 according to an embodiment of the present invention.
10 and 11 are graphs comparing the results of a community detection system and method of a partially observed network according to an embodiment of the present invention and performances when the prior arts 1 and 2 are tested under the same conditions.

With reference to the accompanying drawings will be described in detail with respect to the community detection system and method of the network partially observed according to embodiments of the present invention. As the inventive concept allows for various changes and numerous modifications, particular embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific form disclosed, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Also, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

The community detection system 100 of a partially observed network according to an embodiment of the present invention is a system included in a computing device such as a computer and a server to perform a function, and an embodiment of the present invention is included in a single computing device or In an environment in which a plurality of computing devices are organically connected using a communication network, an embodiment of the present invention may be implemented as being distributed to the plurality of computing devices.

3 and 4, the community detection system 100 of a partially observed network according to an embodiment of the present invention is visible in a network environment in which a plurality of nodes form a network to form a community. ) An input unit 110 that receives edges of the node and the visible node, a recovery unit 120 that recovers the missing node using the edges of the visible node and the visible node, and a missing using the preset logic. A selector 140 selects a main node from among nodes, and an estimator 160 estimating a community in a network environment using the visible node and the main node.

The network environment may include a social network, a biological network such as a neural network, and the like that form a friend relationship between a user and share information such as a text, a photo, and a video.

This embodiment assumes that the network environment is a social network by way of example.

A node in a social network becomes a user, and the edge is a friend relationship state with the user or the number of the friend relationships. In other words, it can be understood that the edge is a link connecting the node with the node. Nodes can have zero to multiple edges.

Nodes include at least visible nodes and missing nodes. In this embodiment, the node that is collected when the input unit 110 collects data about the network environment is called a visible node, and the node that exists on the actual network environment but is not collected through the input unit 110 is understood as a missing node.

The recovery unit 120 is a process for recovering missing nodes and generates a matrix corresponding to the number of visible nodes and missing nodes. That is, the matrix corresponds to the number of rows and columns and the number of visible and missing nodes.

Referring to FIG. 5, the recovery unit 120 generates a matrix using a kronecker model and an expectation-maximization algorithm.

In detail, the recovery unit 120 generates a visible matrix reflecting the connection relationship between the visible nodes using only the edges of the visible node and the visible node. The visible matrix is a square matrix where the number of rows and columns is equal to the number of visible nodes. The components constituting the matrix mean edges. For example, if a second user and a third user have a friend relationship in a social network, the component value of the position [2, 3] becomes 1. The component becomes 0 between users who do not have a friendship.

In addition, the recovery unit 120 obtains values of the generation parameter matrix Θ and the node permutation σ using the visible matrix and the expected-maximization algorithm. The obtained generation parameter matrix has a square matrix form such as 2 * 2, 3 * 3, 5 * 5. In addition, the components included in the generation parameter matrix each have a value between 0 and 1 because they represent a probability.

In addition, the recovery unit 120 generates a probabilistic neighbor matrix by performing a Kronecker product, that is, by multiplying the generation parameter matrix Θ by the index K multiple times. The generation parameter matrix is a square matrix, and as the stochastic neighbor matrix is squared by the exponent K, the stochastic neighbor matrix also becomes a square matrix.

Preferably, the number of rows and columns of the probabilistic contiguous matrix is greater than or equal to the number of all nodes that contain visible and missing nodes. For example, if the number of all nodes is 100, the index K may be 7 or more, so that the number of rows and columns of the probabilistic adjacent matrix generated may be 128, which is greater than the number of all nodes. The range of rows and columns beyond the number of all nodes is removed at this or a later stage. The probabilistic adjacency matrix functions as a table that provides a matrix of appropriate scale to represent the network connectivity of all nodes. In the example of FIG. 5, the exponent K is set to 3 to generate a probabilistic neighbor matrix, and a 3x squared generation parameter matrix Θ produces a 8 * 8 probabilistic neighbor matrix.

In addition, the recovery unit 120 reflects the information of the visible matrix in the probabilistic neighbor matrix. In the example of FIG. 5, the visible matrix is implemented with a scale of 6 * 6. The recovery unit 120 replaces an area corresponding to 6 * 6 of the probabilistic neighboring matrix of 8 * 8 scale with the value of the visible matrix.

In addition, the recovery unit 120 derives a success event and a failure event by performing a Bernoulli trial in a region corresponding to a missing node of a probabilistic adjacent matrix (an area not substituted with a visible matrix). Before the Bernoulli trial is implemented, the components that fall into the range of missing nodes have values between 0 and 1, respectively. The Bernoulli trial is carried out to infer the connections (edges) of the missing nodes. Bernoulli enforcement By using this, a success event and a failure event are derived. In this case, u means a row and v means a column. Bernoulli trials are carried out on components falling within the range of the missing node.

As a result of the Bernoulli trial, the component derived from the success event is replaced with 1, and the component derived from the failure event is replaced with 0.

Thus, the resulting matrix is a recovery matrix where the scale of the rows and columns corresponds to the number of all nodes, and all components have a value of zero or one. The recovery matrix includes not only visible nodes but also missing nodes, so that the edges of the missing nodes are recovered.

As illustrated, components of the seventh row and the seventh column corresponding to the seventh node may be symmetrical with each other, and components of the eighth row and the eighth column corresponding to the eighth node may be symmetrical with each other.

The edge of the missing node in the recovery matrix is different from the edge in the real network environment. As described above, the missing node of the recovery matrix is derived by performing a Bernoulli trial on the probabilistic neighbor matrix generated by generating a generation parameter matrix from the visible matrix and multiplying the generation parameter matrix. That is, there is a possibility that an error exists because the edge of the missing node shown in the recovery matrix is derived using statistical inference.

Referring to FIG. 6, the selector 140 selects major nodes from missing nodes by using preset logic. The missing node is the missing node that the recovery unit 120 has recovered.

The preset logic may be at least two ways.

)이 2로 정의되고, 복구 행렬 중 누락된 노드에 해당되는 7번째 행과 8번째 행에 대하여 성분들 값의 합산이 실시되었다. 7번째 행은 [1, 7]과 [3, 7]의 성분이 1이므로 합산 결과는 2가 된다. 8번째 행은 [1, 8]의 성분만 1이므로 합산 결과는 1이 된다. 따라서, 7번째 행만 기 설정된 기준 값의 이상에 해당되므로, 7번째 행에 대응되는 누락된 노드(제7노드)만 주요 노드로 선택된다.The preset logic according to the first embodiment sums the values of the components of the row or column corresponding to each missing node in the recovery matrix. When the values of the components of the row or column for each missing node are added together, the missing node whose summed value is greater than or equal to the predetermined reference value is selected as the main node. For example, the embodiment of FIG. 6 may have a preset reference value (

) Is defined as 2, and the component values are summed over the 7th and 8th rows corresponding to the missing nodes in the recovery matrix. In the seventh row, since the components of [1, 7] and [3, 7] are 1, the sum result is 2. In the eighth row, only the components of [1, 8] are 1, so the sum result is 1. Therefore, since only the seventh row corresponds to a predetermined reference value or more, only the missing node (seventh node) corresponding to the seventh row is selected as the main node.

The preset logic according to the second embodiment analyzes the centrality of the region corresponding to the missing node in the recovery matrix, and selects the missing node included in the preset range as the main node.

When the main node is selected in the selector 140, the estimator 160 estimates the community in the network environment using the visible node and the main node. Community estimation may use non-negative matrix factorization (NMF).

의 가중치를 갖는다.Referring to FIG. 7, the relationship between the community and the node is represented by the matrix F. The probability that it belongs to community c in node u

Has a weight of.

For example, if matrix A has a scale of 100 * 100 and there are 5 communities, the 100 * 100 matrix consists of the product of 100 * 5 matrix and 5 * 100 matrix. When this matrix is called F, A can be derived by calculating the F * F transition matrix.

In this embodiment, matrix A is inferred. If matrix A has an 80 * 80 matrix of visible and principal nodes even though it is not accurate, we know that the matrix consists of (80 * 5) * (5 * 80), and we know that 80 * 5 and 5 * 80 Since we know that matrices are symmetric, we only infer 80 * 5 matrices.

Finding an 80 * 5 matrix uses non-shading matrix factorization. Using a non-shading matrix factorization technique, a node of five communities belongs to which community, and whether the node belongs to a duplicate, etc., is given to the information of the node. Also, nodes that do not belong to the community are found.

As mentioned above, the edge of the missing node is derived using statistical inference, so there is a possibility of error. If all missing nodes with errors are used for community estimation, errors will accumulate, resulting in larger errors. In addition, when the community is completely excluded or only a few missing nodes are used when estimating the community, the correct community structure cannot be estimated because there is not enough information necessary to detect the community.

In the embodiment of the present invention, since the selected main node has a large number of edges, it can be expected that there will be fewer errors than missing nodes having a small number of edges. In addition, since a node having a plurality of edges generally has a function of representing a community, the node has a higher reliability than estimating the community by including all the missing nodes having a small number of edges. For example, assuming that one major is a community, it is only the department representatives who make up the community but all students (nodes) belonging to the major, but who interact with a large number of students in the community. Even by finding and observing, it is possible to sufficiently estimate the character of the community. In this case, the missing node having a small number of edges may be a student of another engineering department who has a personal acquaintance with the department representative. Rather, including a valid node in the community estimation may cause an error in the community estimation result.

Therefore, the method of estimating a community using only the visible nodes and the main nodes according to an embodiment of the present invention has a higher reliability than the community estimating method using only the visible nodes or the community estimating method using the visible nodes and all missing nodes. Community detection is possible.

Next, a community detection method of a partially observed network according to an embodiment of the present invention will be described.

4 and 8, according to an embodiment of the present invention, a method for detecting a community of a partially observed network is visible in a network environment in which the input unit 110 forms a community by forming a plurality of nodes in a network. Receiving an edge of the node and the visible node (S110), recovering the missing node using the visible node and the edge of the visible node (S120), and the selecting unit (140) selecting the main node from the missing node using the predetermined logic (S140), and the estimator 160 using the visible node and the main node to estimate the community in the network environment (S160) Characterized in that it comprises a.

In step S120, a matrix corresponding to the number of visible nodes and missing nodes is generated. The matrix is generated by the recovery unit 120 using a kronecker model and an expectation-maximization algorithm.

6 and 9, in step S120, a visible matrix reflecting a connection relationship between visible nodes using a visible node and an edge of the visible node is generated (S121), and a visible matrix and an expectation-maximization algorithm are used. Acquiring a generation parameter matrix and a node permutation value (S122); generating a probabilistic neighbor matrix by multiplying the generation parameter matrix (S124); and reflecting information of the visible matrix on the probabilistic neighbor matrix. Step (S126), performing Bernoulli in a region corresponding to the missing node of the probabilistic neighbor matrix, deriving a success event and a failure event (S127), and a component corresponding to the success event is reflected to 1 In operation S128, a recovery matrix is generated by reflecting 0 in the component corresponding to the failure event.

The preset logic of step S140 may be at least two methods.

The preset logic according to the first embodiment sums up the components of the row or column corresponding to the missing node in the recovery matrix, and then selects the missing node as the main node whose summed value is greater than or equal to the preset reference value.

The preset logic according to the second embodiment analyzes the centrality of the region corresponding to the missing node in the recovery matrix, and selects the missing node included in the preset range as the main node.

Step S160 estimates the community using non-negative matrix factorization (NMF).

[simulation]

In order to measure the community detection performance of the community detection system 100 and method of the partially observed network according to an embodiment of the present invention, an experiment was conducted on a data set labeled with a ground-truth community. In addition, even in the prior art, community detection was performed under the same conditions.

Prior art 1 is a method for detecting communities using only visible nodes, and prior art 2 is a method for detecting communities using all missing nodes.

Normalized Mutual Information (NMI), which is a widely used measurement method, was used to measure community detection performance.

Referring to FIG. 10, as a result of community detection using an embodiment of the present invention and the prior art 1 and the prior art 2 for five kinds of synthetic data sets, an embodiment of the present invention exhibits significantly higher community detection performance. Indicated.

In addition, referring to FIG. 11, an embodiment of the present invention is shown in a result of community detection using an embodiment of the present invention and the prior art 1 and the prior art 2 for actual data sets of Amazon and DBLP. It was found to exhibit significantly higher community detection performance.

Although the preferred embodiment of the present invention has been described above, the present invention may use various changes, modifications, and equivalents. It is clear that the present invention can be applied in the same manner by appropriately modifying the above embodiments. Therefore, the above description does not limit the scope of the present invention as defined by the limitations of the following claims.

100: community detection system 110: input unit
120: recovery unit 140: selection unit
160: estimator

Claims (14)

An input unit configured to receive a visible node and an edge of the visible node in a network environment in which a plurality of nodes form a network to form a community;
A recovery unit for recovering a missing node using the visible node and the edge of the visible node;
A selector which selects a main node from the missing nodes by using preset logic;
And an estimator for estimating a community in a network environment using the visible node and the main node. The method of claim 1,
And the recovery unit generates a matrix corresponding to the number of visible nodes and the missing nodes. The method of claim 2, wherein the preset logic is
And summing the components of the row or column corresponding to the missing node in the matrix, and then selecting the missing node whose summed value is greater than or equal to a predetermined reference value as the main node. . The method of claim 2, wherein the preset logic is
After analyzing the centrality of the area corresponding to the missing node in the matrix, community detection of the partially observed network characterized in that the missing node included in the predetermined range is selected as the main node system. The method according to any one of claims 2 to 4,
And the recovery unit generates the matrix using a kronecker model and an expectation-maximization algorithm. The method of claim 5,
The recovery unit generates a visible matrix reflecting a connection relationship between the visible nodes using only the edges of the visible node and the visible node,
Obtain a generation parameter matrix and node permutation values using the visible matrix and the expected-maximization algorithm,
After generating a probabilistic neighbor matrix by multiplying the generation parameter matrix,
Reflects information of the visible matrix in the probabilistic neighbor matrix,
Deriving a success event and a failure event by performing Bernoulli enforcement in the region corresponding to the missing node of the probabilistic neighbor matrix,
And a recovery matrix is generated by reflecting 1 in a component corresponding to the success event and 0 in a component corresponding to the failure event. The method of claim 1,
And the estimator estimates the community using non-negative matrix factorization (NMF). Receiving, by an input unit, a visible node and an edge of the visible node in a network environment in which a plurality of nodes form a network to form a community;
A recovery unit recovering the missing node using the visible node and the edge of the visible node;
Selecting a main node from the missing nodes by using a preset logic;
And an estimator estimating a community in a network environment by using the visible node and the main node. The method of claim 8, wherein the recovery unit recovers the missing node using the visible node and the edge of the visible node,
And a matrix corresponding to the number of visible nodes and the missing nodes is generated. The method of claim 9, wherein the predetermined logic is
Summing the components of the row or column corresponding to the missing node in the matrix, and then selecting the missing node whose summed value is greater than or equal to a predetermined reference value as the main node. . The method of claim 9, wherein the predetermined logic is
After analyzing the centrality of the area corresponding to the missing node in the matrix, community detection of the partially observed network characterized in that the missing node included in the predetermined range is selected as the main node Way. The method according to any one of claims 9 to 11,
And the matrix is generated by the recovery unit using a kronecker model and an expectation-maximization algorithm. The method of claim 12, wherein the recovery unit recovers the missing node by using the visible node and the edge of the visible node.
Generating a visible matrix reflecting a connection relationship between the visible nodes using the edges of the visible node and the visible node; Obtaining a generation parameter matrix and node permutation values using the visible matrix and the expectation-maximization algorithm; Generating a probabilistic neighbor matrix by squaring the generation parameter matrix a plurality of times; Reflecting information of the visible matrix in the probabilistic neighbor matrix; Deriving a success event and a failure event by performing a Bernoulli enforcement in a region corresponding to the missing node of the probabilistic neighbor matrix; And generating a recovery matrix by reflecting 1 to a component corresponding to the success event and 0 to a component corresponding to the failure event. The method of claim 8, wherein the estimating unit estimates a community in a network environment using the visible node and the main node.
A method for community detection of a partially observed network, characterized by estimating a community using non-negative matrix factorization (NMF).

Images