CN111860866A - A network representation learning method and device with community structure - Google Patents

Abstract

The invention discloses a network representation learning method with a community structure, which comprises the following steps: step 1: data collection and processing stage: using a density function, vertex sequence samples S ═ S are obtained by using a random walk strategy on the network G₁,s₂,...,s_n}; step 2: data representation learning phase: optimizing the Skip-gram model, and training vertex sequence samples S-S by using the Skip-gram model₁,s₂,...,s_nObtaining the vector representation of each vertex sequence; and step 3: a data calculation stage: and carrying out similarity calculation on the vector representation of each vertex sequence to obtain community division similarity. The method can better capture the community structure in the network and can obtain higher accuracy in the vertex classification task.

Description

A network representation learning method and device with community structure

technical field

The invention relates to the field of computer technology, in particular to a network representation learning method and device with a community structure.

Background technique

Many complex systems can be abstracted into network structures, and network structures are usually represented by graphs, which consist of a set of nodes and a set of connected edges. For small-scale networks, we can quickly perform many complex tasks on it, such as community mining and multi-label classification. However, performing these complex tasks on large-scale networks (such as those with billions of vertices) is a challenge. To solve this problem, we must find another concise and efficient network representation. An effective strategy for solving this problem is network embedding, which is to learn low-dimensional vector representations of vertices in the network. For each vertex, we map their structural features in the network to low-dimensional spatial vectors, which are then applied to complex tasks in the network.

In the past few years, many network embedding methods have been proposed to characterize the local structure of the network. The DeepWalk method characterizes the neighborhood structure of network vertices by using a truncated random walk strategy. The Node2vec approach proves that DeepWalk does not capture the diversity of connection patterns in the network. It proposes a biased random walk strategy that combines BFS and DFS ideas to explore vertex neighborhood information. The LINE method is mainly applied to large-scale network embedding learning. It preserves the high-order vertex neighborhood structure and scales easily to millions of vertices. Cao et al. proposed a deep graph representation model that employs a random surfing strategy to capture the structural information of the graph. Feng et al. proposed the "degree penalty" principle, which preserves the scale-free property by penalizing the proximity between vertices of high degree. Wang et al. proposed a semi-supervised deep model capable of capturing highly nonlinear network structures by optimizing multiple layers of nonlinear functions. Yanardag et al. proposed a general framework to capture similar structures in mid-level. In addition, they also proposed some methods to preserve the global network structure. Wang et al. proposed a modular non-negative matrix factorization model that preserves the community structure in the network. Tu et al. proposed a heuristic community enhancement mechanism that maps community structure information into vertex vector representations. Chen et al. proposed a multi-level network representation learning paradigm that gradually merges the initial network into smaller but structurally similar networks by recapturing the global structure of the initial network.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is that the three network embedding methods for describing the local structure of the network in the prior art cannot better capture the community structure in the network, and cannot obtain higher accuracy in the vertex classification task. The purpose is to provide A network representation learning method and device with community structure solve the above problems.

The present invention is achieved through the following technical solutions:

A network representation learning method with community structure, including the following steps:

Step 1: Data collection and processing phase: use a density function to obtain vertex sequence samples S={s ₁ ,s ₂ ,...,s _n } by using a random walk strategy on the network G;

Step 2: Data representation learning stage: optimize the Skip-gram model, use the Skip-gram model to train vertex sequence samples S={s ₁ , s ₂ ,...,s _n }, and obtain the vector representation of each vertex sequence;

Step 3: Data calculation stage: Calculate the similarity of the vector representation of each vertex sequence to obtain the similarity of community division.

Further, a network representation learning method with community structure, in the step 1, the vertex sequence in the vertex sequence sample S={s ₁ ,s ₂ ,...,s _n } is represented as s={v1,v2 ...,v|s|}.

Further, a network representation learning method with community structure, the density function in the step 1 is defined as:

和

分别是顶点序列s中所有顶点的内部度之和与外部度之和，而α是分辨率参数，用于控制社团的大小；in

and

are the sum of the internal and external degrees of all vertices in the vertex sequence s, respectively, and α is the resolution parameter used to control the size of the community;

The density function also has a density gain _Δf ^v _s , ^which should satisfy the following formula:

Δf _s =f _s+{v} -f _s

where the symbol s+{v} represents the new vertex sequence obtained by moving vertex v to s.

Further, in a network representation learning method with a community structure, the specific steps of obtaining vertex sequence samples in the step 1 are:

Step 11: Randomly select a vertex v _|s|+1 from the set N'(v _|s| );

Step 12: Calculate Δf _s ^v|s|+1 according to the formula Δf _s =f _s+{v} -f _s ;

Step 13: If Δf _s ^v|s|+1 <0, delete v _|s|+1 from the set N'(v _|s| ), and then return to step 11;

Step 14: If Δf _s ^v|s|+1 >0, add v _|s|+1 to the set s, and mark v _|s|+1 as the current vertex;

where vertex v _|s| is the last added vertex, let the last added vertex v _|s| be the current vertex; N′(v _|s| ) represents the set of all the neighbor vertices of the current vertex v _|s| that are not in s; Repeat steps 11 to 14 until the density of the vertex sequence s cannot be increased.

Further, a network representation learning method with community structure, in the step 2, the Skip-gram model trains vertex sequence samples by minimizing the following objective function:

where t is the window size, v _j is the vertex representation of _{vi in the context network within the window, and the probability p(v j |} vi ₎ in the above formula is defined as

where Φ(s) represents the embedding vector of s, Φ′(s) represents the context vector, and s represents the vertex sequence set.

Further, a network representation learning method with a community structure, in the step 3, the similarity calculation for the vector representation of each vertex sequence specifically includes: calculating the vector representation of each vertex sequence in the network and other vertices. The similarity degree of the vector representation of the sequence, specifically using the NMI formula to calculate the similarity.

A network representation learning device with community structure, comprising:

The data collection and processing module is used to read the vertex sequence samples and obtain the vertex sequence samples S={s ₁ , s ₂ ,...,s _n };

The data representation learning module is used to optimize the Skip-gram model. The Skip-gram model is used to train vertex sequence samples S={s ₁ , s ₂ ,...,s _n } to obtain the vector representation of each vertex sequence;

The similarity calculation module is used to calculate the similarity of the vector representation of each vertex sequence to obtain the similarity of community division.

The inventive method uses the following NMI formula to calculate similarity:

The Normalized Mutual Information measure (NMI) is an information-theoretic metric used to measure the similarity of the divisions A and B of two communities. NMI is defined as follows:

where C is the similarity matrix, where the rows correspond to the "true" communities and the columns correspond to the "probe" communities, and N is the number of nodes. C _ij is the number of common vertices in real community i and probe community j. C _A and C _B represent the number of real communities and probe communities, respectively. C _i . and C _.j denote the sum of the i-th row and j-th column of matrix C, respectively. The value range of NMI is 0 to 1. If the real community partition is exactly the same as the detection community partition, the NMI is equal to 1; otherwise, it is equal to 0.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

The method of the present invention can not only better capture the community structure in the network, but also obtain higher accuracy in the vertex classification task.

Description of drawings

The accompanying drawings described herein are used to provide further understanding of the embodiments of the present invention, and constitute a part of the present application, and do not constitute limitations to the embodiments of the present invention. In the attached image:

FIG. 1 is an overall flow chart of the present invention.

Figure 2 shows the boost ratio of NMI and Q-Walker on artificial network, parameter α=1.5.

Figure 3 shows the optimal interval of parameter α on four real networks.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and the accompanying drawings. as a limitation of the present invention.

Example

As shown in Figure 1, a network representation learning method with community structure includes the following steps:

Step 1: Data collection and processing phase: use a density function to obtain vertex sequence samples S={s ₁ ,s ₂ ,...,s _n } by using a random walk strategy on the network G;

Step 2: Data representation learning stage: optimize the Skip-gram model, use the Skip-gram model to train vertex sequence samples S={s ₁ , s ₂ ,...,s _n }, and obtain the vector representation of each vertex sequence;

Step 3: Data calculation stage: Calculate the similarity of the vector representation of each vertex sequence to obtain the similarity of community division.

Further, a network representation learning method with community structure, in the step 1, the vertex sequence in the vertex sequence sample S={s ₁ ,s ₂ ,...,s _n } is represented as s={v1,v2 ...,v|s|}.

Further, a network representation learning method with community structure, the density function in the step 1 is defined as:

和

分别是顶点序列s中所有顶点的内部度之和与外部度之和，而α是分辨率参数，用于控制社团的大小；in

and

are the sum of the internal and external degrees of all vertices in the vertex sequence s, respectively, and α is the resolution parameter used to control the size of the community;

The density function also has a density gain _Δf ^v _s , ^which should satisfy the following formula:

Δf _s =f _s+{v} -f _s

where the symbol s+{v} represents the new vertex sequence obtained by moving vertex v to s.

Further, in a network representation learning method with a community structure, the specific steps of obtaining vertex sequence samples in the step 1 are:

Step 11: Randomly select a vertex v _|s|+1 from the set N'(v _|s| );

Step 12: Calculate Δf _s ^v|s|+1 according to the formula Δf _s =f _s+{v} -f _s ;

Step 13: If Δf _s ^v|s|+1 <0, delete v _|s|+1 from the set N'(v _|s| ), and then return to step 11;

Step 14: If Δf _s ^v|s|+1 >0, add v _|s|+1 to the set s, and mark v _|s|+1 as the current vertex;

where vertex v _|s| is the last added vertex, let the last added vertex v _|s| be the current vertex; N′(v _|s| ) represents the set of all the neighbor vertices of the current vertex v _|s| that are not in s; Repeat steps 11 to 14 until the density of the vertex sequence s cannot be increased.

Further, a network representation learning method with community structure, in the step 2, the Skip-gram model trains vertex sequence samples by minimizing the following objective function:

where t is the window size, v _j is the vertex representation of _{vi in the context network within the window, and the probability p(v j |} vi ₎ in the above formula is defined as

where Φ(s) represents the embedding vector of s, Φ′(s) represents the context vector, and s represents the vertex sequence set.

Further, a network representation learning method with a community structure, in the step 3, the similarity calculation for the vector representation of each vertex sequence specifically includes: calculating the vector representation of each vertex sequence in the network and other vertices. The similarity degree of the vector representation of the sequence, specifically using the NMI formula to calculate the similarity.

A network representation learning device with community structure, comprising:

The data collection and processing module is used to read the vertex sequence samples and obtain the vertex sequence samples S={s ₁ , s ₂ ,...,s _n };

The data representation learning module is used to optimize the Skip-gram model. The Skip-gram model is used to train vertex sequence samples S={s ₁ , s ₂ ,...,s _n } to obtain the vector representation of each vertex sequence;

The similarity calculation module is used to calculate the similarity of the vector representation of each vertex sequence to obtain the similarity of community division.

In this embodiment, the following NMI formula is used to calculate the similarity:

The Normalized Mutual Information measure (NMI) is an information-theoretic metric used to measure the similarity of the divisions A and B of two communities. NMI is defined as follows:

where C is the similarity matrix, where the rows correspond to the "true" communities and the columns correspond to the "probe" communities, and N is the number of nodes. C _ij is the number of common vertices in real community i and probe community j. C _A and C _B represent the number of real communities and probe communities, respectively. C _i . and C _.j denote the sum of the i-th row and j-th column of matrix C, respectively. The value range of NMI is 0 to 1. If the real community partition is exactly the same as the detection community partition, the NMI is equal to 1; otherwise, it is equal to 0.

Many classical embedding methods, such as DeepWalk, Node2vec and DP-Walker, obtain a set of vertex sequence samples S={s ₁ ,s ₂ ,...,s _n } by using a random walk strategy on the network G, where each The vertex sequence can be represented as s={v ₁ ,...,v _|s| }. By treating each vertex sequence as a sentence in a document, we can use the Skip-gram model to learn vertex representations in the network:

For the DeepWalk method, it adopts a uniform distribution p(v _i+1 |v _i ) in the random walk process, that is, each neighbor of v _i has an equal probability of being selected.

For the Node2vec method, it uses a biased probability p(v _i+1 |v _i ) in the random walk process, which is defined as:

where d _{i-1, i+1} represents the shortest path distance between vertex v _i-1 and vertex v _i+1 . In equation (3), the parameters p and q control the proportions of the breadth-first search strategy and the depth-first search strategy in the random walk process, respectively.

For DP-Walker, the probability p(v _i+1 |v _i ) is defined as

where k _i is the degree of vertex v _i , C _i,i+1 is the number of common neighbors of v _i and v _i+1 , and β is the model parameter.

The mean-shift clustering method is a nonparametric clustering process. Compared with the classic k-means clustering method, it does not need to assume the shape of the distribution, nor the number of clusters. Given n data points x _i ∈ R ^d (i=1,...,n), the multivariate kernel density estimate based on the radially symmetric kernel K(x) is given by:

where h is the radius of the nucleus. For each data point _xi , a gradient ascent optimization strategy is performed on its local estimated density until convergence. All data points associated with the same center point belong to the same cluster.

The following is the experimental analysis:

(1) Real network

In the community mining experiments, we use four real networks, namely Karate, Football, Dolphin and PolBooks networks. Table 1 lists the details of the four networks, including the number of nodes (|V|), the number of edges (|E|), the mean degree (<k>), the mean square degree (<k ² >), the mean clustering Coefficients (cc) and the number of true communities (nc).

Table 1: Four network statistics with real communities

(2) Artificial network

In community mining experiments, we further use artificial networks to evaluate the performance of our method. The Plantedpartition model is a classic artificial benchmark network generator. The model generates a network with n=g z vertices, where z is the number of communities and g is the number of vertices in each community. In the same community, the probability of a connecting edge between any two vertices is p _in , and between different communities, the probability of an edge existing between any two vertices is P _out . Average degree of each vertex <k>= _{pin(g-1)+poutg (} z-1). If p _in >= p _out , the network has a community structure because the density of connections within communities is greater than the density of connections between communities. In the present invention, we adopt a special case of the l-partition model proposed by Girvan and Newman. They set z=4, g=32, <k>=16. Table 2 shows 7 artificial networks, where the larger the internal average degree <k _in > of the vertices, the stronger the community structure.

Table 2: Statistics of 7 artificial networks with different community structures

(3) Benchmark method

We compare our method (Q-Walker) with three network embedding methods (DeepWalk, Node2vec and DP-Walker).

(4) Community detection

After learning the embedding vector representation of each node, we use the mean-shift clustering algorithm to mine communities.

(5) Accuracy index

The Normalized Mutual Information measure (NMI) is an information-theoretic metric used to measure the similarity of the divisions A and B of two communities. NMI is defined as follows:

where C is the similarity matrix, where the rows correspond to the "true" communities and the columns correspond to the "probe" communities, and N is the number of nodes. C _ij is the number of common vertices in real community i and probe community j. C _A and C _B represent the number of real communities and probe communities, respectively. C _i . and C _.j denote the sum of the i-th row and j-th column of matrix C, respectively. The value range of NMI is 0 to 1. If the real community partition is exactly the same as the detection community partition, the NMI is equal to 1; otherwise, it is equal to 0.

(6) Real network analysis

We first evaluate the performance of Q-Walker on four real networks with known communities, and the results are shown in Table 3. As can be seen from Table 3, Q-Walker and DP-Walker outperform the other two algorithms, DeepWalk and Node2vec, in all networks. Therefore, next we only compare the Q-Walker and DP-Walker algorithms. On the Karate network, the NMI of Q-Walker and DP-Walker are 1 and 0.837, respectively. Therefore, Q-Walker can correctly detect all known communities, and its NMI is improved by 19.47% compared with DP-Walker. Similarly, on the Dolphin network, Q-Walker can also find all known communities, and its NMI is improved by 12.48%. On the PolBooks network, the NMI of Q-Walker and DP-Walker are 0.679 and 0.581, respectively. Compared with the NMI of DP-Walker, the NMI of Q-Walker is improved by 16.86%. On the Football network, the NMI of the Q-Walker is still slightly higher than that of the other three methods, although all methods perform fairly well. Compared with DP-Walker, the NMI of Q-Walker is improved by 1.81%.

Table 3: NMI on four real networks with known communities

(7) Artificial network analysis

We also evaluate the performance of our method on artificial networks with different community structures, where Table 2 shows the details of 7 artificial networks. The experimental results are shown in Figure 2. As can be seen from Figure 2, when <k _in >≤10.5, the Q-Walker method outperforms the other three methods. At the same time, we also noticed that compared with the NMI of the other three methods, the ratio of Q-Walker improvement is inversely proportional to <k _in >, that is, the smaller <k _in >, the higher the ratio of Q-Walker improvement. Taking Node2vec as an example, when <k _in >=8.5, the improvement ratio of Q-Walker is higher than 50%, and when <k _in >=11.5, the improvement ratio of Q-Walker is 0%. The reasons for this are explained below. When k=8.5, the network has many weak community structures. Since there are many edges between weak community structures, nodes can easily jump from one weak community to another during random walk. Therefore, the vertex sequence s sampled by Node2vec cannot well describe the weak community structure, because most of the vertices in s come from different weak community structures. However, for Q-Walker, most of the vertices in the vertex sequence s are from the same community structure, so s has relatively tight internal connections. Therefore, Q-Walker can well characterize weak community structure. When k=11.5, the network has many strong community structures. Due to the high density of internal connections and few edges between strong community structures, nodes walk randomly within the same community most of the time. Therefore, the vertex sequence s sampled by Node2vec can well characterize the strong community structure, because most of the vertices in s are from the same community. Similarly, we can use the above analysis to explain two other benchmark methods, DeepWalk and DP-Walker. Based on the above analysis, Q-Walker not only performs well in networks with weak community structure, but also performs well in networks with strong community structure.

(8) Parameter sensitivity

中参数α的值来评估我们方法的性能。实验中，参数α的取值范围为0.05≤α≤1.5的，结果如图3所示。从图3可以看出，每个网络都包含一个最优分辨率区间，在这个区间内，算法的性能稳定且最好。以Karate网络为例，在0.55≤α≤0.7，我们的算法能够发现所有已知的社团结构。另外，每个网络α的最优区间是不同的。以Dolphin和PolBooks网络为例，它们的最优区间分别为0.5≤α≤0.8和0.05≤α≤0.2。最优区间的不同与网络中的社团结构层次相关。通常地，不同网络的社团层次结构是不相同的。因此，不同网络的α最优区间也应该是不相同的。Finally, we change the formula

to evaluate the performance of our method. In the experiment, the value range of parameter α is 0.05≤α≤1.5, and the result is shown in Figure 3. As can be seen from Figure 3, each network contains an optimal resolution interval, and within this interval, the performance of the algorithm is stable and the best. Taking the Karate network as an example, at 0.55≤α≤0.7, our algorithm is able to discover all known community structures. In addition, the optimal interval for each network α is different. Taking the Dolphin and PolBooks networks as examples, their optimal intervals are 0.5≤α≤0.8 and 0.05≤α≤0.2, respectively. The difference in the optimal interval is related to the level of community structure in the network. Generally, the community hierarchy of different networks is not the same. Therefore, the optimal interval of α for different networks should also be different.

(9) Multi-label classification

Furthermore, we further evaluate the performance of our method on multi-label classification tasks. To facilitate the comparison of our method with the other three methods, we used the following experimental steps. Specifically, we randomly select a portion of the vertices as the training set and the rest as the test set. Then a logistic classification model implemented with LibLinear returns the label with the highest probability. We repeated the above process 50 times and then averaged the Micro-F1 and Macro F1 scores. We conducted this experiment on the BlogCatalog network and set the parameter α=1. To speed up the training of multi-label classifiers, we choose a smaller training set on the BlogCatalog network, ranging from 1% to 9%. Table 4 and Table 5 show the Micro F1 and Macro F1 on BlogCatalog, respectively. Bold numbers indicate the highest score in each column. As can be seen from Table 4 and Table 5, according to the scores of Micro-F1 and Macro F1, the Q-Walker algorithm outperforms the other three methods.

Table 4: Micro F1 Score on BlogCatalog

Table 5: Macro F1 Scores on BlogCatalog

The specific embodiments described above further describe the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (7)

1. A network representation learning method with community structure, characterized in that, comprising the following steps: Step 1: Data collection and processing phase: use a density function to obtain vertex sequence samples S={s ₁ ,s ₂ ,...,s _n } by using a random walk strategy on the network G; Step 2: Data representation learning stage: optimize the Skip-gram model, use the Skip-gram model to train vertex sequence samples S={s ₁ , s ₂ ,...,s _n }, and obtain the vector representation of each vertex sequence; Step 3: Data calculation stage: Calculate the similarity of the vector representation of each vertex sequence to obtain the similarity of community division. 2. A network representation learning method with community structure according to claim 1, characterized in that, in the step 1, in the vertex sequence sample S={s ₁ , s ₂ ,...,s _n } The vertex sequence is denoted as s={v1,v2...,v|s|}. 3. a kind of network representation learning method with community structure according to claim 2 is characterized in that, The density function in step 1 is defined as:

in

and

are the sum of the internal and external degrees of all vertices in the vertex sequence s, respectively, and α is the resolution parameter used to control the size of the community; The density function also has a density gain _Δf ^v _s , ^which should satisfy the following formula: Δf _s =f _s+{v} -f _s where the symbol s+{v} represents the new vertex sequence obtained by moving vertex v to s. 4. a kind of network representation learning method with community structure according to claim 3, is characterized in that, the concrete step that obtains vertex sequence sample in described step 1 is: Step 11: Randomly select a vertex v _|s|+1 from the set N'(v _|s| ); Step 12: Calculate Δf _s ^v|s|+1 according to the formula Δf _s =f _s+{v} -f _s ; Step 13: If Δf _s ^v|s|+1 <0, delete v _|s|+1 from the set N'(v _|s| ), and then return to step 11; Step 14: If Δf _s ^v|s|+1 >0, add v _|s|+1 to the set s, and mark v _|s|+1 as the current vertex; where vertex v _|s| is the last added vertex, let the last added vertex v _|s| be the current vertex; N′(v _|s| ) represents the set of all the neighbor vertices of the current vertex v _|s| that are not in s; Repeat steps 11 to 14 until the density of the vertex sequence s cannot be increased. 5. a kind of network representation learning method with community structure according to claim 2, is characterized in that, in described step 2, Skip-gram model trains vertex sequence samples by minimizing following objective function:

where t is the window size, v _j is the vertex representation of _{vi in the context network within the window, and the probability p(v j |} vi ₎ in the above formula is defined as

where Φ(s) represents the embedding vector of s, Φ′(s) represents the context vector, and s represents the vertex sequence set. 6. a kind of network representation learning method with community structure according to claim 1, is characterized in that, in described step 3, carrying out similarity calculation to the vector representation of each vertex sequence specifically comprises: for each vertex in the network The vector representation of the sequence calculates the degree of similarity between it and the vector representation of other vertex sequences. Specifically, the NMI formula is used to calculate the similarity. 7. A network representation learning device with community structure, characterized in that it comprises: The data collection and processing module is used to read the vertex sequence samples and obtain the vertex sequence samples S={s ₁ , s ₂ ,...,s _n }; The data representation learning module is used to optimize the Skip-gram model. The Skip-gram model is used to train vertex sequence samples S={s ₁ , s ₂ ,...,s _n } to obtain the vector representation of each vertex sequence; The similarity calculation module is used to calculate the similarity of the vector representation of each vertex sequence to obtain the similarity of community division.

Images