CN104462565A - Community extraction method based on approximate equivalence structure - Google Patents

Abstract

The invention discloses a community extraction method. The method mainly comprises the three phases that firstly, a social network is constructed into an undirected graph and expressed with a transaction data set, and then the concept of approximate equivalence structure is proposed; secondly, a cosine pattern tree is constructed according to the approximate equivalence structure, a cosine pattern is excavated from the transaction data set based on a CP growth algorithm, and the approximate equivalence structure in the social network is extracted; lastly, community extraction is realized by integrating existing algorithms. By means of the method, low time and space complexity is achieved, nodes unrelated to a community are reduced, coordination between efficiency and quality of community extraction is achieved, efficient extraction of the social network is realized, and the method has great significance in the community extraction field.

Description

A kind of community's abstracting method based on approximately equivalent structure

Technical field

The invention belongs to Data Mining, relate to a kind of community abstracting method, specifically a kind of community's abstracting method based on approximately equivalent structure.

Background technology

Wellman proposed the concept of comparatively ripe community network in 1988, namely " community network be by some individuality between the metastable system that forms of social relationships ", " network " is considered as a series of social relationships (Social Relations) connecting actor (Actor), and its metastable pattern forms social structure (Social Structure).Along with the continuous expansion of range of application, the concept of community network has surmounted the scope of interpersonal relation, and the actor (Actor) of network not only comprises individual, and can be set unit, as family, department, tissue.In recent years, community network obtains and develops rapidly, data volume also sharply increases thereupon, by original several thousand, a few general-purpose family billions of users till now, therefore, be necessary to simplify community network, namely realize community and extract, thus network structure can be understood better, for user, businessman provide better service and create larger value.The object that society extracts extracts strongly connected user in community network, abandons the user of weak rigidity simultaneously.The people such as Yunpeng Zhao were published in " PNAS " paper in 2011 method proposed based on tabu search extracts the community in community network, the method can be good at extracting closely-coupled community, but poor effect when the method extracts intensive community network.The people such as Seok-Ho Yoon were published in " Information Sciences " paper in 2012 proposes the community extracted based on manual intervention in blog, and its process extracts the closely-related blog community with the blog user of artificial selection by multiple technologies.The method can be good at extracting specific community, but cannot extract communities all in community network.This patent proposes the community's abstracting method based on equivalent construction on the basis of above-mentioned paper, extracts for community network community.

The concept of community network equivalent construction starts from the classical paper that Lao Rui and White deliver.They think, actor (Actor) role in a network of structure equity is identical, and they have identical experience or chance, therefore, can exchange between these actors (Actor); Borgatti etc. then define automorphism equivalence: for two point u and v in a signed graph G, after interlocking point u and v, and the isomorphic graphs that a little also can be formed after again being given label, just claims have automorphism equivalence at these 2; Rule thought of equal value is put forward by Sai Le at first, and it requires that a class actor has similar relation between another kind of actor.But above-mentioned traditional structural equivalence condition is too strict, and automorphism Sum fanction equivalence of equal value is too wide in range, the present invention proposes the concept of approximately equivalent structure.

In approximately equivalent structure extraction, FP growth algorithm is a kind of conventional mining mode algorithm, but there are the following problems is for this algorithm: 1) when Mining Frequent Patterns, it needs recursively formation condition FP to set, and often produce a frequent mode and will generate a condition FP tree; 2) when support threshold is less, even if for very little database, also ten hundreds of frequent modes will be produced.Dynamic generation and discharge ten hundreds of condition FP tree, by the Time and place of at substantial; 3) FP tree and condition FP tree need top-down generation, and the excavation of frequent mode needs bottom-up process.Because recursively formation condition is set, therefore FP tree and condition FP tree must two-wayly travel through.Like this, the node of FP tree and condition FP tree just needs more pointer, thus needs larger memory headroom to safeguard FP tree and condition FP tree.

Summary of the invention

In view of the harmony of community's detection efficiency and quality is lower, the object of this invention is to provide a kind of community's abstracting method based on approximately equivalent structure, first community network is configured to a figure by the method, shows with affairs data set table; Then propose the concept of approximately equivalent structure, and build cosine mode tree with this; Then excavate cosine mode by CP growth algorithm from affairs data centralization, extract the approximately equivalent structure in community network; Last integrated existing algorithm realization community extracts.Comprise:

1) data prediction: mainly say that the figure in community network user data structure represents.

2) propose the concept of approximately equivalent structure and build cosine mode tree: the definition such as average specific G, approximately equivalent structure mainly proposing common neighbours, structural equivalence, common neighbours, and proposing the method extracting approximately equivalent structure, particular content is shown in embodiment.

3) the approximately equivalent structure in community network is extracted: excavate cosine mode by CP growth algorithm from affairs data centralization, extract the approximately equivalent structure in community network, give and from arbitrary network, to extract approximate peering structure be equivalent to and concentrate from corresponding Transaction Information the proof excavating cosine mode, and demonstrate the condition antimonotone of cosine mode.

4) integrated existing algorithm realization community extracts: mainly based on the set of the existing cosine mode tree of excavating, then realize community by existing clustering method and extract.

The present invention has lower Time & Space Complexity, decreases community without articulation point, solves the harmony problem of efficiency and the quality extracted community, achieves the efficient decimation of community network, have great importance in extraction field, community.

Accompanying drawing explanation

Fig. 1 is overall framework figure of the present invention;

Fig. 2 is the example of the data set ended up with D based on depth-first traversal strategy use CP hedge clipper branch;

Fig. 3 is Enron, Gowalla data set features, wherein | V| represents the set of network node, | E| represents the set of connection;

Fig. 4 is the relation curve between the nodes that extracts under three kinds of partition tools respectively of Enron data set and modularization Q, and wherein horizontal ordinate is the nodes extracted, and ordinate is corresponding Q value;

Fig. 5 is the relation curve between the nodes that extracts under three kinds of partition tools respectively of Gowalla data set and modularization Q, and wherein horizontal ordinate is the nodes extracted, and ordinate is corresponding Q value.

Embodiment

Below in conjunction with the drawings and specific embodiments, accompanying method of the present invention is further illustrated.The present invention is based on community's abstracting method of approximately equivalent structure, comprising: first according to certain rule, social networks is configured to a non-directed graph, shows with affairs data set table; Then propose the concept of approximately equivalent structure, build cosine mode tree, and excavate cosine mode based on CP growth algorithm from affairs data centralization, extract approximately equivalent structure; Last integrated existing algorithm realization community is detected.The following institute of specific implementation method is not:

Step 1: data prediction

Represented by community network figure G=(V, E), wherein G represents whole network, and V represents the node set of network, i _p, i _q∈ V, represent i _padjoint point collection, E represents the articulation set that there is annexation between node.Non-directed graph G also can represent with transaction data set (TDS) D, wherein the every corresponding node of row, and the project of this row is all the neighbours of this node, namely

Step 2: build cosine mode tree

Before realization builds cosine mode tree algorithm, need to the task dispatching related definition of equal value of the average specific G of common neighbours, structural equivalence, common neighbours, approximately equivalent structure, extraction approximately equivalent structure, and prove to obtain cosine similarity tool condition antimonotone, namely cosine similarity can be used for beta pruning, thus lays a good foundation for the excavation of cosine mode.Specifically comprise:

¹) common neighbours' definition: V represents the set of node of network G, i _p, i _q, i _r∈ V, if meet i _p, i _rand i _q, i _rbe interconnective node between two, then claim i _rit is node i _pand i _qcommon neighbours.

2) structural equivalence definition: arbitrary node collection Y={i ₁..., i _{| Y|}, if meet or

$\frac{\left|{\∩}_{q=1}^{\left|Y\right|}{N}_{i_q}\right|}{\left|N_{i_1}\right|}=...=\frac{\left|{\∩}_{q=1}^{\left|Y\right|}{N}_{i_q}\right|}{\left|N_{i_{\left|Y\right|}}\right|}=1---\left(1\right)$

Then set of node Y is claimed to be structure equity.

Order characterize node i _pcommon neighbours' ratio.

3) the average specific G of common neighbours defines: assuming that the definition expression formula of the average specific of the common neighbours of any node collection Y (/Y/ >=2), set of node Y is as follows:

$G\left(Y\right)=\sqrt[\left|Y\right|]{\underset{i_p\∈Y}{\mathrm{\Π}}{r}_{i_p}}---\left(2\right)$

In formula, obviously, G ∈ [0,1], G is larger, and social compactedness is larger.For filtering out the less node of the degree of association in community network further, variable is proposed: wherein, n=|V| is the node total amount of community network.

4) approximately equivalent structure definition: if set of node Y meets G (Y)>=τ _gand F (Y)>=τ _f, wherein, τ _g, τ _f∈ [0,1] is given threshold value, then claim set of node Y to be approximately equivalent structure.

Work as τ _gwhen=1, approximately equivalent structure just changes equivalent construction into, and this can be derived by formula (1) and formula (2) and obtain.Therefore, approximately equivalent structure should regard the not proper equivalent construction of one as, along with τ _gincrease, the gap of approximately equivalent structure and equivalent construction will reduce gradually.

5) task of equal value of approximately equivalent structure is extracted: assuming that use transaction data set (TDS) D to represent a network, a given set of node Y, if node i _parbitrary neighbor node of set of node Y, namely then Y inherently appears at i _pitem affairs $T_{i_p\∈D}$ In, therefore, $|N_{i_1}\∩...\∩N_{i_{\left|Y\right|}}|=\left|\right\{T_{i_p}|Y\⊆T_{i_p},T_{i_p}\∈D\}|=\mathrm{\σ}\left(Y\right),$ Wherein, σ (Y) is the support of Y in D.Therefore be the support of Y.So be easy to derive:

$G\left(Y\right)=|N_{i1}\∩...N_{i\left|Y\right|}|/\sqrt[\left|Y\right|]{\underset{i_p\∈Y}{\mathrm{\Π}}\left|{\mathrm{Ni}}_p\right|}=\mathrm{\σ}\left(Y\right)/\sqrt[\left|Y\right|]{\underset{i_p\∈Y}{\mathrm{\Π}}\mathrm{\σ}\left(\right\{i_p\left\}\right)}=s\left(Y\right)/\sqrt[\left|Y\right|]{{\mathrm{\Π}}_{p=1}^{\left|Y\right|}\left|s\left(\right\{i_p\left\}\right)\right|}=\cos \left(Y\right).$

If s (Y)>=τ _s, cos (Y)>=τ _c, wherein, τ _sand τ _cbe the minimum threshold of support and cosine similarity respectively, can obtain: to given threshold tau _gand τ _f, work as τ _s=τ _fand τ _c=τ _gtime, from arbitrary network, the approximate peering structure of extraction is equivalent to and excavates cosine mode from corresponding Transaction Information is concentrated.

6) condition antimonotone (OAMP) definition: in order to excavate cosine mode, the most direct method checks set of node in order according to the definition of approximately equivalent structure, but the too complicated O (N2 of this method ^n-1).Another kind method comprises two steps: one is the set of excavating F ', makes f (Y)>=τ _f, be equivalent to Mining Frequent Patterns; Two is that F ' is improved to F, makes g (Y)>=τ _g.This method is far superior to the most direct method, because the first step finds the process that its essence of F ' is exactly classical Frequent Pattern Mining, uses the antimonotone pruning set of node infrequently of F, significantly can improve excavation speed.

This method does not adopt the concept of cosine similarity, because it does not have antimonotone as its support, as to G beta pruning, namely when finding that Y is not cosine mode, all supersets that can not draw it are not thus cosine mode, will cause the waste of a large amount of computational resource.Therefore, the efficient cosine mode based on antimonotone excavates, and the condition antimonotone of The present invention gives is defined as follows:

Make I be generic items collection, suppose if if Y ' Y ≠ φ, i _p' ∈ Y ' Y, s ({ i _p)≤s ({ i _p'), there is M (Y)>=M (Y '), then claim M to have condition antimonotone.

Condition antimonotone can think a kind of special shape of antimonotone, namely any one there is the tolerance of antimonotone must tool condition antimonotone, but conversely may not be correct, as cosine mode.Compared with antimonotone, condition antimonotone also requires that all items in difference collection (Y ' Y) must have higher support than the project in subset (Y).

7), cosine similarity has condition antimonotone proves: generally, assuming that Y={i ₁..., i _pa P item collection (P>=1), Y '=Y ∪ { i _p+1..., i _p+L(P+L) item collection (L>=0), and s ({ i _p+l)>=s ({ i _p), 1≤l≤L, 1≤p≤P.

As L=0, there is Y '=Y, i.e. cos (Y)=cos (Y '); Present hypothesis L ≠ 0, Y ' Y ≠ φ, be easy to obtain s (Y) >=s (Y ').

According to the definition of geometric mean, have:

$\sqrt[P]{\underset{p=1}{\overset{P}{\mathrm{\Π}}}s\left(\right\{i_p\left\}\right)}\≤\sqrt[P+L]{\underset{p=1}{\overset{P+L}{\mathrm{\Π}}}s\left(\right\{i_p\left\}\right)}---\left(3\right)$

Due to s ({ i _p+l)>=s ({ i _p), 1≤l≤L, 1≤p≤P, just has:

$\cos \left(Y\right)=\frac{s\left(Y\right)}{\sqrt[P]{\underset{p=1}{\overset{P}{\mathrm{\Π}}}s\left(\right\{i_p\left\}\right)}}\≥\frac{s\left(Y^{\′}\right)}{\sqrt[P+L]{\underset{p=1}{\overset{P+L}{\mathrm{\Π}}}s\left(\right\{i_p\left\}\right)}}=\cos \left(Y^{\′}\right)---\left(4\right)$

8), cosine mode tree definition: with item integrate X as suffix pattern cosine mode tree be the tree construction meeting following condition: (1) forms with the root node of " null ", item prefix subtree and an item of interest head table by one; (2) node in item prefix subtree comprises three territories: LD item, quantity, chained list node; (3) item head table, when taking X as suffix pattern, likely derives the item of cosine mode, i any one of it _pmeet: wherein D _xfor the condition pattern of X amasss.

Concrete structure cosine mode tree method is as follows:

1) if suffix pattern X=is φ, run-down Transaction Information D, obtains frequent item set F, and F is formed FList by the sequence of support descending series;

2) if suffix item collection X ≠ φ, scan current cosine mode tree, the condition pattern building X amasss D _x.Remove D _xin meet { i _p| i _p∈ D _x, s ({ i _p)>=τ _s, cos (X ∪ { i _p)>=τ _citem;

3), create CP and set Tree _xroot node, mark with symbol " null ";

4), to D _xin each affairs , it is pressed FList _xorder inserts cosine mode tree Tree _x;

Step 3: excavate cosine mode based on CP growth algorithm

CP growth algorithm is similar to FP growth algorithm, but CP growth algorithm no longer distinguish single path tree and multichannel through set.In FP growth algorithm, when condition FP tree is single path tree, as long as FP growth algorithm lists all combination of nodes simply as frequent item set, and do not need further subtree mapper, this is determined by the reverse monotonicity of support.But cosine tolerance only has conditioned reflex monotonicity, CP growth algorithm only can not list combination of nodes as cosine mode in single path tree, and must carry out subtree mapper.

Choose one group of data instance to be described, as shown in Figure 2.First project must be chosen in the affairs from G to A.Initial CP tree is identical with FP tree, is positioned at the lower left of Fig. 2.CP growth algorithm adopts DF-SAT strategy to access successively with the item collection of A to G ending, this ensure that the validity of the reverse monotonicity of cosine condition metric.Make τ _c=0.6, τ _s=0, existing hypothesis needs to excavate the cosine mode with D ending.Obviously, at Tree _{{ D}}head table in have 3 projects: E, F and G.CP tree algorithm is accessed successively, and { E, D}, { F, D} are with { G, D} find to only have cos{F, D}=0.67 > τ _c, therefore, the CP tree building D, as shown in Fig. 2 upper right side, only comprises a node F.But the condition FP tree of D is then containing 4 nodes, and this shows that the performance of CP tree in data compression expression is obviously better than FP tree.Specific algorithm process is as follows:

1) cosine mode tree Tree, is built _x;

2), Tree is worked as _xduring ≠ φ, if X ≠ φ, to Tree _xevery i in head table _p, F=F ∪ { { i _p∪ X};

3), to Tree _xevery i in head table _pgenerate candidate pattern, X '={ i _p∪ X;

4), recursive call utilizes CP growth algorithm to produce cosine mode set.

Step 4: integrated existing algorithm realization community extracts

The present invention mainly adopts three kinds of Clustering tools to verify the validity that the community based on cosine mode extracts, comprise based on text cluster instrument CLUTO, figure Clustering tool METIS of the classical clustering method of k-means and based on modularization optimization method, integrated layered cluster policy tool FN (Fast Newman).

Beneficial effect

Choose two kinds of social networks in experiment: Enron and Gowalla data set, its feature as shown in Figure 3.Enron data set comprises the data from about 150 users (being safe senior executive mostly), is the data about E-mail communication; Gowalla data set is a network about friends, derives from the social networks being called Gowalla based on geographic position, and user shares the geographic position of oneself by registration.

Adopt three kinds of community detection methods to verify in experiment and the validity that the community based on cosine mode extracts comprise CLUTO, METIS, FN.The validity of community's extraction algorithm judges mainly through modularization Q, and modularization Q is defined as follows:

Modularization Q defines: Q is used to the validity verifying that in community's extraction, cosine mode excavates.Computing method are as follows:

$Q_S=\underset{S}{\mathrm{\Σ}}(\frac{m_s}{E}-{\left(\frac{d_s}{2\left|E\right|}\right)}^2)---\left(5\right)$

In formula, m _sfor community S internal edges total amount, for the node number of degrees total amount of S.Q is larger, illustrates that excavation performance is better.

Fig. 4 and Fig. 5 respectively describes the relation curve between nodes and modularization Q that Enron data set and Gowalla data set extract respectively under three kinds of Clustering tools.In experiment, make τ _g=0.5, increase τ gradually _f, can find out that from Fig. 4 and Fig. 5 the nodes of extraction will correspondingly decline.Fig. 4 and Fig. 5 is the result that three kinds of Clustering tools act directly on initial network, mainly plays the effect of comparing here.

Can obtain two conclusions from Fig. 4 and Fig. 5: the first, no matter to take which kind of Clustering tool, it is in fact all the improvement showing to excavate performance by Q value that the community based on cosine mode extracts.The Q value that FN obtains at initial Gowalla network is in Figure 5 less than 0.4, but after using cosine mode to excavate, Q value increases to 0.85 rapidly, shows that the community based on cosine mode extracts the excavation performance effectively improving system; The second, less node is not likely show one relation more clearly.In the diagram, when extracting the quantity of node and being reduced to about 7000, the Q value that three kinds of Clustering tools are corresponding is all similar to and reaches maximal value.Work as τ _fduring further increase, the node be drawn into is fewer and feweri, and the performance that cosine mode excavates is then poorer.This shows if τ _farrange too high, just may miss much meaningful, ND approximately equivalent structures, finally cause cosine mode to excavate the reduction of performance.

Claims (4)

1., based on community's abstracting method of approximately equivalent structure, comprising:

1), the concept of approximately equivalent structure is proposed;

2) equivalence of approximately equivalent structure and cosine mode, is proved;

3), build cosine mode tree, and excavate cosine mode based on CP growth algorithm from affairs data centralization, extract the approximately equivalent structure in community network.

2. the community's abstracting method based on approximately equivalent structure according to claim 1, it is characterized in that, described step 1) in, the approximately equivalent structural condition proposed both had not had traditional structural equivalence condition too strict, did not have again the shortcoming that automorphism Sum fanction of equal value is of equal value too wide in range.

3. the community's abstracting method based on approximately equivalent structure according to claim 1, is characterized in that, described step 2) in, the equivalence of approximately equivalent structure and cosine mode, makes to extract approximately equivalent structure and can be converted into excavation cosine mode.

4. the community's abstracting method based on approximately equivalent structure according to claim 1, it is characterized in that, described step 3) in, designed cosine mode tree and CP growth algorithm, in Time and place efficiency, be more better than traditional FP tree and FP growth algorithm.