CN104994170B

CN104994170B - Distributed clustering method based on hybrid cytokine analysis model in sensor network

Info

Publication number: CN104994170B
Application number: CN201510414218.6A
Authority: CN
Inventors: 魏昕; 周亮; 周全; 陈建新; 王磊; 赵力
Original assignee: Nanjing Post and Telecommunication University
Current assignee: Nanjing Tian Gu Information Technology Co ltd; Information and Telecommunication Branch of State Grid Jiangsu Electric Power Co Ltd
Priority date: 2015-07-15
Filing date: 2015-07-15
Publication date: 2018-06-05
Anticipated expiration: 2035-07-15
Also published as: CN104994170A

Abstract

The invention discloses the distributed clustering methods based on hybrid cytokine analysis model in sensor network, this method models data to be clustered at each node in sensor network with hybrid cytokine analysis model, each node is based on its data and calculates local sufficient statistic, amount diffusion is then broadcast to its neighbor node, after node receives all local sufficient statistics from neighbor node, it can obtain joint sufficient statistic, and the parameters in hybrid cytokine analysis model are estimated based on the statistic, it is based ultimately upon the model estimated and completes cluster.The present invention establishes hybrid cytokine analysis model can complete the dimensionality reduction of data while cluster, using Distributed Cluster mode, avoid the periods of network disruption brought in traditional centralized processing mode by Centroid.In distributed clustering method of the present invention, what is transmitted between each node is sufficient statistic rather than data, and communication overhead is not only greatly saved, but also can preferably protect the privacy information in data.

Description

Distributed clustering method based on mixed factor analysis model in sensor network

Technical Field

The invention relates to a distributed clustering method based on a mixed factor analysis model in a sensor network, and belongs to the technical field of parallel distributed processing methods and application of data.

Background

The sensor network is composed of a large number of stationary or mobile miniature sensor nodes deployed in a monitoring area, and the capacity of collecting, storing, processing and transmitting data of each sensor node is very limited. Therefore, for data processing in sensor networks, improvements to conventional data processing are needed. Currently, there are two main ways of data processing in sensors, centralized processing and distributed processing. In a centralized processing mode, one node is designated as a central node, other nodes transmit and gather the acquired original data to the central node, the data processing is completed at the central node, and then the processing result is returned to each node. The disadvantage of this approach is that the entire network is fatally affected once the central node fails. Another approach is distributed processing. In the method, all nodes have the same status, and the data processing task is finally completed through communication and cooperation among the neighbor nodes. Compared with centralized data processing, distributed processing can avoid adverse effects caused by failure of the central node, and the robustness of the whole network is stronger. The present invention can solve the above problems well.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a distributed clustering method based on a mixed factor analysis model in a sensor network, wherein clustering refers to a process of dividing data into a plurality of classes by a certain method. The class generated by a cluster is a collection of a set of data objects that are similar to objects in the same cluster and distinct from objects in other clusters. Since the class label to which the data belongs is unknown in the clustering, the clustering of the data is an unsupervised learning process in the field of machine learning. The existing data clustering methods are many, but most of the existing data clustering methods assume that all data clustering is completed on one processing center, and in a sensor network, distributed processing is very critical, so that the method is designed based on a mixed factor analysis model to solve the problem. Its advantages mainly include: (1) the mixed factor analysis model can effectively process high-dimensional data; (2) by designing a cooperation mode among nodes, a satisfactory clustering result can be obtained only by transmitting an intermediate result, and compared with a mode of transmitting original data, the method reduces the communication overhead, protects the privacy information on the data and ensures the data security in the network.

The technical scheme adopted by the invention for solving the technical problems is as follows: a distributed clustering method based on a mixed factor analysis model in a sensor network comprises the following steps:

if M sensor nodes exist in the sensor network, the mth node collects N_mThe number of the data is one,is shown asWherein y is_m,nRepresenting the nth data at node m with dimension p. Description of Y with Mixed factor analysis Model (MFA)_mDistribution of (M1...., M), note that the data of all nodes share the same MFA. MFA is a mixed model with component I; for each data y_m,nIt can be expressed as:

y_m,n＝μ_i+A_iu_m,n+e_m,n,iwith probability pi_i(i＝1,...,I)，

Wherein, mu_iIs the p-dimensional mean vector of the ith mixture component, u_m,nIs and data y_m,nCorresponding factor in low dimensional space with dimension q (q < p) obeying Gaussian distribution N (u)_m,n|0,I_q) The value of q is selected according to the size of p in the specific problem, and is generally an arbitrary integer between p/6 and p/2; a. the_iA factor load matrix of (p × q); error variable e_m,n,iObeying Gaussian distribution N (e)_m,n,i|0,D_i) Wherein D is_iA diagonal matrix of (p × p); probability pi_iSatisfy the requirement ofThen the parameter set Θ of the MFA is { π }_i,A_i,μ_i,D_i}_i＝1,...,I. Note that the values of the individual parameters in the MFA parameter set that it is to estimate are the same for all nodes.

In addition, the data transmission range of each node is set as W, for the current node m, all nodes with the distance smaller than W are neighbor nodes, and the neighbor node set of the node m is represented as R_m. The relationship between nodes in a sensor network is shown in fig. 1, where circles represent nodes, and if two nodes are connected by edges, it means that the two nodes can communicate with each other to transmit information. The dotted boxes in FIG. 1 represent the R of m for a node_m. In the present invention, the netThe network topology is well established before distributed clustering is implemented and interworking between any two nodes is guaranteed either directly or over multiple hops.

After the sensor network topology and the MFA describing the data distribution are established, the distributed clustering process is started, as shown in fig. 2, and the specific steps include:

step 1: initializing; there are M sensor nodes in the sensor network, and the M-th node acquires N_mData, expressed asWherein y is_m,nRepresenting the nth data at the node m with the dimension p; the network topology is determined in advance, and the neighbor node set of the node m is represented as R_m(ii) a Description of Y with Mixed factor analysis Model (MFA)_mA distribution of (M ═ 1.. said., M), the data of all nodes sharing the same MFA; the parameter set of the MFA is { pi_i,A_i,μ_i,D_i}_i＝1,...,IIn which pi_iIs the weight of the ith mixture component, A_iA (p × q) factor load matrix of the ith mixed component, wherein q is the dimension of a low-dimensional factor and is an arbitrary integer between p/6 and p/2; mu.s_iIs the p-dimensional mean vector of the ith mixture component, D_iA covariance matrix which is an error of the ith mixture component;

firstly, setting a mixed fraction I in the MFA, which is also the number of categories to be clustered; setting initial values of parameters in the MFA according to I, p and q; wherein at each node Randomly selected from the data collected by the node,andeach element in (a) is generated from a standard normal distribution N (0, 1); in addition, the number N of data collected by each node l_lBroadcast to its neighbor nodes; when a certain node m receives all its neighbor nodes l (l is belonged to R)_m) After the number of data broadcast, the node calculates the weight c according to the following equation_lm：

After the initialization is completed, the iteration counter iter is equal to 1, and the iteration process is started;

step 2: local calculation; at each node l, based on the data Y it has collected_lFirst, the intermediate variable g is calculated_i，Ω_i，And<z_l,n,i>，(n＝1,...,N_l；i＝1,...,I)：

wherein,for the parameter values obtained after the previous iteration is completed (initial values of the parameters at the first iteration)<z_l,n,i>Represents the nth data y at the node l_l,nProbability of belonging to the ith class (mixture component);

next, the node calculates local sufficient statisticsThe method comprises the following steps:

and step 3: broadcasting diffusion; each node l in the sensor network will calculate the local sufficient statistic LSS_lBroadcasting and diffusing to the neighbor nodes;

and 4, step 4: performing joint calculation; when node M (M ═ 1.. times.m) receives information from all its neighbor nodes l (l ∈ R)_m) LSS of_lThen, the node m calculates the joint sufficient statistic

And 5: estimating parameters; the node M (M1.., M) is calculated according to the CSS calculated in the previous step_mEstimate Θ to be ═ pi_i,A_i,μ_i,D_i}_i＝1,...,IWherein, { pi-_i,μ_i}_i＝1,...,IThe estimation process of (2) is as follows:

for { A_i,D_i}_i＝1,...,IThe process is as follows:

step 6: judging convergence; node M (M ═ 1.., M) computes the log-likelihood value at the current iteration:

if logp (Y)_m|Θ)-logp(Y_m|Θ^old) If the value is less than epsilon, convergence is carried out, and iteration is stopped; otherwise, executing step 2, and starting the next iteration (iter + 1); where Θ represents the value of the parameter estimated for the current iteration, Θ^oldRepresenting the value of the parameter estimated in the last iteration, i.e. the pair of two adjacent iterationsThe number likelihood value is smaller than a threshold value epsilon, and the algorithm converges; taking 10 from epsilon^-5～10^-6Any value of (1); because each node in the network processes data in parallel, all nodes cannot converge at the same time in one iteration; when node l has converged and node m has not, then node l no longer sends LSS_lAnd the information transmitted by the neighbor node is not received any more; the node m uses the LSS sent by the last received node l_lUpdating its CSS_m(ii) a The nodes which are not converged continue to iterate until all the nodes in the network are converged;

and 7: clustering and outputting; after steps 1 to 6, node M (M1.., M) obtains each data thereofCorresponding to<z_m,n,i>(n＝1,...,N_m(ii) a I1.., I), will be<z_m,n,i>The sequence number corresponding to the maximum value in (I ═ 1.. multidot.i) is defined as y_m,nClass C to which it is finally assigned_m,nNamely:

obtaining clustering results of all data on all nodes in this way

The method comprises the steps of modeling data to be clustered at each node in a sensor network by using a mixed factor analysis model, calculating local sufficient statistics by each node based on own data, then broadcasting the local sufficient statistics to neighbor nodes in a diffusion mode, obtaining joint sufficient statistics by the nodes after the nodes receive all the local sufficient statistics from the neighbor nodes, estimating each parameter in the mixed factor analysis model based on the statistics, and finally finishing clustering based on the estimated model. The mixed factor analysis model established by the invention can complete the dimensionality reduction of data while clustering, and a distributed clustering mode is adopted, so that the network collapse caused by a central node in the traditional centralized processing mode is avoided.

Has the advantages that:

1. the mixed factor analyzer adopted in the invention can reduce the dimension of the high-dimensional data, thereby smoothly completing clustering while reducing the dimension and obtaining better clustering performance.

2. The distributed clustering method based on the mixed factor analysis model adopted by the invention enables each node in the sensor network to fully utilize information contained in data of other nodes, and the clustering performance is superior to that of a centralized method.

3. According to the distributed clustering method based on the mixed factor analysis model, local sufficient statistics are exchanged instead of directly transmitting original data in the node cooperation process, and the number and the dimensionality of the local sufficient statistics are far smaller than those of the data, so that the method saves communication overhead on one hand, and is beneficial to fully protecting privacy information in the data on the other hand, and the safety performance of a system adopting the method is improved.

Drawings

FIG. 1 is a set R of neighbor nodes of a node m in a sensor network according to the present invention_mAnd a schematic diagram of locally sufficient statistics (i.e., LSS) transceived between nodes.

FIG. 2 is a flowchart of a distributed clustering method based on a mixed factor analysis model in a sensor network according to the present invention.

Fig. 3 is a schematic diagram of a data clustering result at each node in the embodiment of the present invention.

Detailed Description

The invention is described in further detail below with reference to the accompanying drawings.

In order to better illustrate the distributed clustering method based on the mixed factor analysis model in the sensor network, the method is applied to the clustering of the wine composition data. In some countries, several detection stations are distributed in different areas for detecting the content of the ingredients in the wine. The types of wine sent to the testing station vary. It is therefore desirable to cluster similar classes of wine. If the detection station can form a sensor network with other detection stations, the data of wine in other detection stations can be fully utilized through mutual cooperation, and therefore clustering accuracy is improved. Here, the wine data to be clustered is derived from the UCI machine learning database, where there are 178 data, for a total of 3 classes. The dimension of each datum is 13, and represents the content of each component in the wine. The sensor network has 8 nodes, the average number of neighbor nodes of each node is 2, and the network is connectable (direct or indirect arriving paths exist between any two nodes). Thus in this example, M is 8, p is 13, I is 3 and q is 3. Further, the data amount at each node: n is a radical of₁＝21,N₂＝22,N₃＝21,N₄＝21,N₅＝22,N₆＝22,N₇＝21,N₈28; neighbor nodes of each node: r₁＝{3,5,6},R₂＝{3,5},R₃＝{1,4,2},R₄＝{3},R₅＝{1,2},R₆＝{1,7},R₇＝{6,8},R₈＝{3,7}。

According to the flow of the inventive content (shown in fig. 2), distributed clustering is started:

(1) initialization: initial values of parameters in the MFA are set. Wherein at each node Is randomly selected from the data of the node,andis generated from a standard normal distribution N (0, 1). In addition, each node l (1.,. M) has the number N of data collected by it_lBroadcast to its neighbor nodes. When a node m receives the number of data broadcast by all its neighbor nodes, the node calculates the weight c_lm：

The meaning of the weight is that each neighbor node l (l belongs to R) for measuring the node m_m) The importance of each transmitted message at node m. After the initialization is completed, the iteration counter iter is equal to 1, and the iteration process is started.

(2) Local calculation: this step does not require information of the neighbor nodes. At each node l, based on the data Y it has collected_lFirst, g is calculated_i，Ω_i，And<z_l,n,i>，(n＝1,...,N_l；i＝1,...,I)：

whereinFor the parameter values obtained after the previous iteration is completed (initial values of the parameters at the first iteration)<z_l,n,i>Represents the nth data y at the node l_l,nProbability of belonging to the ith class (mixture component).

Second, the node calculates local sufficient statisticsThe following were used:

(3) broadcasting diffusion: each node l in the sensor network will calculate the local statistic LSS_lThe broadcast is flooded to its neighbor nodes as shown in fig. 1.

(4) Joint calculation: when node M (M ═ 1.. times.m) receives information from all its neighbor nodes l (l ∈ R)_m) LSS of_lThen, the node m calculates the joint sufficient statistic

(5) And (3) estimating parameters: the node M (M1.., M) is calculated according to the CSS calculated in the previous step_mEstimate Θ to be ═ pi_i,A_i,μ_i,D_i}_i＝1,...,IWherein, { pi-_i,μ_i}_i＝1,...,IThe estimation process of (2) is as follows:

for { A_i,D_i}_i＝1,...,IThe process is as follows:

(6) and (4) judging convergence: node M (M ═ 1.., M) computes the log-likelihood value at the current iteration:

if logp (Y)_m|Θ)-logp(Y_m|Θ^old) If the value is less than epsilon, convergence is carried out, and iteration is stopped; otherwise, executing step (2), and starting the next iteration (iter + 1); where Θ represents the value of the parameter estimated for the current iteration, Θ^oldRepresenting the parameter value estimated in the last iteration, namely, the logarithm likelihood value of two adjacent iterations is less than a threshold value epsilon, and the algorithm converges; taking 10 from epsilon^-5～10^-6Any value of (1); it is worth noting that since each node in the network processes data in parallel, it is not possible to converge simultaneously in one iteration; for example, when node l has converged and node m has not, then node l no longer sends LSS_lAnd the information transmitted by the neighbor node is not received any more; the node m uses the LSS sent by the last received node l_lUpdating its CSS_m(ii) a The non-converged nodes continue to iterate until all nodes in the network converge.

(7) And (6) clustering and outputting. After steps (1) - (6), node M (M ═ 1.., M), with which each datum { y } is obtained_m,n}_n＝1,...,NmCorresponding to<z_m,n,i>(n＝1,...,N_m(ii) a I1.., I), will be<z_m,n,i>I is 1, the serial number corresponding to the maximum value in I is used as y_m,nClass C to which it is finally assigned_m,nNamely:

obtaining clustering results of all data on all nodes in this way

Performance evaluation:

the results obtained by the clustering method according to the inventionAnd the correct generic result, so that the effectiveness and accuracy of the method can be evaluated and measured. The clustering result at each node is shown in fig. 3, in which the abscissa indicates 178 data, the non-vacant position indicates the node to which the data is classified, and the ordinate indicates the category number (total 3 categories) to which the data is classified. In the figure, "o" indicates correctly clustered data and "x" indicates incorrectly clustered data. From fig. 3, the clustering accuracy at 8 nodes is: 100%, 100%, 95.2%, 95.5%, 100%, 95.5%, 100%, 92.9%. Only five data in total were clustered by error, with an average accuracy of 97.2% for the entire network. Compared with the result (98%) obtained by adopting the centralized transmission method, the accuracy is basically the same. The disadvantage of centralized transmission is quite obvious, and firstly, once a central node fails, the whole network crashes; secondly, each node directly transmits the original data to the central node, so that the communication burden in the network is increased, and the privacy in the data is easily revealed. Therefore, the method of the invention overcomes the defects and obtains good distributed clustering performance.

The scope of the invention is not limited to the description of the embodiments.

Claims

1. A distributed clustering method based on a mixed factor analysis model in a sensor network is characterized by comprising the following steps:

step 1: initializing; there are M sensor nodes in the sensor network, and the M-th node acquires N_mData, expressed asWherein y is_m,nRepresenting the nth data at the node m with the dimension p; the network topology has been determined in advance, the neighborhood of node mThe set of nodes is denoted R_m(ii) a Description of Y with Mixed factor analysis Model (MFA)_mA distribution of (M ═ 1.. said., M), the data of all nodes sharing the same MFA; the parameter set of the MFA is { pi_i,A_i,μ_i,D_i}_i＝₁,...,_IIn which pi_iIs the weight of the ith mixture component, A_iA (p × q) factor load matrix of the ith mixed component, wherein q is the dimension of a low-dimensional factor and is an arbitrary integer between p/6 and p/2; mu.s_iIs the p-dimensional mean vector of the ith mixture component, D_iA covariance matrix which is an error of the ith mixture component;

step 2: local calculation;at each node l, based on the data Y it has collected_lFirst, the intermediate variable g is calculated_i，Ω_i，And<z_l,_n,i>，(n＝1,...,N_l；i＝1,...,I)：

wherein,the initial value of the parameter is obtained after the previous iteration is completed and is the initial value of the parameter during the first iteration<z_l,n,i>Represents the nth data y at the node l_l,nProbability of belonging to the ith class of mixed components;

；

and step 3: broadcasting diffusion; in a sensor networkEach node l will calculate the local sufficient statistic LSS_lBroadcasting and diffusing to the neighbor nodes;

And 5: estimating parameters; the node M (M1.., M) is calculated according to the previous stepEstimate outWherein, { pi-_i,μ_i}_i＝1,...,IThe estimation process of (2) is as follows:

for { A_i,D_i}_i＝1,...,IThe process is as follows:

if logp (Y)_m|Θ)-logp(Y_m|Θ^old) If the value is less than epsilon, convergence is carried out, and iteration is stopped; otherwise, executing step 2, and starting the next iteration (iter + 1); where Θ represents the value of the parameter estimated for the current iteration, Θ^oldRepresenting the parameter value estimated in the last iteration, namely, the logarithm likelihood value of two adjacent iterations is less than a threshold value epsilon, and the algorithm converges; taking 10 from epsilon^-5～10^-6Any value of (1); because each node in the network processes data in parallel, all nodes cannot converge at the same time in one iteration; when node l has converged and node m has not, then node l no longer sends LSS_lAnd the information transmitted by the neighbor node is not received any more; the node m uses the LSS sent by the last received node l_lUpdating its CSS_m(ii) a The nodes which are not converged continue to iterate until all the nodes in the network are converged;

obtaining the clustering result of all data on all nodes

2. The distributed clustering method based on the mixed factor analysis model in the sensor network according to claim 1, characterized in that: the method is applied to clustering of wine ingredient data.