CN104994170B - Distributed clustering method based on hybrid cytokine analysis model in sensor network - Google Patents

Abstract

The invention discloses a distributed clustering method based on a mixed factor analysis model in a sensor network. The method uses a mixed factor analysis model to model the data to be clustered at each node in the sensor network, and each node calculates local sufficient statistics based on its own data. Then the quantity is spread and broadcasted to its neighbor nodes. When the node receives all the local sufficient statistics from the neighbor nodes, it can obtain the joint sufficient statistics, and based on the statistics, estimate each factor in the mixed factor analysis model. parameters, and finally complete the clustering based on the estimated model. The present invention establishes a mixed factor analysis model to complete data dimensionality reduction while clustering, adopts a distributed clustering mode, and avoids network collapse caused by a central node in a traditional centralized processing mode. In the distributed clustering method of the present invention, sufficient statistics are transmitted between nodes instead of data, which not only greatly saves communication overhead, but also better protects private information in data.

Description

A Distributed Clustering Method Based on Mixed Factor Analysis Model in Sensor Networks

technical field

The invention relates to a distributed clustering method based on a mixed factor analysis model in a sensor network, and belongs to the technical field of data parallel distributed processing methods and applications.

Background technique

The sensor network is composed of a large number of stationary or mobile miniature sensor nodes deployed in the monitoring area, and its single sensor node has very limited ability to collect, store, process and transmit data. Therefore, for data processing in sensor networks, traditional data processing needs to be improved. At present, there are mainly two ways of data processing in sensors, centralized processing and distributed processing. In the centralized processing method, one of the nodes is designated as the central node, and other nodes transmit and summarize the collected raw data to the central node, complete the data processing at the central node, and then return the processing results to each node. The disadvantage of this method is that once the central node fails, it will have a fatal impact on the entire network. Another processing method is distributed processing. In this method, all nodes have the same status, and through communication and cooperation among neighboring nodes, the data processing task is finally completed. Compared with centralized data processing, distributed processing can avoid the adverse effects caused by the failure of the central node, and the robustness of the entire network is stronger. And the present invention can well solve the above problems.

Contents of the invention

The purpose of the present invention is to solve the defects of the prior art, and propose a distributed clustering method based on a mixed factor analysis model in a sensor network. Clustering refers to the process of dividing data into multiple classes by a certain method. A class generated by clustering is a collection of data objects that are similar to objects in the same cluster and different from objects in other clusters. Since in clustering, the class label to which data belongs is unknown, so in the field of machine learning, clustering data is an unsupervised learning process. There are many existing data clustering methods, but most of them assume that the clustering of all data is completed on a processing center, and in sensor networks, distributed processing is very critical, so this method is just to solve this problem , to design a distributed clustering method based on a mixed factor analysis model. Its main advantages are: (1) The mixed factor analysis model can effectively process high-dimensional data; (2) By designing the cooperation mode between nodes, only the intermediate results can be transmitted to obtain satisfactory clustering results, compared with the original data transmission method , which not only reduces the communication overhead, but also protects the privacy information on the data and ensures the data security in the network.

The technical scheme adopted by the present invention to solve the technical problems is: a distributed clustering method based on a mixed factor analysis model in a sensor network, the method comprising the following steps:

Suppose there are M sensor nodes in the sensor network, and the mth node collects N _m data, expressed as Among them, y _m,n represents the nth data at node m, and the dimension is p. Use the mixed factor analysis model (MFA) to describe the distribution of Y _m (m=1,...,M), and note that the data of all nodes share the same MFA. MFA is a mixed model with component number I; for each data y _m,n , it can be expressed as:

y _m,n =μ _i +A _i u _m,n +e _m,n,i with probability π _i (i=1,...,I),

Among them, μ _i is the p-dimensional mean value vector of the i-th mixed component, u _m,n is the factor in the low-dimensional space corresponding to the data y _m,n , and its dimension is q (q<<p), Obey the Gaussian distribution N(u _m,n |0,I _q ), the value of q is selected according to the size of p in the specific problem, generally any integer between q=p/6～p/2; A _i is ( p×q) factor loading matrix; error variables e _m,n,i obey Gaussian distribution N(e _m,n,i |0,D _i ), where D _i is a diagonal matrix of (p×p); probability π _i satisfies Then the parameter set Θ of the MFA is {π _i , A _i , μ _i , D _i } _i=1,...,I . Note that for all nodes, each parameter value in the MFA parameter set to be estimated is the same.

In addition, the data transmission range of each node is set to W. For the current node m, all nodes whose distance is less than W are its neighbor nodes, and the set of neighbor nodes of node m is denoted as R _m . Figure 1 shows the relationship between nodes in a sensor network, where a circle represents a node, and if two nodes are connected by an edge, it means that the two nodes can communicate with each other and transmit information. The dotted box in Fig. 1 represents _Rm for m of the node. In the present invention, the network topology has been determined before the implementation of distributed clustering, and it is necessary to ensure that any two nodes communicate directly or after multiple hops.

After the sensor network topology of the present invention and the MFA describing data distribution are established, the distributed clustering process starts, as shown in Figure 2, and its specific steps include:

Step 1: Initialization; there are M sensor nodes in the sensor network, and the mth node collects N _m data, expressed as Among them, y _{m, n} represent the nth data at node m, and the dimension is p; the network topology has been determined in advance, and the set of neighbor nodes of node m is represented as R _m ; use mixed factor analysis model (MFA) to describe Y _m (m =1,...,M), the data of all nodes share the same MFA; the parameter set of MFA is {π _i ,A _i ,μ _i ,D _i } _i=1,...,I , where π _i is the weight of the i-th mixed component, A _i is the (p×q) factor loading matrix of the i-th mixed component, and q is the dimension of the low-dimensional factor, which is between q=p/6～p/2 Any integer of ; μ _i is the p-dimensional mean vector of the i-th mixed component, and D _i is the covariance matrix of the error of the i-th mixed component;

First, set the mixture fraction I in MFA, which is also the number of categories to be clustered; set the initial value of each parameter in MFA according to I, p and q; among them, the Randomly select from the data collected by the node, and Each element in is generated from the standard normal distribution N(0,1); in addition, each node l broadcasts the number of data N _l it collects to its neighbor nodes; when a node m receives it After the number of data broadcasted by all neighbor nodes l(l∈R _m ), the node calculates the weight c _lm according to the following formula:

After the initialization is completed, the iteration counter iter=1, and the iteration process starts;

Step 2: Local calculation; at each node l, based on the collected data Y _l , first calculate the intermediate variables g _i , Ω _i , and < _zl,n,i >, (n=1,..., _Nl ; i=1,...,I):

in, The parameter value obtained after the previous iteration is completed (the initial value of the parameter in the first iteration <z _l,n,i > indicates the probability that the nth data y _l,n at node l belongs to the i-th class (mixed component);

Next, the node computes the local sufficient statistics include:

Step 3: broadcast diffusion; each node l in the sensor network broadcasts the calculated local sufficient statistic LSS _l to its neighbor nodes;

Step 4: Joint calculation; when node m(m=1,...,M) receives LSS _l from all its neighbor nodes l(l∈R _m ), node m calculates joint sufficient statistics

Step 5: Estimate parameters; node m (m=1,...,M) estimates Θ={π _i ,A _{i ,} μ _i ,D _i } _{i=1,. ..,I} , where {π _i ,μ _i } _i=1,...,I, the estimation process is as follows:

For the estimation of {A _i ,D _i } _i=1,...,I , the process is as follows:

Step 6: Judgment convergence; node m (m=1,...,M) calculates the logarithmic likelihood value under the current iteration:

If logp(Y _m |Θ)-logp(Y _m |Θ ^old )<ε, it converges and stops iteration; otherwise, execute step 2 and start the next iteration (iter=iter+1); where Θ indicates that the current iteration estimates , Θ ^old represents the parameter value estimated in the last iteration, that is, the logarithmic likelihood value of two adjacent iterations is less than the threshold ε, and the algorithm converges; ε takes any value from 10 ^-5 to 10 ^-6 ; Since each node in the network processes data in parallel, it is impossible to converge at the same time in one iteration; when node l has converged but node m has not yet converged, node l will no longer send LSS _l , and will no longer receive LSS l transmitted by neighbor nodes. information; node m updates its CSS _m with the last received LSS _l from node l; unconverged nodes continue to iterate until all nodes in the network converge;

Step 7: Clustering output; after step 1-step 6, node m (m=1,...,M) gets each data Corresponding <z _m,n,i >(n=1,...,N _m ; i=1,...,I), will <z _m,n,i >(i=1,.. .,I) The serial number corresponding to the maximum value is the class C _m,n to which y m _,n is finally assigned, namely:

Get the clustering results of all data on all nodes in this way

The present invention uses the mixed factor analysis model to model the data to be clustered at each node in the sensor network. Each node calculates local sufficient statistics based on its own data, and then spreads the amount to its neighbor nodes. When the node receives all After local sufficient statistics of neighbor nodes, it can obtain joint sufficient statistics, and estimate each parameter in the mixed factor analysis model based on the statistics, and finally complete clustering based on the estimated model. The mixed factor analysis model established by the present invention can complete data dimensionality reduction while clustering, and adopts a distributed clustering method, which avoids the network collapse caused by the central node in the traditional centralized processing method. In addition, in this In the distributed clustering method invented, sufficient statistics are transmitted between nodes instead of data, which not only greatly saves communication overhead, but also better protects private information in data, making the system security using this method greatly improved. Increase.

Beneficial effect:

1. The mixed factor analyzer used in the present invention can reduce the dimensionality of high-dimensional data, thereby successfully completing clustering while reducing dimensionality, and obtaining better clustering performance.

2. The distributed clustering method based on the mixed factor analysis model adopted in the present invention enables each node in the sensor network to fully utilize the information contained in the data of other nodes, and the clustering performance is better than the centralized method.

3. The distributed clustering method based on the mixed factor analysis model used in the present invention, in the process of node cooperation, exchanges local sufficient statistics instead of directly transmitting original data, because the quantity and dimension of local sufficient statistics are much smaller than data , so on the one hand, this method saves the communication overhead, on the other hand, it is beneficial to fully protect the private information in the data, and improves the security performance of the system using this method.

Description of drawings

FIG. 1 is a schematic diagram of the neighbor node set R _m of node m in the sensor network of the present invention, and local sufficient statistics (ie: LSS) sent and received between nodes.

Fig. 2 is a flow chart of the distributed clustering method based on the mixed factor analysis model in the sensor network involved in the present invention.

FIG. 3 is a schematic diagram of data clustering results at each node in an embodiment of the present invention.

Detailed ways

The invention will be described in further detail below in conjunction with the accompanying drawings.

In order to better illustrate the distributed clustering method based on the mixed factor analysis model in the sensor network involved in the present invention, it is applied to the clustering of wine component data. In some countries, there are some testing stations distributed in different regions for testing the content of various components in wine. The type of wine sent to the testing station varies. Therefore, it is necessary to cluster wines of similar categories. If the detection station can form a sensor network with other detection stations, through mutual cooperation, it can make full use of the wine data in other detection stations, thereby improving the clustering accuracy. Here, the wine data to be clustered comes from the UCI machine learning database. There are 178 data here, which come from 3 categories in total. The dimension of each data is 13, indicating the content of each component in the wine. There are 8 nodes in the sensor network, the average number of neighbor nodes of each node is 2, and the network is connectable (there is a direct or indirect path between any two nodes). So in this example, M=8, p=13, I=3, q=3. In addition, the amount of data at each node: N ₁ =21, N ₂ =22, N ₃ =21, N ₄ =21, N ₅ =22, N ₆ =22, N ₇ =21, N ₈ =28; Neighbor nodes of each node: R ₁ ={3,5,6}, R ₂ ={3,5}, R ₃ ={1,4,2}, R ₄ ={3}, R ₅ ={1, 2}, R ₆ ={1,7}, R ₇ ={6,8}, R ₈ ={3,7}.

According to the process of the content of the invention (shown in Figure 2), start distributed clustering:

(1) Initialization: Set the initial value of the parameters in the MFA. Among them, at each node randomly selected from the node's data, and Each element in is generated from a standard normal distribution N(0,1). In addition, each node l (l=1, . . . , M) broadcasts the number N _l of collected data to its neighbor nodes. When a node m receives the number of data broadcasted by all its neighbor nodes, the node calculates the weight c _lm :

The meaning of the weight is to measure the importance of the information transmitted by each neighbor node l(l∈R _m ) of the node m at the node m. After the initialization is completed, the iteration counter iter=1, and the iteration process starts.

(2) Local calculation: This step does not require the information of neighbor nodes. At each node l, based on the collected data Y _l , first calculate g _i , Ω _i , and < _zl,n,i >, (n=1,..., _Nl ; i=1,...,I):

in The parameter value obtained after the previous iteration is completed (the initial value of the parameter in the first iteration <z _l,n,i > represents the probability that the nth data y _l,n at the node l belongs to the i-th class (mixture component).

Second, the nodes compute local sufficient statistics as follows:

(3) Broadcast diffusion: Each node l in the sensor network broadcasts the calculated local statistics LSS _l to its neighbor nodes, as shown in Figure 1.

(4) Joint calculation: when node m (m=1,...,M) receives LSS _l from all its neighbor nodes l(l∈R _m ), node m calculates joint sufficient statistics

(5) Estimated parameters: node m (m=1,...,M) estimates Θ={π _i ,A _{i ,} μ _i ,D _i } _{i=1,. ..,I} , where {π _i ,μ _i } _i=1,...,I, the estimation process is as follows:

For the estimation of {A _i ,D _i } _i=1,...,I , the process is as follows:

(6) Judgment convergence: node m (m=1,...,M) calculates the logarithmic likelihood value under the current iteration:

If logp(Y _m |Θ)-logp(Y _m |Θ ^old )<ε, converge and stop iteration; otherwise, execute step (2) and start the next iteration (iter=iter+1); where Θ represents the current iteration Estimated parameter value, Θ ^old represents the parameter value estimated in the last iteration, that is, the logarithmic likelihood value of two adjacent iterations is less than the threshold ε, and the algorithm converges; ε takes any value from 10 ^-5 to 10 ^-6 It is worth noting that since each node in the network processes data in parallel, it is impossible to converge at the same time in one iteration; for example, when node l has converged but node m has not yet converged, node l will no longer send LSS _l , and no longer receive information transmitted by neighbor nodes; node m updates its CSS _m with the last received LSS l sent by node _l ; unconverged nodes continue to iterate until all nodes in the network converge.

(7) Cluster output. After steps (1)-(6), node m (m=1,...,M) obtains <z corresponding to each of its data {y _m,n } _{n=1,...,Nm m,n,i} >(n=1,...,N _m ; i=1,...,I), will <z _m,n,i >, i=1,...,I The serial number corresponding to the maximum value is used as the class C _m,n to which y _m,n is finally assigned, namely:

Get the clustering results of all data on all nodes in this way

Performance evaluation:

Will adopt the result that the clustering method involved in the present invention obtains Compared with the correct generic results, the effectiveness and accuracy of the method involved in the present invention can be evaluated and measured. The clustering results at each node are shown in Figure 3. The abscissa of the figure represents 178 data, the non-vacant position represents the node to which the data is assigned, and the ordinate represents the category number to which the data is assigned (a total of 3 kind). In this figure, "o" indicates correctly clustered data and "x" indicates incorrectly clustered data. From Figure 3, the clustering accuracy at 8 nodes is: 100%, 100%, 95.2%, 95.5%, 100%, 95.5%, 100%, 92.9%. In total, only five data were incorrectly clustered, and the average correct rate of the entire network was 97.2%. Compared with the result (98%) obtained by the centralized transmission method, the correct rate is basically the same. The shortcomings of centralized transmission are very obvious. First, once the central node fails, the entire network will collapse; second, each node directly transmits the original data to the central node, which not only increases the communication burden in the network, but also easily leaks Privacy in data. Therefore, the above disadvantages are overcome by adopting the method of the present invention, and good distributed clustering performance is obtained.

The scope of protection claimed in the present invention is not limited only to the description of this specific embodiment.

Claims (2)

1. the distributed clustering method based on mixed factor analysis model in the sensor network, it is characterized in that, described method comprises the steps: Step 1: Initialization; there are M sensor nodes in the sensor network, and the mth node collects N _m data, expressed as Among them, y _{m, n} represent the nth data at node m, and the dimension is p; the network topology has been determined in advance, and the set of neighbor nodes of node m is represented as R _m ; use mixed factor analysis model (MFA) to describe Y _m (m =1,...,M), the data of all nodes share the same MFA; the parameter set of MFA is {π _i ,A _i ,μ _i ,D _i } _i = ₁ ,..., _I , where π _i is the weight of the i-th mixed component, A _i is the (p×q) factor loading matrix of the i-th mixed component, and q is the dimension of the low-dimensional factor, which is between q=p/6～p/2 Any integer of ; μ _i is the p-dimensional mean vector of the i-th mixed component, and D _i is the covariance matrix of the error of the i-th mixed component; First, set the mixture fraction I in MFA, which is also the number of categories to be clustered; set the initial value of each parameter in MFA according to I, p and q; among them, the Randomly select from the data collected by the node, and Each element in is generated from the standard normal distribution N(0,1); in addition, each node l broadcasts the number of data N _l it collects to its neighbor nodes; when a node m receives it After the number of data broadcasted by all neighbor nodes l(l∈R _m ), the node calculates the weight c _lm according to the following formula: After the initialization is completed, the iteration counter iter=1, and the iteration process starts; Step 2: Local calculation; at each node l, based on the collected data Y _l , first calculate the intermediate variables g _i , Ω _i , and <z _l , _{n, i} >, (n=1,...,N _l ; i=1,...,I): in, It is the parameter value obtained after the previous iteration is completed, and it is the initial value of the parameter at the first iteration <z _l,n,i >indicates the probability that the nth data y _l,n at node l belongs to the i-th class mixture component; Next, the node computes the local sufficient statistics include: ; Step 3: broadcast diffusion; each node l in the sensor network broadcasts the calculated local sufficient statistic LSS _l to its neighbor nodes; Step 4: Joint calculation; when node m(m=1,...,M) receives LSS _l from all its neighbor nodes l(l∈R _m ), node m calculates joint sufficient statistics Step 5: Estimate parameters; node m (m=1,...,M) is calculated according to the previous step , estimated , where {π _i ,μ _i } _{i=1,..., the estimation process of I} is as follows: For the estimation of {A _i ,D _i } _i=1,...,I , the process is as follows: Step 6: Judgment convergence; node m (m=1,...,M) calculates the logarithmic likelihood value under the current iteration: If logp(Y _m |Θ)-logp(Y _m |Θ ^old )<ε, it converges and stops iteration; otherwise, execute step 2 and start the next iteration (iter=iter+1); where Θ indicates that the current iteration estimates , Θ ^old represents the parameter value estimated in the last iteration, that is, the logarithmic likelihood value of two adjacent iterations is less than the threshold ε, and the algorithm converges; ε takes any value from 10 ^-5 to 10 ^-6 ; Since each node in the network processes data in parallel, it is impossible to converge at the same time in one iteration; when node l has converged but node m has not yet converged, node l will no longer send LSS _l , and will no longer receive LSS l transmitted by neighbor nodes. information; node m updates its CSS _m with the last received LSS _l from node l; unconverged nodes continue to iterate until all nodes in the network converge; Step 7: Clustering output; after step 1-step 6, node m (m=1,...,M) gets each data Corresponding <z _m,n,i >(n=1,...,N _m ; i=1,...,I), will <z _m,n,i >(i=1,.. .,I) The serial number corresponding to the maximum value is the class C _m,n to which y m _,n is finally assigned, namely: Get the clustering results of all data on all nodes 2. The distributed clustering method based on the mixed factor analysis model in the sensor network according to claim 1, characterized in that: the method is applied to the clustering of wine component data.