CN110334757A - Privacy-preserving clustering method and computer storage medium for big data analysis - Google Patents

Abstract

The invention discloses a privacy protection clustering method and computer storage medium for big data analysis. The method includes the following steps: data normalization and center point selection, calculation of minimum privacy budget and distribution of privacy budget sequence, and division of sample points to the nearest The center point of , generate Laplacian noise, add noise to the parameters in the process of updating the center point, iterate continuously until the difference between the sum of squared errors of two adjacent iterations is less than the threshold or reaches the maximum number of iterations. The present invention protects the sensitive information in the data set by adding noise that obeys the Laplace distribution to the intermediate parameters in the clustering algorithm execution process, solves the problem of leaking sensitive information in the data set during the clustering algorithm execution process, and improves differential privacy The method of protecting the privacy budget allocation of clustering algorithms improves the usability of clustering results under the same degree of privacy protection, and solves the problem of privacy leakage in big data clustering mining.

Description

Privacy-preserving clustering method and computer storage medium for big data analysis

technical field

The invention relates to a privacy protection clustering method and a computer storage medium, in particular to a privacy protection clustering method and a computer storage medium for big data analysis.

Background technique

At present, data mining has been paid more and more attention by people. Using machine learning algorithms to mine and analyze massive data can obtain a lot of valuable new knowledge and new laws. As a commonly used method in the field of data mining, cluster analysis is widely used in scenarios such as data preprocessing, target group classification, pattern recognition, and image segmentation. K-means is the simplest, most effective and most used algorithm in big data clustering analysis. However, during the execution of the algorithm, it is necessary to calculate the number of samples of each cluster and the sum of each attribute when updating the centroid. These operations will leak the data set. Sensitive information.

Differential privacy is a data privacy protection technology that disturbs the data by adding noise while preserving the statistical properties of the data. Therefore, the combination of differential privacy protection technology and clustering algorithm can protect the sensitive information of the dataset from disclosure and obtain relatively accurate clustering results. There are some deficiencies in the existing privacy-preserving clustering algorithms. Random selection of initial points and excessive consumption of privacy budget will lead to unsatisfactory availability of clustering results. In addition, the problem of excessive random noise easily caused by traditional privacy budget allocation is still unsolved.

Contents of the invention

Purpose of the invention: The technical problem to be solved by the present invention is to provide a privacy-preserving clustering method and computer storage medium for big data analysis, which solves the problem that traditional privacy budget allocation easily leads to excessive random noise, thereby affecting the quality of clustering results , improved the privacy budget allocation method of the differential privacy-preserving clustering algorithm, and proposed a differential privacy budget allocation method, which improves the availability of clustering results under the same degree of privacy protection, and solves the privacy problem in big data cluster mining Leakage problem.

Technical solution: the privacy protection clustering method for big data analysis described in the present invention comprises the following steps:

(1) Normalize the data in the data set;

(2) Divide the data set into k subsets on average, and randomly select a sample point in each subset as the initial center point;

(3) Set the total privacy budget ε and the maximum number of iterations t _m , calculate the minimum privacy budget ε ^m and the number of iterations t=ε/ε ^m , if t>t _m , use the equal difference privacy budget allocation method to allocate the privacy budget sequence , if t≤t _m , use the average privacy budget allocation method to allocate the privacy budget sequence, and obtain the privacy budget sequence ε _p , where 1≤p≤t _m ;

(4) For all sample points in the data set, calculate their Euclidean distances to k center points, assign the sample points to the nearest center point, and divide the data set into k clusters C={C ₁ ,C ₂ ,...,C _k };

(5) Generate random numbers of Laplace distribution according to the corresponding items in the privacy budget sequence ε _p ;

(6) For each cluster C _j , where 1≤j≤k, calculate the number of sample points num of the cluster and the sum vector sum of the sample points, and add noise to them respectively to obtain num′ and sum′, the above noise is the step (5) The random number of the Laplace distribution in the middle;

(7) Update the central point of each cluster C _j to be sum'/num', where 1≤j≤k;

(8) Calculate the sum of squared errors. If the absolute value of the difference between the sum of squared errors of this and the previous iteration is less than the set threshold or the number of iterations reaches the upper limit t _m , then end the execution and obtain the clustering result, otherwise go to step 4 and continue Execute the next iteration.

Further, the calculation method of the minimum privacy budget ε ^m in step (3) is:

where N is the number of records in the data set, d is the dimensionality of the data, and ρ is the average value of centroid estimates for each dimension.

Further, the allocation method of the differential privacy budget in step (3) is specifically:

The total privacy budget ε is decomposed into an increasing arithmetic sequence of length t _m , the initial item of the sequence is ε ^m , the sum of all items of the sequence is ε, and the privacy budget sequence ε _p is obtained by inverting the sequence.

Further, the average privacy budget allocation method in step (3) is specifically:

Decompose the total privacy budget ε into an average sequence of length t _m , and the sequence is the privacy budget sequence ε _p .

Further, in step (5), the random number is a Laplace distribution random number that obeys the position parameter of 0 and the scale parameter of b, where b=d+1/ε', d is the dimension of the data, and ε ' is the value of the corresponding position searched from the privacy budget sequence ε _p according to the current iteration number.

Further, the initial central point in step (2) is obtained by randomly selecting a sample point in each subset and adding random noise.

The computer storage medium of the present invention stores a computer program thereon, and when the computer program is executed by a computer processor, the above-mentioned privacy protection clustering method oriented to big data analysis is realized.

Beneficial effects: the present invention has the following technical effects:

1. Use the arithmetic privacy budget allocation method to generate the privacy budget sequence. First calculate the minimum privacy budget ε ^m , and then use the arithmetic sequence summation formula and the general term formula to calculate the privacy budget sequence. The privacy budget sequence is flat and solves the problem There is a problem that the privacy budget in the method is consumed too quickly;

2. Using the arithmetic privacy budget allocation method, the total privacy budget is allocated linearly, which solves the problem that the privacy budget allocated by the existing method is too large in the early stage and too small in the later stage. When the total privacy budget is very small, or even smaller than the minimum privacy budget ε ^m , the present invention adopts an even distribution method to avoid affecting the execution of the algorithm if the allocated privacy budget is too small. Compared with existing methods, the present invention has higher clustering availability and better clustering quality.

Description of drawings

Fig. 1 is the method flowchart of the embodiment of the present invention;

Fig. 2 is a flow chart of the differential privacy budget allocation method of the present invention;

Fig. 3 is a comparison diagram of the clustering usability index of the method of the present invention and the comparative algorithm;

Fig. 4 is a comparison chart of privacy budget sequence results between the method of the present invention and the binary allocation method, series and allocation method.

Detailed ways

The method flowchart of this embodiment is shown in Figure 1, and specifically implemented according to the following steps:

Step 1, the existing Image.csv dataset, which comes from the clustering dataset of the School of Computer Science, University of Eastern Finland (http://cs.joensuu.fi/sipu/datasets/). Record the data set as D, the number of records N in the data set is 34112, and the data dimension d is 3, that is, each piece of data has 3 attributes. The total privacy budget ε controls the degree of privacy protection. The smaller ε is set, the greater the added noise and the higher the degree of privacy protection. Here, the total privacy budget ε is set to 0.8, the number of clusters k is 3, and each piece of data can be regarded as a sample point in the k-dimensional space. Normalize each dimension of the data set D to [0,1].

Data normalization is to scale each dimension of data to [0,1], which is performed by the following formula:

Among them, for any dimension of the data, x is the data of this dimension, min and max are the minimum and maximum values respectively, and x′ is the normalized data.

Step 2. Divide the preprocessed data set D into k subsets {S ₁ , S ₂ ,…,S _k }, randomly select a sample point o _i from each subset S _i , where 1≤i≤ k is the initial center point {u ₁ ,u ₂ ,…,u _k } after adding random noise. Here, the data set D is evenly divided into three subsets {S ₁ , S ₂ , S ₃ }, a sample point is randomly selected from each subset, and the initial center point is obtained after adding noise. The result is:

u ₁ [0 0.08130081 0.00473934]

u ₂ [0.44230769 0.27235772 0.16587678]

u ₃ [0.65384615 0.43089431 0.1943128].

Step 3, obtain the privacy budget sequence ε _p , where 1≤p≤t _m . Set the maximum number of iterations t _m , calculate the minimum privacy budget ε ^m , and then calculate the number of iterations t=ε/ε ^m , if t>t _m , use the arithmetic privacy budget allocation method to allocate the privacy budget sequence; if t <t _m , the average privacy budget allocation method is used to allocate the privacy budget sequence; finally the privacy budget sequence {ε ₁ ,ε ₂ ,…,ε _tm } is obtained. The privacy budget sequence allocation process is shown in Figure 2.

The calculation formula of the minimum privacy budget ε ^m is:

Among them, N represents the number of records in a given data set, d is the number of dimensions, k is the number of clusters, and ρ is the average value of centroid estimates for each dimension. When the data is normalized to [0,1], it takes The value is 0.45.

The arithmetic privacy budget allocation method decomposes the total privacy budget into an increasing arithmetic sequence of length t _m , and each item in the sequence is the privacy budget consumed in the corresponding number of iterations. The specific operation is to use the ε ^m obtained in step 3 as the initial item a ₁ of the arithmetic sequence, and the total privacy budget ε as the sum S _n of all items of the sequence, and the tolerance d _t of the arithmetic sequence can be calculated by the following formula:

a _n =a ₁ +(n-1)d _t ,

After the tolerance d _t is obtained, an increasing arithmetic sequence of length t _m is obtained, and the privacy budget sequence is obtained by reversing the sequence, and the privacy budget sequence may not be completely consumed.

The average privacy budget allocation method is to evenly distribute the total privacy budget according to the maximum number of iterations, and the privacy budget consumed each time is ε/t _m . The average distribution can also be regarded as a special arithmetic distribution with a tolerance of 0.

Specifically, set the maximum number of iterations t _m to 8, calculate the minimum privacy budget ε ^m = 0.031, then t = ε/ε ^m = 25.806, because t>t _m , so use the arithmetic privacy budget allocation method to calculate the privacy budget sequence ε _p , where 1≤p≤8. First calculate the tolerance d _t = 0.0197, then calculate the specific value of each item according to the formula of the general term of the arithmetic sequence, and finally obtain the desired decreasing privacy budget sequence through inversion, the result is {0.169, 0.14928571, 0.12957143, 0.10985714, 0.09014286 , 0.07042857, 0.05071429, 0.031}.

Step 4, for all points in the calculation data set D, calculate the Euclidean distance to k center points respectively, and assign this sample point to the nearest center point, the data set D is divided into k clusters C={ C ₁ ,C ₂ ,...,C _k }.

Specifically, calculate the Euclidean distances from all points in the data set D to the three center points, and assign this sample point to the nearest center point, and the data set D is divided into three clusters C={C ₁ ,C ₂ ,C ₃ }.

Step 5. Calculate the noise to be added in this iteration. The noise is a random number that obeys the Laplace distribution whose position parameter is 0 and scale parameter is b. It is recorded as Lap(b), where b=Δf/ε', Δf represents the sensitivity and ε' is the privacy budget. The probability density function of the Laplace distribution is The sensitivity of the data here is related to the dimension, Δf=d+1, and the privacy budget is ε', which is the value of the corresponding position searched from the privacy budget sequence ε _p according to the current iteration number, so the noise is expressed as Lap(Δf/ε') .

Specifically, according to the number of iterations, find the corresponding privacy budget ε _p from the privacy budget sequence obtained in step 3, and the sensitivity Δf=3+1=4, so in the first iteration, ε ₁ is 0.169, and the noise size is Lap (4/0.169); in the second iteration, ε ₂ is 0.1493, the noise size is Lap(4/0.1493), and so on.

Step 6, for each cluster C _j , where 1≤j≤k, calculate the number of sample points num of the cluster and the sum vector sum of the sample points, and add the noise in step 5 to obtain num' and sum' respectively. Specifically, for each cluster C _j , where 1≤j≤3, the number of sample points num of the cluster and the sum vector sum of the sample points are calculated. The specific result of the first iteration is:

The num of cluster C ₁ is 1406, and the vector sum is [240.29 177.76 107.42];

The num of cluster C ₂ is 12301, and the vector sum is [4665.25 3686.47 2473.31];

The num of cluster C ₃ is 20405, and the vector sum is [13469.21 11385.21 8768.39];

Then add the noise in step 5 to get num' and sum', the noise added in the first iteration is Lap(4/0.169), the specific result is:

The num' of cluster C ₁ is 1421.99, and the vector sum' is [284.77 190.18 108.46];

The num' of cluster C ₂ is 12281.82, and the vector sum' is [4688.87 3697.67 2566.92];

The num' of cluster C ₃ is 20396.29, and the vector sum' is [13466.97 11402.30 8739.17];

Step 7, update the center u _j ′=sum’/num’ of each cluster C _j , where 1≤j≤3; then the specific result of the updated center of the first iteration is:

u ₁ '[0.20026401 0.13374381 0.07627629]

u ₂ '[0.38177298 0.30106816 0.20900154]

u ₃ '[0.66026546 0.55903804 0.42846875].

Step 8: Calculate the sum of squared errors. If the absolute value of the difference between the sum of squared errors of this iteration and the previous iteration is less than the set threshold or the number of iterations reaches the upper limit t _m , then end the execution and obtain the clustering result, otherwise go to step 4 Continue to execute. The sum of squared errors specifically refers to the sum of the distances between the points in each cluster and the center point of this class. The threshold can be set by yourself. The set threshold determines the number of iterations. In theory, it can be set to 0. However, due to the randomness of noise, setting it to 0 will lead to too many iterations. Therefore, the threshold can be appropriately relaxed. Here, it is set to 100.

Compare the method of this embodiment with the two existing algorithms. For different ε values, the three algorithms were run 10 times, and the F-measure index was calculated by using their results and the results of the standard K-means algorithm to evaluate the clustering usability of the algorithm. The value range of F-measure is [0,1], the closer to 1, the more similar the clustering result of the algorithm is to the standard noise-free result, indicating the higher availability of clustering. The comparison chart of the F-measure indicators of the three algorithms on the Image dataset is shown in Figure 3.

FIG. 4 is a comparison diagram of the method of this embodiment and the two existing methods for allocating privacy budget sequences. In the early iterations, the existing two methods have already consumed most of the total privacy budget, and the privacy budget allocated in the middle and late stages is very small. Too small privacy budgets will easily lead to a lot of noise and affect the algorithm convergence. However, the privacy budget sequence allocated by the method of the present invention is linearly distributed, and the privacy budget allocated in the middle period is relatively sufficient, and it is not easy for excessive noise to interfere with the convergence of the algorithm.

The present invention is a privacy protection clustering method oriented to big data analysis. The method improves the privacy budget allocation method of the existing differential privacy clustering algorithm, uses the arithmetic privacy budget allocation method, and solves the excessive consumption of the privacy budget in the existing method. , too much noise in the later stage of iteration, and the usability of clustering results is improved under the same degree of privacy protection. The present invention can be applied to the process of cluster analysis of big data, during which personal information is protected from being disclosed. For example, when clustering and mining medical data, commercial consumption data, and location data, etc., these data contain a large amount of user privacy. Using the method of the present invention can effectively prevent privacy leaks in the process of data collection and algorithm execution, while retaining data Statistical properties and mining utility.

If the embodiment of the present invention is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for Make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media capable of storing program codes such as U disk, mobile hard disk, read only memory (ROM, Read Only Memory), magnetic disk or optical disk. Thus, examples of the invention are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the present invention also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the aforementioned privacy-preserving clustering method for big data analysis can be realized. For example, the computer storage medium is a computer readable storage medium.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Claims (7)

1. A privacy protection clustering method for big data analysis, characterized in that, comprising the following steps: (1) Normalize the data in the data set; (2) Divide the data set into k subsets on average, and randomly select a sample point in each subset as the initial center point; (3) Set the total privacy budget ε and the maximum number of iterations t _m , calculate the minimum privacy budget ε ^m and the number of iterations t = ε/ε ^m , if t>t _m , use the equal difference privacy budget allocation method to allocate the privacy budget sequence , if t≤t _m , use the average privacy budget allocation method to allocate the privacy budget sequence, and obtain the privacy budget sequence ε _p , where 1≤p≤t _m ; (4) For all sample points in the data set, calculate the Euclidean distances to k center points respectively, assign the sample points to the nearest center point, and divide the data set into k clusters C={C ₁ ,C ₂ ,...,C _k }; (5) Generate random numbers of Laplace distribution according to the corresponding items in the privacy budget sequence ε _p ; (6) For each cluster C _j , where 1≤j≤k, calculate the number of sample points num of the cluster and the sum vector sum of the sample points, and add noise to them respectively to obtain num′ and sum′, the above noise is the step (5) The random number of the Laplace distribution in the middle; (7) Update the central point of each cluster C _j to be sum'/num', where 1≤j≤k; (8) Calculate the sum of squared errors. If the absolute value of the difference between the sum of squared errors of this and the previous iteration is less than the set threshold or the number of iterations reaches the upper limit t _m , then end the execution and obtain the clustering result, otherwise go to step 4 and continue Execute the next iteration. 2. the privacy protection clustering method facing big data analysis according to claim 1, is characterized in that, the calculation method of minimum privacy budget ε ^m in step (3) is: where N is the number of records in the data set, d is the dimensionality of the data, and ρ is the average value of centroid estimates for each dimension. 3. the privacy protection clustering method facing big data analysis according to claim 1, is characterized in that, the differential privacy budget allocation method in step (3) is specifically: The total privacy budget ε is decomposed into an increasing arithmetic sequence of length t _m , the initial item of the sequence is ε ^m , the sum of all items of the sequence is ε, and the privacy budget sequence ε _p is obtained by inverting the sequence. 4. the privacy protection clustering method facing big data analysis according to claim 1, is characterized in that, the average privacy budget allocation method in step (3) is specifically: Decompose the total privacy budget ε into an average sequence of length t _m , and the sequence is the privacy budget sequence ε _p . 5. The privacy-preserving clustering method for big data analysis according to claim 1, characterized in that: in step (5), the random number is random according to the Laplace distribution where the position parameter is 0 and the scale parameter is b. number, where b=d+1/ε', d is the dimension of the data, and ε' is the value of the corresponding position searched from the privacy budget sequence ε _p according to the current iteration number. 6. The privacy-preserving clustering method for big data analysis according to claim 1, characterized in that: the initial central point in step (2) is obtained by randomly selecting a sample point in each subset and adding random noise. 7. A computer storage medium, on which a computer program is stored, wherein the computer program implements the method according to any one of claims 1 to 6 when executed by a computer processor.