CN109165664B - Attribute-missing data set completion and prediction method based on generation of countermeasure network - Google Patents

Abstract

The invention discloses a method for completing and predicting a missing attribute data set based on a generative adversarial network, comprising the steps of: 1) normalizing the data minmax, using one hot coding for the attributes of discrete types, and marking the missing value as 0; 2) Use the dataset to establish the missing position encoding vector about the sample; 3) Build a generative adversarial network and an auxiliary prediction network for data filling and label prediction; 4) Restore the result before minmax normalization according to the maximum and minimum values in the attribute 5) Select suitable hyperparameters through testing; the present invention makes full use of data distribution information and label information in the data set, and can effectively fill in the high-dimensional missing data set, and after the training is completed, another method included in the method is used. The auxiliary prediction network can directly give the prediction result of the label according to the input missing attribute data, the process is simple and has a higher prediction accuracy.

Description

A Generative Adversarial Network-based Completion and Prediction Method for Attribute Missing Datasets

technical field

The invention relates to the technical field of data preprocessing, in particular to a method for completing and predicting missing attribute data sets based on generative adversarial networks.

Background technique

The phenomenon of missing data set attributes is widespread in various data sets, and is usually caused by information loss during data collection or transmission. The loss of one or more attributes of the samples in the data set will reduce the prediction accuracy of the subsequent prediction and classification models. How to complete these missing data and use the information contained in the samples with missing attributes to build a high-precision prediction model is a key problem in data preprocessing.

Most statistical tools deal with the problem of missing attributes by deleting the rows and columns corresponding to missing samples, or use the median and mean of the column to fill in the missing positions; although these methods are efficient and convenient, they fail to fully utilize the sample data. distribution information, resulting in inaccurate calculation results. In the process of multi-dimensional data processing, there are often many correlations between different attributes of data, and the correlation between these attributes can provide more information for data filling. There will be less bias in the estimation of missing values, so that the information contained in the missing samples can be deeply mined.

On this basis, further data filling methods are used to fill in missing values by modeling. For example, the regression imputation method uses the missing attribute as the dependent variable to establish a regression equation to achieve prediction. The EM algorithm first initializes the missing value, and the final filling result is obtained through E-step and M-step iteration. The k-neighbor algorithm (KNN) calculates according to the non-missing attributes The Euclidean distance matches the most similar k samples in the sample set, and the filling result is obtained by weighted average. These algorithms often achieve more accurate filling results than the mean and median when the amount of data is sufficient, and then there are usually some problems: in the regression filling method, there needs to be a significant linear relationship between attributes, and based on EM The filling method of the algorithm has high computational complexity and is easy to fall into the local optimum; the filling method based on k-nearest neighbors is simple to implement, but when faced with a large amount of data, the large amount of calculation and the extremely high complexity lead to computational difficulties.

In addition, the main purpose of data filling is to provide more complete data for subsequent modeling predictions. The above method does not involve the modeling process, and the filled data often has some relationship with the predicted label. Combining the prediction model with the filling method can make the filled data have a better prediction effect. Aiming at the two problems of high computational complexity when processing high-dimensional data with traditional data filling methods, the label information cannot be fully mined to correct filling results; The prediction network fully mines the association between data and labels to maximize their mutual information.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a method for completing and predicting missing attribute data sets based on generative adversarial networks, making full use of the data distribution information and label information in the data set, and being able to carry out analysis on high-dimensional missing data sets. Effective data filling, and after the training is completed, another auxiliary prediction network included in the method can directly give the prediction result of the label for the input missing attribute data, the process is simple and has a higher prediction accuracy.

In order to achieve the above purpose, the technical solution provided by the present invention is: a method for completing and predicting a data set with missing attributes based on a generative adversarial network. First, data preprocessing is performed for the data set with missing attributes, mainly including minmax normalization. One hot coding conversion of discrete numerical variables; then, for the samples with missing attributes, the coding vector of the missing position is constructed to express the missing position information; then the filling network of the missing data and the auxiliary prediction network are constructed to complete the filling of the missing data synchronously and label prediction; after the network training is completed, the output result of the generated network in the filling network is used as the filling result, and the scale is restored according to the maximum and minimum values of the columns recorded during minmax normalization; finally, it is observed by continuously modifying the hyperparameters. The loss of the prediction results on the validation set completes the setting of hyperparameters; it includes the following steps

1) Data preprocessing;

2) construct the missing position encoding vector;

3) Construct missing data filling network and auxiliary prediction network;

4) Fill data scale restoration;

5) Testing and hyperparameter setting.

In step 1), different data types are preprocessed differently. The main data types involved are divided into continuous values and discrete values. For continuous values, minmax is directly used for normalization; for discrete values, conversion After encoding one hot, use minmax to normalize, and add 0 to the missing position uniformly; in addition, whether to divide the data set into two parts: data with missing attributes and data with no missing attributes.

In step 2), the missing position encoding vector is constructed, and the situation is: when the data is filled, the missing attribute position of the sample is also an important information. When using the neural network to fill, only these missing positions need to be filled. , when constructing the missing position coding vector, traverse each column of all samples. If the attribute is missing, record it as "1", otherwise, record it as "0". According to this process, each sample will have a missing position code vector correspondence.

In step 3), a missing data filling network and an auxiliary prediction network are constructed, and the situation is that: the network has made the following improvements on the original generative adversarial network: (1) The noise obtained by random sampling is removed from the input of the generative network; ②Use the generated data and the missing position vector coding to form the filled data; in addition, the introduction of the auxiliary prediction network fully considers the relationship between attributes and labels, while using the attribute missing data for prediction, the auxiliary prediction network is used to predict The loss between the label and the real label updates the generation network through the feedback calculation of the BP algorithm, so that the generated filling data has a better effect when building a prediction model; the loss function in the joint generative adversarial network is combined with the auxiliary prediction network. The loss function, which controls its weight ratio through hyperparameters, determines that the distribution of the generated filling data is closer to the distribution of the complete data or can make the prediction model prediction more accurate; among them, the structure of the data filling network and the auxiliary prediction network includes the generation network, discriminant Network, auxiliary prediction network; the following is a detailed introduction to the structure of these three networks:

Generative network: The input part is composed of data with missing attributes and their corresponding missing position encoding vectors; depending on the structure of the data, the hidden layer can be composed of a fully connected layer or a deconvolution layer, especially when the input data is a picture type When using the data, use the deconvolution operation to obtain the generated padding data; here, it is assumed that the input data is denoted as I, which is a 100-dimensional vector, so the corresponding missing position encoding vector is denoted as E, which is also 100-dimensional, and the input obtained by splicing The vector dimension is 200; the hidden layer is composed of a fully connected layer, and the activation function uses relu; the final output layer has 100 output units, denoted as O, and the activation function of the output layer adopts sigmoid; the filled data is finally composed of I·( 1-E)+O·E composition;

Discriminant network: The input data has two parts. The first part is the filling data result based on the output of the generation network, and the second part is the sample data whose attributes are not missing. The output result is a decimal between 0 and 1, which means that the discriminant network thinks The probability of whether the received input data comes from data with no missing attributes; the settings of the network structure are different depending on the type of input data. When the input data is image type data, it is constructed by a convolutional neural network; here, it is assumed that the input data is 100 dimensional vector, then the hidden layer can be selected to be composed of a fully connected layer, and the activation function is set to relu; the output layer only contains one unit, and the activation function is selected to be sigmoid, representing the probability;

Auxiliary prediction network: the input is exactly the same as the discriminant network, and the output is the predicted value of the input sample about the label. When the prediction problem is a classification problem, the cross entropy is used as the loss function, and when the prediction problem is a regression problem, the L2 norm is used Or the L1 norm is used as the loss function; the network structure is the same as the setting method of the discriminant network; here, it is assumed that the input data is a 100-dimensional vector, then the hidden layer can choose to be composed of a fully connected layer, and the activation function is set to relu; the output layer contains only one unit , the activation function is set as above.

In step 4), the generated filling data is scaled. Since minmax is used for data normalization in the preprocessing stage, the final filling result can be restored according to the recorded maximum and minimum values of each attribute. .

In step 5), test and hyperparameter settings, the situation is: in the process of network training, the loss comes from two parts: the loss in the generative adversarial network and the prediction loss of the auxiliary prediction network; the two parts of the loss are different The ratio of λ is combined to obtain a comprehensive loss; different λ will affect the training of the model; during the operation, the data set is divided into training set and test set, and different scales of λ are selected on the training set, which are 0.1, 0.3, 0.5, 0.7, 0.9 are used for training, and at the same time, the test set is used for testing, and the minimum loss of the auxiliary prediction network on the test set is used as the selection criterion of hyperparameters.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The traditional filling methods such as median and mean filling are simple and the filling effect is not good enough. However, the methods based on KNN and EM often have large time complexity. When dealing with high-dimensional data sets, the time complexity is extremely large. There are even situations that cannot be handled. The generative adversarial network has an excellent effect on the distribution learning of high-dimensional data, so it can solve the trouble caused by high-dimensional data sets; in addition, usually the samples without missing attributes and the missing samples with attributes obey the same distribution , so that the filled data approximates the data set with no missing attributes from the distribution, so that the filled results will not deviate from the data distribution, which will have a negative impact on the prediction model.

2. The traditional filling method does not take into account the impact of the filled data on the prediction results of the subsequent prediction model establishment. The steps are usually to fill in the missing data to obtain the completed data, and then use the filled data to establish a prediction model. Therefore, the predicted effect cannot be used to guide the filling of the data. In the present invention, an auxiliary prediction network is introduced to calculate the loss between the predicted value of the data filled each time and the real label, and backpropagation guides the data filling of the generation network, so that the performance of the filled data on the prediction model can be observed and selected. The prediction effect, combined with the loss of the discriminant network, limits the difference between the filled data and the real data distribution, so as to achieve a good filling effect and a good prediction result. In addition, after the training is completed, an end-to-end network is obtained. After inputting the data, the prediction results of the auxiliary prediction network can be directly obtained.

Description of drawings

Figure 1 is a flowchart of missing data filling and prediction.

Figure 2 shows the data flow diagram of the generative adversarial network and prediction network filled with data.

Detailed ways

The present invention will be further described below in conjunction with specific embodiments.

As shown in Figure 1, the method for completing and predicting missing attribute datasets based on generative adversarial networks provided in this example is as follows:

1) Data preprocessing: The data types of different attributes are different, and the corresponding processing methods are also different. The main data types involved are divided into continuous values and discrete values. For continuous values, minmax is directly used for normalization; for discrete values, after conversion to one hot encoding, minmax is used for normalization, and the missing positions are unified. Add 0. In addition, the dataset is divided into two parts: data with missing attributes and data with no missing attributes.

2) Construct the missing position encoding vector: When filling data, the missing attribute position of the sample is also an important information. When filling with neural network, only these missing positions need to be filled. When constructing the missing position encoding vector, traverse each column of all samples, if the attribute is missing, record it as "1", otherwise, record it as "0". According to this process, each sample will have a corresponding missing position encoding vector.

3) Constructing missing data filling network and auxiliary prediction network: The present invention proposes a comprehensive network based on generative adversarial network and joint auxiliary prediction network for data filling and prediction. The network makes the following improvements over the original generative adversarial network: (1) the sampled noise is removed from the input of the generative network; (2) the padded data is composed of the generated data and the missing position vector encoding. In addition, the introduction of the auxiliary prediction network fully considers the relationship between attributes and labels. While using the attribute missing data for prediction, the auxiliary prediction network is used to predict the loss between the label and the real label, and the BP algorithm is used for feedback calculation to update the generation network. , so that the generated padding data has a better effect when building a predictive model. The loss function in the joint generative adversarial network and the loss function in the auxiliary prediction network are controlled by hyperparameters to determine the weight ratio of the generated fill data distribution to be closer to the distribution of the complete data or to make the prediction model more accurate. Fig. 2 is the structure diagram of the most important data filling network and auxiliary prediction network in the present invention, including generating network, discriminating network, auxiliary prediction network; The structure of these three networks is introduced in detail below:

Generative network: The input part consists of the concatenation of data with missing attributes and their corresponding missing position encoding vectors. Depending on the structure of the data, the hidden layer can be composed of a fully connected layer or a deconvolution layer, especially when the input data is image type data, the deconvolution layer is used to obtain the generated padding data. It is assumed here that the input data (denoted as I) is a 100-dimensional vector, so the corresponding missing position encoding vector (denoted as E) is also 100-dimensional, and the dimension of the input vector obtained by splicing is 200; the hidden layer is composed of a fully connected layer , the activation function uses relu; the final output layer has 100 output units (denoted as O), and the activation function of the output layer adopts sigmoid. The padded data is finally composed of I·(1-E)+O·E.

Discriminant network: The input data has two parts. The first part is the filling data result based on the output of the generation network, and the second part is the sample data whose attributes are not missing. The output result is a decimal between 0 and 1, which means that the discriminant network thinks The probability that the received input data is from data whose attributes are not missing. Depending on the type of input data, the settings of the network structure are also different. When the input data is image type data, it can be constructed by a convolutional neural network. It is assumed here that the input data is a 100-dimensional vector, then the hidden layer can be selected to be a fully connected layer, and the activation function is set to relu; the output layer only contains one unit, and the activation function is selected to be sigmoid to represent the probability.

Auxiliary prediction network: the input is exactly the same as the discriminant network, and the output is the predicted value of the input sample about the label. When the prediction problem is a classification problem, the cross entropy is used as the loss function, and when the prediction problem is a regression problem, the L2 norm is used Or L1 norm as loss function. The network structure is the same as the setting method of the discriminant network. Assuming that the input data is a 100-dimensional vector, the hidden layer can be selected as a fully connected layer, and the activation function is set to relu; the output layer only contains one unit, and the activation function is set as described above.

4) Scale restoration of filling data: Since minmax is used for data normalization in the preprocessing stage, the final filling result can be restored according to the recorded maximum and minimum values of each attribute.

5) Test and hyperparameter settings: During the training process of the network, the loss comes from two parts, the loss in the generative adversarial network and the prediction loss of the auxiliary prediction network; the two parts of the loss are combined in different proportions λ to obtain a comprehensive loss . Different λ will affect the training of the model. During the operation, the data set is divided into training set and test set, and λ of different scales is selected on the training set, which are 0.1, 0.3, 0.5, 0.7, 0.9 for training, and at the same time, the test set is used for testing to test The minimum loss of the auxiliary prediction network on the set is used as the selection criterion for the hyperparameters.

The above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made according to the shape and principle of the present invention should be included within the protection scope of the present invention.

Claims (4)

1. A method for complementing and predicting attribute-missing data sets based on a generated countermeasure network is characterized by comprising the following steps: firstly, data preprocessing is carried out on a data set with lost attributes, wherein the data preprocessing comprises minmax normalization and one hot code conversion of discrete numerical variables; then, aiming at the samples with attribute deletion, constructing a coding vector of a deletion position so as to express the deletion position information; then constructing a filling network of missing data and an auxiliary prediction network to synchronously complete the filling and label prediction of the missing data; after the network training is finished, taking an output result of a network generated in a filling network as a filling result, and carrying out scale reduction according to the maximum and minimum values of the columns recorded during minmax normalization; finally, the super-parameters are continuously modified to observe the loss of the prediction result of the super-parameters in the verification set so as to complete the setting of the super-parameters; which comprises the following steps

1) Preprocessing data;

2) constructing a missing position coding vector, the case of which is: when data is filled, the attribute positions of the missing samples are important information, when a neural network is used for filling, only the missing positions need to be filled, when the missing position coding vectors are constructed, each column of all the samples is traversed, if the attribute is missing, the attribute is marked as '1', otherwise, the attribute is marked as '0', the process is executed according to the process, and each sample has a corresponding missing position coding vector;

3) constructing a missing data filling network and an auxiliary prediction network, wherein the conditions are as follows: the network is improved in the original generative countermeasure network as follows: removing noise from the input of the generated network; forming filled data by using the generated data and the missing position vector code; in addition, the introduction of the auxiliary prediction network fully considers the relation between the attributes and the tags, and when the attribute missing data is used for prediction, the loss between the tags and the real tags is predicted by using the auxiliary prediction network, and the generated network is updated by performing feedback calculation through a BP algorithm; jointly generating a loss function in the countermeasure network and a loss function in the auxiliary prediction network, and determining that the distribution of the generated filling data is close to the distribution of the complete data by controlling the weight ratio of the loss functions through the hyper-parameters; the structure of the data filling network and the auxiliary prediction network comprises a generation network, a judgment network and an auxiliary prediction network; the structure of these three networks is described in detail below:

generating a network: the input part is formed by splicing data with attribute missing and missing position coding vectors corresponding to the data; the input data is picture type data, and generated filling data is obtained by using deconvolution operation;

and (3) network discrimination: the input data comprises two parts, wherein the first part is a filling data result obtained based on the output of the generated network, the second part is sample data with non-missing attributes, and the output result is a decimal between 0 and 1 and represents the probability of judging whether the input data received by the network is from the data with non-missing attributes; the input data is image type data, and the network structure of the discrimination network is constructed by a convolution neural network;

auxiliary prediction network: the input is completely consistent with the discrimination network, the output is a predicted value of the input sample about the label, when the prediction problem is a classification problem, cross entropy is adopted as a loss function, and when the prediction problem is a regression problem, an L2 norm or an L1 norm is adopted as the loss function; the network structure is the same as the setting method of the judgment network;

4) filling data scale reduction;

5) and testing and super-parameter setting.

2. The method of claim 1, wherein the method comprises the following steps: in the step 1), different data types are preprocessed differently, the related data types are divided into a continuous numerical value and a discrete numerical value, and the continuous numerical value is normalized by directly using minmax; for discrete numerical values, after being converted into one hot codes, minmax normalization is used, and 0 is uniformly supplemented for missing positions; in addition, the data set is divided into two parts according to whether there is attribute missing: data with missing attributes versus data without missing attributes.

3. The method of claim 1, wherein the method comprises the following steps: in step 4), the generated filling data is subjected to scale reduction, and as the data normalization is performed by using minmax in the preprocessing stage, the final filling result can be obtained through reduction according to the maximum value and the minimum value of each recorded attribute.

4. The method of claim 1, wherein the method comprises the following steps: in step 5), testing and hyper-parameter setting, wherein the conditions are as follows: during the training process of the network, loss is generated from two parts: generating a predicted loss of the antagonistic network and the auxiliary predictive network; the two losses are combined in different proportions lambda to obtain a comprehensive loss; different λ may affect the training of the model; in the operation process, the data set is segmented into a training set and a testing set, lambadas with different scales are selected from the training set and are respectively 0.1,0.3,0.5,0.7 and 0.9 for training, and meanwhile, the testing set is used for testing, and the minimum loss of the auxiliary prediction network on the testing set is used as a selection standard of the hyper-parameter.

Images