CN108564166A - Based on the semi-supervised feature learning method of the convolutional neural networks with symmetrical parallel link - Google Patents

Abstract

The invention discloses a semi-supervised feature learning method based on a convolutional neural network with symmetrical cross-layer connections, comprising the following steps: generating a damaged image data set without class labels; constructing a cross-layer connection convolutional neural network; pre-training the image restoration neural network ; Extract network parameters to build a classification network; train the classification network. The invention utilizes the recovery task of image data without class labels to pre-train neural networks, thereby improving the classification effect of images with class labels and realizing semi-supervised feature learning. In addition, by adding symmetric cross-layer connections to the traditional convolutional neural network autoencoder, the network is easier to optimize, and the feature abstraction ability of the middle layer of the network is enhanced, so that the network weight obtained from the unsupervised image restoration task can be transferred to the supervised learning task more easily. The invention realizes an efficient and accurate semi-supervised learning method based on convolutional neural network, and thus has high practical value.

Description

A Semi-supervised Feature Learning Method Based on Convolutional Neural Networks with Symmetrical Cross-Layer Connections

technical field

The invention relates to image semi-supervised feature learning, in particular to a semi-supervised feature learning method based on a convolutional neural network (Convolutional Neural Network, CNN) with symmetric cross-layer connections.

Background technique

With the continuous rapid development of information technology, various fields are generating various types of image data at an astonishing speed every day. In the process of acquiring and disseminating a large amount of image data, how to better understand the semantic information of the image and use it to complete the tasks that humans can complete is an important challenge in the field of artificial intelligence and pattern recognition today. People urgently hope that computers can help humans better acquire and utilize massive image data.

Image data on the Internet often exists in a form without class labels, and only a small amount of structured data or image data used for scientific research has class labels. Therefore, how to use a large amount of unlabeled data to assist the understanding and learning of a small amount of labeled data has become an urgent problem in the field of artificial intelligence. As an important method of using unlabeled data, the semi-supervised feature learning of images has been widely concerned by industry and academia, and is often used as an important topic of various image-related international academic conferences. It is a very important topic in the field of artificial intelligence and pattern recognition. important research topic. The basic idea is to use the structural information extracted from the unlabeled image and use certain technical means to associate the unlabeled information with the features of the labeled data, thereby assisting the understanding and learning of the labeled image.

In recent years, methods based on deep neural networks, especially deep convolutional neural networks, have been widely used in many computer vision and pattern recognition tasks, and have achieved remarkable results in many high-level image understanding tasks, such as image classification, image segmentation, etc. Effect. However, there are still some shortcomings that restrict its application, one of which is that it requires a large amount of class-labeled image data. In the case of limited amount of labeled data, the performance of deep learning methods is often unsatisfactory. How to apply the idea of semi-supervised feature learning to the field of deep learning has become a current research hotspot and plays an important role in actively promoting the process of social informatization. While creating irreplaceable social value, there are still many key technical issues in this field that have not yet been resolved, and there are still many functional realizations that need to be further improved. Therefore, how to use deep convolutional neural networks more effectively in semi-supervised It is of far-reaching significance to understand the image to realize the research of computer vision more flexibly.

Contents of the invention

Purpose of the invention: the technical problem to be solved by the present invention is to provide a kind of semi-supervised feature learning method based on the convolutional neural network with cross-layer connection for the deficiencies in the prior art. The network is pre-trained to eventually improve performance on labeled data.

In order to solve the above-mentioned technical problems, the present invention discloses a semi-supervised feature learning method based on a convolutional neural network (Convolutional Neural Network, CNN) with cross-layer connections, comprising the following steps:

Step 1, generate unlabeled and labeled data sets: collect labeled and unlabeled image data, perform random cropping and normalization processing on each image, and obtain the labeled image set X ₀ and unlabeled image set X 0 The image set Y, according to the image resolution, destroys the images in the set Y differently, and obtains the destroyed image set X ₁ without class labels. Let Z be the class label vector of images with class labels, Z={z ₁ ,z ₂ ,...,z _n }, z _i represents the i-th image class label, and the value of i is 1~n, then (X ₁ ,Y) constitutes a non-label training data set for unsupervised pre-training, (X ₀ , Z) as a set of labeled training data for supervised training;

Step 2, build a pre-trained image recovery network: build an image recovery network according to the size of the input image, set the total depth of the network to be D layers, D is an even number, where the first D/2 layer is a convolutional layer, and the last D/2 layer is a deconvolution layer In the product layer, the size of the convolution kernel is 3x3, and the step size is 1 or 2. The rate of change of the step size is determined according to the depth of the network and the size of the image. The input is the image in the image set X ₁ destroyed in step 1, and the output is the image after the network restores;

Step 3, training image restoration network: use ADAM (Kingma, Diederik P., and Jimmy Ba."Adam:A method for stochastic optimization."arXiv preprint arXiv:1412.6980(2014).) Optimization algorithm, using the training obtained in step 1 Set (X ₁ , Y) to train the network constructed in step 2, take the damaged image in the set X ₁ as input, and use the corresponding lossless image in the set Y as the network supervision information, record the image after training and restore the network before D/ 2 layers of weight W and bias b for each layer;

Step 4, build a supervised classification network, use the image restoration network built in step 2 as a template, and build a D/2 layer network according to the size of the input image, all of which are convolutional layers, and the step size change is consistent with the network built in step 2. And add the Max-pooling layer and Softmax layer, and initialize the parameters of the convolutional layer using the corresponding weight W and bias b of the network trained in step 3;

Step 5, train the classification network, use the ADAM optimization algorithm to train the classification network constructed and initialized in step 4 on the image data with class labels until the algorithm converges.

Step 1 specifically includes the following steps:

Step 1-1, collecting image data with and without class labels, cutting each image, and using random cutting to obtain image blocks of the same size, where the size of the image block depends on the size of the original image and the size of the model, for For low-resolution images smaller than 50*50 (such as the CIFAR-10 dataset), the crop size is 29*29, and for natural images with high resolution larger than 225*225 (such as the PASCAL VOC dataset), the crop size is 225*225 , if the resolution is between the two, first scale to a similar resolution and then crop. Record all image collections after cropping as X';

Step 1-2, normalize and centralize the cropped image blocks, first calculate the mean and standard deviation of each pixel on the cropped image data set X', set the mean image of all images on X' as The standard deviation is std. For a specific image x, it is normalized and centered as follows:

x' is the processed image of image x; in the processed image, the set of images with class labels is marked as X ₀ , and the set of images without class labels is marked as Y.

Step 1-3, for the labeled image, the processed image set X ₀ and the corresponding labeled vector Z form the labeled training data (X ₀ , Z), Z={z ₁ ,z ₂ ,…,z _n }, z _i represents the class label of the i-th image.

Steps 1-4, destroy the images in the unlabeled image set Y, add Gaussian noise or set the pixel value in the image to 0, if it is a low-resolution image after cropping (resolution less than 50*50), then The method of adding Gaussian noise is adopted. If the cropped image is a high-resolution image (resolution greater than or equal to 50*50), the pixel point is set to 0, and the pixel point set to 0 is randomly selected from 10 adjacent 8*8 areas. , to obtain the destroyed unlabeled image set X ₁ , which forms the unlabeled training data set (X ₁ , Y) with the unlabeled image set Y.

Step 2 specifically includes the following steps:

Step 2-1, set the total depth of the image restoration network to be D layers, D is an even number, where the first D/2 layer uses a convolution layer, and the last D/2 layer uses a deconvolution layer, the convolution kernel size is 3x3, and the step size is is 1 or 2, the step size of every k layer is 1, the step size of the k+1 layer is 2, 0<k<D/2-1, repeat n times. The k and n sizes are determined by the network depth and image block size: for 29*29 low-resolution images, k=4, n=3, for 225*225 high-resolution images, k=2, n=5, in each layer A BatchNormalization layer and a ReLU (Rectified Linear Unit) nonlinear layer are added after the convolutional layer and the deconvolutional layer. The input of the network is the image in the destroyed image set X ₁ generated in step 1, and the output is the restored image of the network. The network parameters include the weight W and bias b of the convolutional layer and the deconvolution layer, and the weight γ and bias β of the BatchNormalization layer.

Step 2-2, add symmetrical cross-layer connections between the convolutional layer and the deconvolution layer every two layers: let CO _i represent the output of the i-th convolutional layer, and DI _i represent the input of the i-th deconvolutional layer , DO _i represents the output of the i-th deconvolution layer, then the cross-layer connection is expressed as:

DI _D-i+1 = DO _Di + CO _i ,

Then DI _D+1 is the network output, CO ₀ is the network input, the first cross-layer connection starts from the input layer and connects to the output layer, the final layer input of the network and the corresponding image in the lossless image set Y calculate the Euclidean distance as the subsequent network training loss function

Where X ⁱ is the i-th image in the image set X ₁ , Y ⁱ is the i-th image in the image set Y, is the function represented by the neural network, N is the number of training images, θ is all trainable parameters of the network, including the weight W and bias b of the convolutional layer and the deconvolution layer, and the weight γ and bias β of the BatchNormalization layer.

Step 3 specifically includes the following steps:

Step 3-1, use the ADAM optimization algorithm to train the neural network with gradient backpropagation, the learning rate is set to 1e-4, and the training lasts for n ₁ rounds (generally 20 rounds), and at n ₂ rounds (generally the 8th round) and after the n _3rd round (generally the 16th round), the learning rates are set to 1e-5 and 1e-6 respectively;

Step 3-2, to illustrate the step of gradient backpropagation with cross-layer connections, set the depth of the image restoration network to 7 layers, add cross-layer connections in the manner in step 2-2, let X ₀ be the network input, and _Xi be the first The output of the i-layer convolutional layer, the cross-layer link specifically connects X ₁ to the input of layer 5, and connects X ₀ to the input of layer 7. At this time, in the forward calculation, the image restoration network output X ₇ is obtained as:

X ₇ = f ₇ (X ₀ , X ₆ );

Step 3-3, further expand X ₇ as:

X ₇ = f ₇ (X ₀ , X ₆ )

= f ₇ (X ₀ , f ₆ (X ₅ ))

= f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , X ₄ )))

= f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , f ₄ (X ₃ ))))

= f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , f ₄ (f _k (X ₂ )))))

Step 3-4, during gradient backpropagation, the i-th layer in the network directly obtains the gradient from its top layer to update the parameters θ _i of this layer. In this method, θ _i specifically includes the convolution/deconvolution layer weights W _i and bias b _i , and BatchNormalization layer weight γ _i and bias β _i take the first layer of the network as an example, in order to update the first layer parameter θ ₁ , it is necessary to calculate the partial derivative of the loss function ζ with respect to θ ₁ :

Step 3-5, after obtaining the partial derivative of each layer corresponding to the top layer, use the update rule corresponding to the ADAM algorithm to update the parameters of each layer, and the training is performed on the unlabeled training data set (X ₁ , Y) obtained in step 1 , take the damaged image in X ₁ as input, and use the corresponding clear image in Y as supervision information to update the parameters in steps 3-4 until all training data are used for 20 rounds of training.

Step 4 specifically includes the following steps:

To build a supervised classification network, first extract the parameters W, b, γ, and β of the convolutional layer of the image restoration network trained in step 3, and construct a D/2 layer network according to the size of the input image, each layer is a convolutional layer, and the step size changes Consistent with the construction of the network in step 2, in which the maximum pooling layer (Max-pooling) layer is added after the last convolutional layer, and then according to the number of class labels for classification tasks N, N is the class label for supervised training The maximum possible value of the class label vector Z in the training data set (X ₀ , Z) is added to the Softmax layer of N classes. Using the extracted parameters W, b, γ and β, the corresponding parameters of the supervised classification network are initialized by direct assignment.

Step 5 specifically includes the following steps:

To train the classification network, use the ADAM optimization algorithm to train the classification network constructed and initialized in step 4 on the training data set (X ₀ , Z) with class labels. The initial learning rate is set to 1e-4, and the training lasts for n ₄ rounds ( Generally 200 rounds), after the n ₅ , n ₆ and n ₇ rounds (generally n ₅ is 80, n ₆ is 120, n ₇ is 160), multiply the current learning rate by 0.2 to get the new learning rate, n After ₄ rounds until the network converges.

The present invention is aimed at the deep convolutional neural network method of image semi-supervised feature learning. The present invention has the following characteristics: 1) when the present invention uses the deep neural network for pre-training, a cross-layer connection is added, so that the network can converge faster, and at the same time The middle-level features of the network can retain more image abstract information; 2) The method of the present invention is different from the previous semi-supervised feature learning method for specific data types, and can be applied to almost all image data and has universal applicability.

Beneficial effects: the present invention fully considers the layer-by-layer connection of the convolutional neural network in unsupervised feature learning, adding cross-layer connections to ensure that the network can extract enough useful abstract information from data without class labels, thereby better assisting class-labeled data. Data classification improves the accuracy of image classification.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the advantages of the above and other aspects of the present invention will become clearer.

Fig. 1 is the flow chart of the present invention.

Figure 2 is a schematic diagram of the network structure.

Fig. 3a is a pair of original pictures in the embodiment.

Figure 3b is the image after adding noise in Figure 3a.

Figure 3c is the restored image of Figure 3a.

Fig. 4a is a pair of original pictures in the embodiment.

Fig. 4b is a pair of original pictures in the embodiment.

Figure 4c is a feature map corresponding to Figure 4a.

Figure 4d is the feature map corresponding to Figure 4b.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

As shown in Figure 1, the present invention discloses an image repair method based on a convolutional neural network with cross-layer connections, comprising the following steps:

Step 1, generate unlabeled and labeled data sets: collect labeled and unlabeled image data, perform random cropping and normalization on each image, and obtain labeled image set X ₀ and unlabeled image data The clear image set Y, according to the image resolution, destroys the images in Y differently, and obtains the image set X ₁ without class labels after destruction. Let Z be the class label vector of images with class labels, Z={z ₁ ,z ₂ ,...,z _n }, z _i represents the class label of the i-th image, then (X ₁ ,Y) constitutes the training set of the neural network for unsupervised pre-training, and (X ₀ ,Z) is used as the training set for supervised training set;

Step 2, build a pre-trained image restoration network: build according to the size of the input image, set the total depth of the network to be D layers, D is an even number, where the first D/2 layer is a convolutional layer, and the last D/2 layer is a deconvolutional layer, The size of the convolution kernel is 3x3, the step size is 1 or 2, and the step size change rate is determined according to the network depth and image size. The input is the image in the damaged image set X ₁ in step 1, and the output is the image after the network restoration;

Step 3, training the image restoration network: using the ADAM (adaptive moment estimation, adaptive moment estimation) optimization algorithm, using the training set (X ₁ , Y) obtained in step 1 to train the network constructed in step 2, to set X ₁ The damaged image is used as input, and the corresponding lossless image in the set Y is used as the network supervision information, and the weight W and bias b of each layer of D/2 layers before the image restoration network are recorded after training;

Step 4, build a supervised classification network, use the image restoration network built in step 2 as a template, and build a D/2 layer network according to the size of the input image, all of which are convolutional layers, and the step size change is consistent with the network built in step 2. And add the Max-pooling layer and Softmax layer, and initialize the parameters of the convolutional layer using the corresponding weight W and bias b of the network trained in step 3;

Step 5, train the classification network, use the ADAM optimization algorithm to train the classification network constructed and initialized in step 4 on the image data with class labels until the algorithm converges.

Step 1 specifically includes the following steps:

This step describes the data preprocessing process, collecting image data with and without class labels, cutting each image, and using random cutting to obtain image blocks of the same size, where the size of the image block depends on the size of the original image and the model Size, for low-resolution images smaller than 50*50 (such as the CIFAR-10 data set), the cropping size is 29*29, and for natural images with high resolution larger than 225*225 (such as the PASCAL VOC data set), the cropping size is 225*225, if the resolution is between the two, first scale to a similar resolution and then crop. Record all image sets after cropping as X'; normalize and centralize the cropped image blocks, first calculate the mean and standard deviation of each pixel on the cropped image data set X', let X' be All images mean image is The standard deviation is std. For a specific image x, it is normalized and centered as follows:

x' is the processed image of image x; in the processed image, the set of images with class labels is marked as X ₀ , and the set of images without class labels is marked as Y.

For a labeled image, the processed image set X ₀ and the corresponding labeled vector Z form the labeled training data (X ₀ , Z), Z={z ₁ ,z ₂ ,…,z _n }, z _i Indicates the class label of the i-th image.

For the image in the unclassified image set Y, destroy it, add Gaussian noise or set the pixel value in the image to 0, if it is a low-resolution image after cropping (resolution less than 50*50), then add Gaussian noise method, if the cropped image is a high-resolution image (resolution greater than or equal to 50*50), the pixel point is set to 0, and the pixel point set to 0 is randomly selected from 10 adjacent 8*8 areas. The destroyed unlabeled image set X ₁ is obtained, which forms the unlabeled training data (X ₁ , Y) with the clear image set Y.

Step 2 specifically includes the following steps:

This step describes the construction process of the pre-trained neural network model. The total depth of the network is set to D layers, and D is an even number. The first D/2 layer uses a convolution layer, and the last D/2 layer uses a deconvolution layer, and the convolution kernel The size is 3x3, the step size is 1 or 2, the step size of every k layer is 1, the step size of the k+1 layer is 2, 0<k<D/2-1, repeat n times. Adjust k and n sizes according to network depth and image patch size. Add BatchNormalization layer and ReLU (Rectified Linear Unit) nonlinear layer after each convolution layer and deconvolution layer. The input of the network is the damaged image generated in step 1, and the output is the restored image of the network; every two layers, a symmetric cross-layer connection is added between the convolutional layer and the deconvolutional layer. Specifically, let CO _i represent the output of the i-th convolutional layer, DI _i represent the input of the i-th deconvolution layer, and DO _i represent the output of the i-th deconvolution layer, then the cross-layer connection can be expressed as:

DI _D-i+1 ＝DO _Di +CO _i

In particular, DI _D+1 is the network output, and CO ₀ is the network input, that is, the first cross-layer connection is connected from the input layer to the output layer. Calculate the Euclidean distance between the input of the final layer of the network and the corresponding image in the original image data set Y as the loss function:

Where X ⁱ is the i-th image in the unlabeled damaged image set X ₁ , and Y ⁱ is the i-th image in the unlabeled clear image set Y, is the function represented by the neural network, N is the number of training images, θ is all trainable parameters of the network, including the weight W and bias b of the convolutional layer and the deconvolution layer, and the weight γ and bias β of the BatchNormalizatiion layer.

Figure 2 is a simple schematic diagram of the network structure. In the left figure, Corrupted data is the network input data, restored data is the network output data, conv1, conv2 and c3...c6 are convolution layers, d3...d6,, deconv1, deconv2 are deconvolution Floor. The figure on the right describes the details of a cross-layer connection. In the figure, conv is the convolution layer, deconv is the deconvolution layer, and ReLU and BatchNorm represent the ReLU layer and the BatchNormalization layer, respectively.

Step 3 specifically includes the following steps:

This step describes the training process of the pre-trained neural network model. The ADAM optimization algorithm is used to perform gradient backpropagation to train the neural network, and the learning rate is set to 1e-4. The training lasts for 20 epochs, and after the 8th and 16th epochs, the learning rate is set to 1e-5 and 1e-6, respectively.

To illustrate the step of gradient backpropagation with cross-layer connections, let the network depth be 7 layers, add cross-layer connections in the way in step 2-2, let X ₀ be the network input, and X _i be the i-th convolutional layer output , the cross-layer connection specifically connects X ₁ to the layer 5 input and X ₀ to the layer 7 input. At this time, in the forward calculation, the image restoration network output X ₇ is obtained as:

X ₇ = f ₇ (X ₀ , X ₆ )

_X7 can be further expressed as:

X ₇ = f ₇ (X ₀ , X ₆ )

= f ₇ (X ₀ , f ₆ (X ₅ ))

= f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , X ₄ )))

= f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , f ₄ (X ₃ ))))

= f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , f ₄ (f _k (X ₂ )))))

Where X ₁ and X ₂ represent the feature maps obtained by the first and second convolutional layers.

During gradient backpropagation, the i-th layer in the network directly obtains the gradient from its top layer to update the parameters θ _i of this layer. In this method, θ _i specifically includes the convolution/deconvolution layer weight W _i and bias b _i , and BatchNormalization layer weight γ _i and bias β _i take the first layer of the network as an example, in order to update the first layer parameters θ ₁ need to calculate the partial derivative of the loss function ζ with respect to θ ₁ :

After obtaining the partial derivative of each layer corresponding to the top layer, use the update rule corresponding to the ADAM algorithm to update the parameters of each layer, and the training is carried out on the unlabeled data set (X ₁ , Y) obtained in step ₁ . The damaged image is used as input, and the corresponding clear image in Y is used as the supervisory information to update the parameters in steps 3-4 until all training data are used for 20 rounds of training.

Step 4 specifically includes the following steps:

This step describes the construction process of the supervised classification network. First, extract the parameters W, b, γ, and β of the convolutional layer of the image restoration network trained in step 3, and construct a D/2 layer network according to the size of the input image, and each layer is a convolution Layer, the step size change is consistent with the network constructed in step 2, where the maximum pooling layer (Max-pooling) layer is added after the last convolutional layer, and then according to the number of classification tasks N, N is supervised The maximum possible value of the class label vector Z in the training data (X ₀ , Z) is added to the Softmax layer of N classes. Using the extracted parameters W, b, γ and β, the corresponding parameters of the supervised classification network are initialized by direct assignment.

Step 5 specifically includes the following steps:

This step describes the training process of the supervised classification network. The classification network constructed and initialized in step 4 is trained on the labeled image dataset (X ₀ , Z) using the ADAM optimization algorithm, and the initial learning rate is set to 1e-4 , the training lasts for n ₄ rounds (generally 200 rounds), after the n ₅ , n ₆ and n ₇ rounds (generally n ₅ is 80, n ₆ is 120, n ₇ is 160), the current learning rate is multiplied by 0.2 to get the new learning rate, n after ₄ rounds until the network converges.

Example 1

This example describes semi-supervised feature learning on CIFAR-10, including the following parts:

1. First, the 50,000 natural images in the CIFAR-10 dataset are evenly divided into two parts, one part contains 4000 images with class labels, and the other part contains 46000 images without class labels.

2. For each unlabeled image of size 32*32, during training, a 29*29 image block is randomly intercepted, and Gaussian noise with a mean value of 0 and a standard deviation of 30 is added to the image. Normalize the image after adding noise and the image without noise to form a non-labeled training set.

3. Construct an 18-layer convolutional neural network with cross-layer connections, and use the ADAM algorithm to train on the generated unlabeled images. After the network converges, the first 9 layers are retained, and the corresponding classification network is constructed using their network weights.

4. Train the classification network on another 4,000 images with class labels, use the ADAM algorithm to train until convergence, test on the 50,000 original image test set, and report the accuracy rate as shown in Table 1:

Table 1

The last line is the accuracy of the method. It can be seen that the method has reached the accuracy of many semi-supervised learning using GAN and compared with no pre-training (No pre-training line) and no cross-layer connection (Pre-training Without shortcut line) the same network, the accuracy rate has been greatly improved.

Example 2

This embodiment describes the large-scale semi-supervised feature learning using Imagenet dataset and Pascal VOC 2007 data, including the following parts:

1. First, randomly intercept 225*225 image blocks on the Imagenet natural image data set, and set 35 random 8*8 image area pixels to 0 for each image block, and set the 0-set image and the original image respectively Perform normalization to form a non-labeled training set.

2. Construct a 32-layer convolutional neural network with cross-layer connections, and use the ADAM algorithm to train on the generated unlabeled images. After the network converges, the first 16 layers are retained, and the corresponding classification network is constructed using their network weights.

3. For PASCAL VOC 2007 natural image data, 225*225 image blocks are randomly intercepted, and the image blocks are horizontally flipped and normalized with a probability of 50% to obtain class-labeled data.

4. Train the classification network on the generated class-labeled data, use the ADAM algorithm to train until convergence, use the test set for testing, and report the accuracy rate as follows:

Table 2

The last line is the accuracy rate of this method. It can be seen that the accuracy rate of this method is about 1% higher than that of the current similar methods, and compared with the same network without cross-layer connection (Ours without shortcut line), the accuracy rate has been greatly improved. Figures 3a to 3c are the image recovery effects of the pre-trained network in this embodiment. Figure 3a is the original image, Figure 3b is the image after adding noise, and Figure 3c is the restored image. It can be seen that the pre-trained network can learn well Image details. Figures 4a to 4d are the feature visualization effects learned in this embodiment. Figures 4a and 4b are the original images, and Figures 4c and 4d are the feature maps corresponding to Figures 4a and 4b respectively. The dog faces in each image Some of them are very obvious in the feature map, which shows that the features learned by this method capture the deep semantic information of the image well.

The present invention provides a semi-supervised feature learning method based on a convolutional neural network with symmetric cross-layer connections. There are many methods and approaches to specifically realize the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for this technology Those of ordinary skill in the art can make some improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components that are not specified in this embodiment can be realized by existing technologies.

Claims (6)

1. Based on the convolutional neural network semi-supervised feature learning method with symmetric cross-layer connection, it is characterized in that, comprising the steps: Step 1, generate unlabeled and labeled data sets: collect labeled and unlabeled image data, perform random cropping and normalization processing on each image, and obtain the labeled image set X ₀ and unlabeled image set X 0 The image set Y, according to the image resolution, destroys the images in the set Y differently, and obtains the destroyed image set X ₁ without class labels. Let Z be the class label vector of images with class labels, Z={z ₁ ,z ₂ ,...,z _n }, z _i represents the i-th image class label, and the value of i is 1~n, then (X ₁ ,Y) constitutes a non-label training data set for unsupervised pre-training, (X ₀ , Z) as a set of labeled training data for supervised training; Step 2, build a pre-trained image recovery network: build an image recovery network according to the size of the input image, set the total depth of the network to be D layers, D is an even number, where the first D/2 layer is a convolutional layer, and the last D/2 layer is a deconvolution layer Multilayer, the size of the convolution kernel is 3x3, the input is the image in the image set X ₁ destroyed in step 1, and the output is the image restored by the network; Step 3, training image restoration network: use the ADAM optimization algorithm, use the training set (X ₁ ,Y) obtained in step 1 to train the network constructed in step 2, take the damaged image in the set X ₁ as input, and use the set Y The corresponding lossless image in is used as network supervision information, and after training, record the weight W and bias b of each layer in the D/2 layer of the image restoration network; Step 4, build a supervised classification network: use the image recovery network built in step 2 as a template, build a D/2 layer network according to the size of the input image, both are convolutional layers, and add Max-pooling layer and Softmax layer, and simultaneously The parameters of the product layer are initialized with the corresponding weight W and bias b of the network trained in step 3; Step 5, train the classification network, use the ADAM optimization algorithm to train the classification network constructed and initialized in step 4 on the image data with class labels until the algorithm converges. 2. The method according to claim 1, wherein step 1 comprises the steps of: Step 1-1, collect image data with and without class labels, and crop each image, and use random cropping to obtain image blocks of the same size, where the size of the image block depends on the size of the original image and the size of the model. After cropping, all image collections are denoted as X'; Step 1-2, normalize and centralize the cropped image blocks, first calculate the mean and standard deviation of each pixel on the cropped image data set X', set the mean image of all images on X' as The standard deviation is std. For a specific image x, it is normalized and centered as follows: x' is the processed image of image x, in the processed image, mark the set of images with class labels as X ₀ , and the set of images without class labels as Y; Step 1-3, for the labeled image, the processed image set X ₀ and the corresponding labeled vector Z form the labeled training data (X ₀ , Z), Z={z ₁ ,z ₂ ,…,z _n }, z _i represents the class label of the i-th image; Steps 1-4, for the images in the unlabeled image set Y, destroy, add Gaussian noise or set the pixel value in the image to 0, if the cropped image is a low-resolution image, then adopt the method of adding Gaussian noise, if cropped After the high-resolution image, the method of setting the pixels to 0 is adopted, and the pixels set to 0 are randomly selected 10 adjacent 8*8 areas, and the destroyed image set without class label X ₁ is obtained, which is similar to the class-free image set X 1 . The labeled image set Y constitutes the unlabeled training data set (X ₁ , Y). 3. The method according to claim 2, wherein step 2 comprises the steps of: Step 2-1, set the total depth of the image restoration network to be D layers, D is an even number, where the first D/2 layer uses a convolution layer, and the last D/2 layer uses a deconvolution layer, the convolution kernel size is 3x3, and the step size is is 1 or 2, after the step size of the k layer is 1, the step size of the k+1 layer is 2, repeat n times, the specific value of k is 0<k<D/2, adjust the size of k and n according to the actual training image size , if it is a low-resolution image after cropping, k=4, n=3; if it is a high-resolution image after cropping, k=2, n=5; add a BatchNormalization layer after each convolution layer and deconvolution layer and the ReLU nonlinear layer, the network input is the image in the destroyed image set X ₁ generated in step 1, and the output is the image after the network restoration; the network parameters include the weight W and the bias b of the convolutional layer and the deconvolutional layer, And BatchNormalization layer weight γ and bias β; Step 2-2, add symmetrical cross-layer connections between the convolutional layer and the deconvolution layer every two layers: let CO _i represent the output of the i-th convolutional layer, and DI _i represent the input of the i-th deconvolutional layer , DO _i represents the output of the i-th deconvolution layer, then the cross-layer connection is expressed as: DI _D-i+1 = DO _Di + CO _i , Then DI _D+1 is the network output, CO ₀ is the network input, the first cross-layer connection starts from the input layer and connects to the output layer, the final layer input of the network and the corresponding image in the lossless image set Y calculate the Euclidean distance as the subsequent network training loss function Where X ⁱ is the i-th image in the image set X ₁ , Y ⁱ is the i-th image in the image set Y, is the function represented by the neural network, N is the number of training images, θ is all trainable parameters of the network, including the weight W and bias b of the convolutional layer and the deconvolution layer, and the weight γ and bias β of the BatchNormalization layer. 4. The method according to claim 3, wherein step 3 comprises the steps of: Step 3-1, using the ADAM optimization algorithm to train the neural network with gradient backpropagation, the learning rate is set to 1e-4, and the training lasts for n ₁ rounds. After the n _2nd and n _3rd rounds, the learning rates are respectively set to 1e-5 and 1e-6; Step 3-2, set the depth of the image restoration network to 7 layers, add cross-layer connections in the manner in step 2-2, set X ₀ as the network input, X _i as the output of the i-th convolutional layer, and cross-layer connections specifically set X ₁ is connected to the input of layer 5, and X ₀ is connected to the input of layer 7. At this time, in the forward calculation, the image restoration network output X ₇ is obtained as: X ₇ = f ₇ (X ₀ , X ₆ ); Step 3-3, further expand X ₇ as: X ₇ = f ₇ (X ₀ , X ₆ ) = f ₇ (X ₀ , f ₆ (X ₅ )) = f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , X ₄ ))) = f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , f ₄ (X ₃ )))) = f ₇ (X ₀ , f ₆ (f ₅ (X ₁ , f ₄ (f _k (X ₂ ))))); Step 3-4, when the gradient is backpropagating, the i-th layer in the network directly obtains the gradient from its top layer to update the parameter θ _i of this layer; Step 3-5, after obtaining the partial derivative of each layer corresponding to the top layer, use the update rule corresponding to the ADAM algorithm to update the parameters of each layer, and the training is performed on the unlabeled training data set (X ₁ , Y) obtained in step 1 , take the damaged image in X ₁ as input, and use the corresponding image in Y as supervision information to update the parameters in steps 3-4 until all training data are used for 20 rounds of training. 5. The method according to claim 4, wherein step 4 comprises: To build a supervised classification network, first extract the parameters of the convolutional layer of the image restoration network trained in step 3, and construct a D/2 layer network according to the size of the input image, each layer is a convolutional layer, and the step size change is consistent with the network constructed in step 2 , where the maximum pooling layer Max-pooling layer is added after the last convolutional layer, and then according to the number of classification tasks N, N is a supervised training training data set with classification (X ₀ , Z) The maximum possible value of the class label vector Z is added to the Softmax layer of N classes, and the extracted parameters are used to initialize the corresponding parameters of the supervised classification network by direct assignment. 6. The method according to claim 5, wherein step 5 comprises the steps of: To train the classification network, use the ADAM optimization algorithm to train the classification network constructed and initialized in step 4 on the training data set (X ₀ , Z) with class labels. The initial learning rate is set to 1e-4, and the training lasts for n ₄ rounds. After the n ₅ , n ₆ and n ₇ rounds, multiply the current learning rate by 0.2 to get a new learning rate, n ₄ rounds until the network converges.