CN111489305B - Image Enhancement Method Based on Reinforcement Learning - Google Patents

Abstract

The invention relates to an image enhancement method based on reinforcement learning, which comprises the following steps: making a distorted picture data set: preprocessing a training set and a testing set by using matlab by adopting three types of processing modes with different degrees to generate a distortion picture; designing an image enhancement processing tool: respectively training enhancement algorithm parameters aiming at different types or different degrees of distortion pictures, generating corresponding meta files, and obtaining a plurality of processing tools, wherein each tool correspondingly processes distortion of a specific degree and a specific type; training an optimal processing tool to select a network; and (5) testing the performance of the model.

Description

Image enhancement method based on reinforcement learning

Technical Field

The present invention belongs to the technical field of image processing and relates to a method for improving image quality by using reinforcement learning based on machine learning technology.

Background Art

With the advent of the big data era, image information has become an indispensable source of information in daily life because it is more intuitive and easier to understand. However, the necessary condition for understanding information through images is to obtain a high-quality image. Degraded images contain a lot of noise, which brings various obstacles to subsequent analysis. With the development of artificial intelligence, people's requirements for image quality are getting higher and higher. In the field of computer vision, image enhancement has always been a hot research topic.

In many digital image processing applications, high-quality pictures or videos are often needed for processing and analysis. Due to the limitations of existing technologies, imaging devices often obtain low-quality pictures, including but not limited to Gaussian white noise, low resolution, blur, etc., which will bring difficulties to subsequent image processing and analysis, and how to improve the quality of these pictures has become a focus of attention.

In recent years, deep learning, especially convolutional neural networks, has been proven to be an efficient data-driven framework and has shown good results in low-level image processing problems. Dong et al. designed a convolutional neural network architecture [1] to solve the problem of single image super-resolution reconstruction. The designed three-layer convolutional neural network simulates sparse coding-based super-resolution reconstruction. Wang et al. integrated neural networks into the sparse coding framework to solve the super-resolution reconstruction problem [2]. They used a neural network design for fast decomposition of sparse coding and cleverly applied this design to the image super-resolution problem. Schuler et al. developed a multi-layer perceptron method constructed by deconvolution [3] to remove noise and hand-crafted traces. Xu et al. used a deep convolutional neural network [4] to restore noisy images and used the singular value decomposition method to reduce the parameters in the network. Kim et al. [5] found that more layers can be added to the network by learning the residual between low-resolution and high-resolution images to help the neural network converge.

[1]Dong C,Chen C L,He K,et al.Image super-Resolution Using DeepConvolutional Networks[J].IEEE Transactions onPatternAnalysis&MachineIntelligence,2016,38(2):295-307

[2]Wang Z, Liu D, Yang J, et al.Deep networks for Image Super-Resolution with Sparse Prior[C].In proceedingofICCV,Santiago,Chile,2015:370-378.

[3]Schuler C J, Burger H C, Haemeling S, et al.A Machine LearningApproach for Non-blind Image Deconvolution[C].InProceedingofCVPR,Portland,ORUSA,2013:1067-1074

[4]Xu L,Ren J S,Liu C,et al.Deep convolutional neural network forimage deconvolution[C].In Proceeding ofNIPS,Montreal,Quebec,Canada,2014:1790-1798.

[5]Kim J, Lee J K, Lee KM.Accurate Image Super-Resolution Using VeryDeep ConvolutionNetworks[C].InProceedings ofCVPR,LasVegas,NV,USA,2016:1646-1654

Summary of the invention

The purpose of the present invention is to provide an image enhancement method based on reinforcement learning. The present invention selects the most appropriate image enhancement method to process the image, enrich the image details, enhance the image quality, facilitate subsequent processing, and improve efficiency and accuracy. The technical solution is as follows:

An image enhancement method based on reinforcement learning comprises the following steps:

Step 1: Create a distorted image dataset

The public image dataset is divided into a training set and a test set. Matlab is used to preprocess the training set and the test set using three types of processing methods of different degrees to generate distorted images, including different degrees of JPEG compression processing methods, different degrees of Gaussian noise processing methods, and different degrees of Gaussian blur processing methods.

Step 2: Design image enhancement processing tools

For different types or degrees of distorted images, the enhancement algorithm parameters are trained respectively, and the corresponding meta files are generated to obtain multiple processing tools. Each tool is used to process a specific degree or type of distortion. For different degrees of JPEG compression processing, the generative adversarial network recovery and reconstruction algorithm is used for recovery; for different degrees of Gaussian noise processing, the convolutional neural network denoising algorithm is used for recovery; for different degrees of Gaussian blur processing, the convolutional neural network deblurring algorithm is used for recovery;

Step 3: Train the optimal processing tool selection network

When reconstructing distorted images, different processing tools have different restoration effects, and different processing orders have different restoration effects. It is necessary to design a network that can autonomously select the optimal processing tool. The DQN reinforcement learning algorithm is used to regard the selection problem as a Markov process, and each action is evaluated by a reward function. In the face of different current states, the most appropriate action is taken to transform the state to maximize the reward function. The selection of processing tools is regarded as a discrete action, and the most appropriate processing tools and processing order are selected in the face of distorted images of different degrees.

Determine the training optimal processing tool selection network parameter adjustment plan: Batch is set to 32, learning rate is set to 0.0001, exploration rate initial value is 0.1, and number of iterations is set to 100,000 times; Batch is set to 1 during testing, that is, only one image is processed each time; Training optimal processing tool selection network parameters maximize the objective function cumulative reward function; When the training iteration ends or the cumulative reward function converges, the optimal processing tool selection network for distorted images is obtained: input the distorted image, and output the label and processing order corresponding to the optimal processing tool;

Step 4: Model performance testing

The distorted images in the test set are input into the optimal processing tool selection network to obtain the corresponding labels and processing orders of each image processing tool, and the corresponding operations are performed on the distorted images to obtain enhanced images. The performance of the model is evaluated by calculating the peak power signal-to-noise ratio (PSNR) between the original image without distortion and the enhanced image. The higher the PSNR value, the better the restoration effect.

The beneficial effects of the present invention are shown in Table 1.

Table 1 Results Statistics

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 Image enhancement model structure diagram

Figure 2: JPEG compression image

Figure 3: Compressed image model enhancement result diagram

Figure 4 Gaussian noise diagram

Figure 5: Noise image model enhancement result

Figure 6 Blurred image

Figure 7: Blurred image model enhancement result

DETAILED DESCRIPTION

In order to make the technical solution of the present invention clearer, the specific implementation manner of the present invention is further described below in conjunction with the accompanying drawings.

Step 1: Create a distorted image dataset.

The DIV2K dataset is divided into a training set and a test set at a ratio of 15:1. Matlab is used to preprocess the training set and the test set using 12 processing methods to generate distorted images, and 750 training set images and 50 test set images are obtained, where each image is processed multiple times. The 12 methods used in the present invention are shown in Table 2.

Table 2 Distortion processing

(1) Gaussian blur processing

1/N ... 1/N ... 1/N ... 1/N ... 1/N

Figure 1 [N, N] Gaussian blur convolution kernel

The image is convolved using the convolution kernel shown in Figure 1 to obtain a blurred image.

(2) Adding Gaussian noise

An input image f(x,y) is processed to produce a degraded image g(x,y). Given g(x,y), the degradation function H and the additive noise term η(x,y), the degraded image in the spatial domain can be given by the following formula:

g(x,y)=h(x,y)*f(x,y)+eta(x,y)

In the frequency domain:

G(u,v)＝H(u,v)F(u,v)+N(u,v)

Step 2: Design image enhancement processing tools.

For different types or degrees of distortion, the enhancement algorithm parameters are trained respectively, and the corresponding meta files are generated to obtain 12 processing tools for processing specific degrees and specific types of distortion. In the present invention, for 4 different degrees of JPEG compression, the generative adversarial network restoration and reconstruction algorithm is used for restoration; for 4 different degrees of Gaussian noise, the convolutional neural network denoising algorithm is used for restoration; for 4 different degrees of blur, the convolutional neural network deblurring algorithm is used for restoration.

Step 3: Train the optimal processing tool selection network.

When reconstructing distorted images, different processing tools have different restoration effects, and different processing orders have different restoration effects, so a network that can autonomously select the optimal processing tool should be designed. The reinforcement learning algorithm regards the selection problem as a Markov process and uses a reward function to evaluate each action. Faced with different current states, the most appropriate action is taken to transform the state to maximize the reward function. The present invention adopts the DQN reinforcement learning algorithm, regards the selection of processing tools as discrete actions, and selects the most appropriate processing tools and processing order for distorted images of different degrees, as shown in Figure 2.

In the DQN algorithm of the present invention, the environment state S _t ={I _t ,v _t }, where I _t represents the input distorted image vector, v _t represents the historical action vector, and in the first step, v _t is a 0 vector; the action taken by the individual at time t is A _t ∈{12 tools}, and taking an action means selecting a tool to process the distorted image, obtaining the reconstructed image, switching the state to S _t+1 , and obtaining the environment reward R _t , and the calculation formula is as follows:

R _t =||I _target -I _t-1 || ² -||I _target -I _t || ²

Where I _target represents the undistorted original image; the cumulative reward function Q(t) = E(R _t+1 +λR _t+2 +λ ² R _t+3 +…|S _t ), where E is the expectation function and λ is the attenuation factor. Maximizing the cumulative reward function Q(t) is equivalent to the problem of selecting the optimal processing tool.

Since the training images used in the present invention are large in size, after multiple experimental results, the optimal training processing tool is determined to select the network parameter adjustment scheme: Batch is set to 32, the learning rate is set to 0.0001, the initial value of the exploration rate is 0.1, and the number of iterations is set to 100,000 times. During the test, Batch is set to 1, that is, only one image is processed each time. The optimal training processing tool selects network parameters to maximize the cumulative reward function Q(t) of the objective function. The experimental environment is the Ubuntu 16.04 operating system, and the RTX 2060 GPU with 6GB of video memory of NVIDIA is used for training and CUDA is used to accelerate the training.

When the training iteration ends or the cumulative reward function Q(t) converges, the optimal processing tool selection network for the distorted image is obtained: the distorted image is input, and the label and processing order corresponding to the optimal processing tool are output.

Step 4: Model performance testing.

The distorted images in the test set are input into the optimal processing tool selection network to obtain the corresponding labels and processing orders of each image processing tool, and the corresponding operations are performed on the distorted images to obtain enhanced images. The performance of the model is evaluated by calculating the peak power signal-to-noise ratio (PSNR) between the original image without distortion and the enhanced image. The higher the PSNR value, the better the restoration effect.

PSNR is defined as:

Wherein m, n, c represent the size of the image, which is 256, 256, and 8 in the present invention; x is the original image without distortion, y is the reconstructed image, and MAX _I is the maximum pixel value, which is 255.

The experimental data were analyzed and processed to evaluate the image quality enhancement performance of the present invention. The test results are shown in Table 1. By comparison, it can be seen that the present invention has a better effect on image quality enhancement.

Claims (1)

1. An image enhancement method based on reinforcement learning, comprising the following steps: Step 1: Create a distorted image dataset Divide the public image data set into training set and test set, and use matlab to preprocess the training set and test set with three types of processing methods of different degrees to generate distorted images, including different degrees of JPEG compression processing methods, different degrees Gaussian noise processing methods, different degrees of Gaussian blur processing methods; Step 2: Design image enhancement processing tools For different types or different degrees of distorted pictures, train the enhancement algorithm parameters separately, generate corresponding meta files, and obtain multiple processing tools. Each tool corresponds to a specific degree and type of distortion. Among them, for different degrees of JPEG compression The processing method is restored by the generative confrontation network restoration and reconstruction algorithm; for different degrees of Gaussian noise processing methods, the convolutional neural network denoising algorithm is used for restoration; for different degrees of Gaussian blur processing methods, the convolutional neural network defuzzification algorithm is used for restoration; Step 3: Train the optimal processing tool selection network When reconstructing distorted images, different processing tools have different recovery effects, and different processing sequences have different recovery effects. It is necessary to design a network that independently selects the optimal processing tool; using the DQN reinforcement learning algorithm, the selection problem is regarded as a Markov process , use the reward function to evaluate each action, face different current states, take the most appropriate action to convert the state to maximize the reward function, treat the selection of processing tools as a discrete action, and face different degrees of distortion image selection the most appropriate processing tools and sequence of processing; Determine the optimal processing tool for training and select a network parameter adjustment scheme: Batch is set to 32, the learning rate is set to 0.0001, the initial value of the exploration rate is 0.1, and the number of iterations is set to 100,000; when testing, set Batch to 1, that is, only one batch is processed each time. image; training the optimal processing tool to select network parameters to maximize the cumulative return function of the objective function; when the training iteration ends or the cumulative return function converges, the optimal processing tool selection network for the distorted picture is obtained: input the distorted picture, and output the optimal processing tool Corresponding labels and processing order; Step 4: Model performance testing Input the distorted pictures in the test set into the optimal processing tool selection network, obtain the corresponding labels and processing order of each picture processing tool, and perform corresponding operations on the distorted pictures to obtain enhanced pictures; by calculating the original picture without distortion and the enhanced picture The peak power signal-to-noise ratio (PSNR) between images is used to evaluate the performance of the model. The higher the PSNR value, the better the restoration effect.

Images