WO2019136772A1 - 一种模糊图像的复原方法、装置、设备及存储介质 - Google Patents

一种模糊图像的复原方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2019136772A1
WO2019136772A1 PCT/CN2018/073794 CN2018073794W WO2019136772A1 WO 2019136772 A1 WO2019136772 A1 WO 2019136772A1 CN 2018073794 W CN2018073794 W CN 2018073794W WO 2019136772 A1 WO2019136772 A1 WO 2019136772A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
image
target
blurred image
sampling
Prior art date
Application number
PCT/CN2018/073794
Other languages
English (en)
French (fr)
Inventor
张勇
何泽裕
赵东宁
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Publication of WO2019136772A1 publication Critical patent/WO2019136772A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/086—Learning methods using evolutionary algorithms, e.g. genetic algorithms or genetic programming
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the invention belongs to the technical field of image processing, and in particular relates to a method, a device, a device and a storage medium for restoring a blurred image.
  • Image is the main way for humans to obtain visual information. Most of the external information acquired by humans comes from the image information received by the visual system. However, in the process of image acquisition, transmission and preservation, for various reasons, such as imaging system Imperfections, the influence of transmission media, the relative motion of objects and imaging systems, environmental noise, etc., can inevitably cause degradation phenomena such as image defocus, distortion, noise interference, etc., due to image degradation, images displayed at the image receiving end It is no longer the original image transmitted, and the image rendering effect is very poor. In many research fields such as optics, medicine, astronomy and meteorology, the definition and quality of the image are usually high. Therefore, it must be Degraded images are processed to be restored to real original images.
  • Image restoration technology usually uses optimized criteria to reconstruct blurred images and improve the quality of the images to obtain clearer images.
  • Image restoration technology has attracted the attention and research of many researchers, and has been widely applied to medical imaging, satellite imaging and other fields.
  • the blurred image can be divided into noise blur, motion blur and other types, and noise blur can be divided into Gaussian noise, Poisson noise and salt and pepper noise.
  • the fuzzy restoration method is divided into two types: non-blind restoration method and blind restoration method. Both methods have some problems.
  • non-blind restoration method a known point spread function is needed, and the function is usually unknown, but for blind restoration. In terms of law, there are generally problems such as slow convergence rate, large amount of computation, and large uncertainty of results.
  • these two methods of fuzzy restoration can only be used for a certain type of fuzzy restoration, and there is a problem of small application range.
  • the object of the present invention is to provide a method, a device, a computing device and a storage medium for restoring a blurred image, which aims to solve the problem that the restoration method of the blurred image cannot be provided due to the inability of the prior art to provide an effective method for restoring the blurred image. Slow speed problem.
  • the present invention provides a method for restoring a blurred image, the method comprising the steps of:
  • the obtained restored data is subjected to inverse normalization processing to obtain a restored image of the target blurred image.
  • the present invention provides a device for restoring a blurred image, the device comprising:
  • An image normalization unit configured to normalize the target blurred image to obtain a normalized image when receiving a request for restoring the target blurred image
  • a target sample sampling unit configured to perform sliding sampling on the normalized image by using a sliding window to obtain a target sampling sample corresponding to the target blurred image
  • a restoration data acquisition unit configured to perform restoration processing on the target sample sample by using a preset optimized BP neural network to obtain restoration data of the target blurred image
  • the restored image obtaining unit is configured to perform inverse normalization processing on the obtained restored data to obtain a restored image of the target blurred image.
  • the present invention also provides a computing device including a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor implementing the computer program The steps of the method as described above.
  • the present invention also provides a computer readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as previously described.
  • the present invention When receiving the request for restoring the target blurred image, the present invention firstly normalizes the target blurred image to obtain a normalized image, and then performs sliding sampling on the normalized image through the sliding window to obtain the target blur.
  • the target sample sample corresponding to the image is then restored by the preset optimized BP neural network to obtain the restored data of the target blurred image.
  • the obtained restored data is inverse normalized to obtain the target.
  • the restored image of the blurred image improves the restoration accuracy and the recovery speed of the blurred image, thereby improving the sharpness of the restored image.
  • FIG. 1 is a flowchart of an implementation of a method for restoring a blurred image according to Embodiment 1 of the present invention
  • FIG. 2 is a diagram showing an example of a sampling process for performing sliding window sampling in a method for restoring a blurred image according to Embodiment 1 of the present invention
  • FIG. 3 is a diagram showing an example of an optimized sliding window type in a method for restoring a blurred image according to Embodiment 1 of the present invention
  • FIG. 4 is a flowchart showing an implementation of optimizing and training a BP neural network in a method for restoring a blurred image according to Embodiment 2 of the present invention
  • FIG. 5 is a diagram showing an example of cross operation in a method for optimizing and training a BP neural network according to Embodiment 2 of the present invention
  • FIG. 6 is a diagram showing an example of application testing of a method for restoring a blurred image according to Embodiment 2 of the present invention.
  • FIG. 7 is a diagram showing an example of application testing of a method for restoring a blurred image according to Embodiment 2 of the present invention.
  • FIG. 8 is a schematic structural diagram of a device for restoring a blurred image according to Embodiment 3 of the present invention.
  • FIG. 9 is a schematic diagram showing an optimized structure of a fuzzy image restoration apparatus according to Embodiment 3 of the present invention.
  • FIG. 10 is a schematic structural diagram of a device for restoring a blurred image according to Embodiment 4 of the present invention.
  • FIG. 11 is a schematic structural diagram of a computing device according to Embodiment 5 of the present invention.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • FIG. 1 is a flowchart showing an implementation process of a method for restoring a blurred image according to Embodiment 1 of the present invention. For convenience of description, only parts related to the embodiment of the present invention are shown, which are described in detail as follows:
  • step S101 when a request to restore the target blurred image is received, the target blurred image is normalized to obtain a normalized image.
  • Embodiments of the present invention are applicable to computing devices, such as personal computers, smart phones, tablets, and the like.
  • the blurred image that needs to be restored may be a type of blurred image such as a Gaussian blurred image, a Poisson noise blurred image, a salt and pepper noise blurred image, or a motion blurred image.
  • the normalization of the image is to maintain the invariance of the affine and improve the accuracy of the calculation.
  • the formula can be adopted. Normalize the pixels of the image and transform the image pixels from [0, 255] to [0, 1], where Represents the value of the kth pixel after normalization, x k represents the value of the kth pixel before normalization, and T represents the number of pixels.
  • step S102 the normalized image is subjected to sliding sampling through a sliding window to obtain a target sampling sample corresponding to the target blurred image.
  • the sliding window is used for sampling.
  • the size of the sliding window is 3 ⁇ 3, 5 ⁇ 5, 7 ⁇ 7, 9 ⁇ 9, etc.
  • the sliding window is selected differently. For example, for slight blur, select 3 ⁇ 3 or 5 ⁇ 5.
  • Sliding window, for depth blur, select 7 ⁇ 7 or 9 ⁇ 9 sliding window the blur degree of the image can be judged according to the image quality evaluation index Brenner gradient function or Tenengrad gradient function, and the lower the Brenner gradient function value or the Tenengrad gradient function value, Indicates that the image is more blurred.
  • a sampling process of a blurred image is introduced by taking a 3 ⁇ 3 sliding window as an example.
  • the size of the blurred image is M ⁇ N. Since the 3 ⁇ 3 sliding window is used for sampling, the outermost layer of pixels of the image does not have enough neighbors to be a central pixel, so the first A central pixel is the pixel P 1 corresponding to the second row and the second column, and the region is a pixel included in the black solid frame, and then the region is expanded from top to bottom and left to right to form a column.
  • the sliding window is shifted to the right by one pixel, and the pixel corresponding to the second row and the third column is taken as the central pixel, and sampling is continued to obtain a nine-dimensional column vector.
  • the sliding window samples the second row from left to right, after traversing the second row, goes to the third row, samples the pixel with the third row as the center pixel, and so on, and traverses the entire image.
  • a matrix X of nine rows [(M-1) ⁇ (N-1)] columns is formed, which is used as a target sample sample of the BP neural network input.
  • the size of the sampling sliding window of the normalized image is performed according to the degree of blurring of the blurred image. Setting, then, according to the Euclidean distance of the edge pixel point and the center pixel point sampled by the sliding window, the size of the sliding window is reduced to obtain a sliding window of a preset shape, and finally, a sliding window passing the preset shape
  • the normalized image is subjected to sliding sampling, thereby reducing the amount of calculation of the sample and increasing the sampling speed.
  • the sampling area of 7 ⁇ 7 sliding window is composed of 37 pixels
  • the sampling area of 9 ⁇ 9 sliding window is composed of 69 pixels , as shown in 3b, 3c of Figure 3, respectively.
  • the optimized 5 ⁇ 5, 7 ⁇ 7, 9 ⁇ 9 and other types of sliding window sampling processes are consistent as shown in Fig. 2, all of which are sampled by sliding.
  • step S103 the target sample sample is restored by a preset optimized BP neural network to obtain the restored data of the target blurred image.
  • a differential evolution algorithm (DE) and a Levenberg-Marquardt (LM) are combined to form a new hybrid algorithm, referred to as DE-
  • DE-LM Levenberg-Marquardt
  • the LM algorithm uses the fuzzy image and its corresponding clear image data samples to learn and train the BP neural network optimized by DE-LM algorithm, and then uses the trained BP neural network to recover the target blurred image.
  • the BP neural network optimization process Please refer to the description of the subsequent embodiments, and details are not described herein again.
  • step S104 the obtained restored data is subjected to inverse normalization processing to obtain a restored image of the target blurred image.
  • the target blurred image is restored by the optimized BP neural network, the restored data is output, the restored data is processed to form an image matrix, and then the image matrix is inverse normalized to obtain the target blurred image. Restore the image, which is a sharp image.
  • the processing is performed to obtain a restored image of the target blurred image, thereby improving the restoration accuracy and the recovery speed of the blurred image, thereby improving the sharpness of the restored image.
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • FIG. 4 is a flowchart showing an implementation process of optimizing and training a BP neural network in a method for restoring a blurred image according to Embodiment 2 of the present invention. For convenience of description, only parts related to the embodiment of the present invention are shown, which are described in detail below. :
  • step S103 of the first embodiment the target sample sample is restored by a preset optimized BP neural network, and the BP neural network needs to be performed by a differential evolution algorithm and a Levenberg-Magner algorithm before the restoration. Optimization and training, the detailed process is as follows:
  • step S401 the weights and thresholds of the pre-built three-layer BP neural network layers are initially optimized by a differential evolution algorithm, and the initial optimized weights and thresholds are performed by the Levenberg-Magnell algorithm. optimization.
  • the differential evolution algorithm is a method based on population evolution and self-organizing optimization, and the process of initial optimization of the weights and thresholds of each layer of the pre-built three-layer BP neural network by differential evolution algorithm may be performed. This is achieved by the following steps:
  • the jth "gene” of i "chromosome” the gene is a variable
  • x j,i (0) corresponds to the weight and threshold of each layer of the three-layer BP neural network
  • D is the number of genes, that is, the number of variables
  • NP is the size of the pre-set population size
  • rand(0,1) is a uniformly distributed random number in the interval (0,1).
  • the differential evolution algorithm randomly selects two different member vectors in the population for differential processing, and then scales the difference vector and merges with the current optimal member vector to form a new vector to implement individual variation. .
  • v i (g+1) x best (g)+( ⁇ rand(0,1)+F) ⁇ (x R2 (g)+x r1 (g)) to improve the differential strategy, which realizes the diversity of the mutation operation and improves the convergence speed of the algorithm.
  • i ⁇ r1 ⁇ r22 is the scaling factor
  • ⁇ represents the new scaling.
  • the fine tuning parameter, x best (g) is the optimal individual obtained after the evolution of the gth generation
  • x i (g) is the i-th individual of the g-generation population
  • v i (g+1) is the g-generation population.
  • Binomial crossover is used to achieve crossover between individuals
  • the binomial crossover method is adopted to realize the inter-individual cross operation, that is, the g-generation population ⁇ x i (g) ⁇ and the mutated intermediate ⁇ v i (g+1) ⁇ are in accordance with a certain
  • the rules are mixed to achieve cross-operation between individuals, generating the test "chromosome" ⁇ u i (g+1) ⁇ , the formula is as follows: Where CR is the crossover probability and j rand is a random integer of [1, 2, ..., D].
  • a "chromosome” with six gene positions is cross-operated.
  • the first gene to be cross-operated is The j th rand allele "gene” in v i (g+1) was randomly taken as the cross-tested "chromosome” u i (g+1) j rand allele "gene”.
  • Subsequent cross-operations are based on the comparison of the random number and the crossover probability CR to select the allele of x i (g) or v i (g+1) as the allele of u i (g+1). .
  • the differential evolution algorithm uses a greedy criterion to compare the test "chromosome" u i (g+1) with the currently predetermined target member x i (g) in the population to determine the test "chromosome” u i ( g+1) Can it be a member of the next generation population? If it is to solve the optimization problem, then the "chromosome" that better optimizes the objective function will become a member of the next generation population. After the selection operation, all members of the next generation should be better or at least as good as the objective function optimization effect by the corresponding members in the present invention.
  • the test "chromosome" is compared with only one corresponding member. Rather than comparing to all members of an existing population. Specifically, the member x i (g+1) in the next generation population passes the formula Make a choice.
  • the optimization problem to be solved is: minf(x 1 , x 2 ,..., x D ), where D is the dimension of the optimization problem, Respecting the lower and upper limits of the range of values of the jth solution component x j respectively, and stopping when the value of the objective function corresponding to the population vector minimizes the value of the objective function f(x 1 , x 2 , . . . , x D )
  • Initial optimization outputting the best individual in the current population.
  • the evolutionary algebra G G+1 updates the population and continues to evolve.
  • the number of evolutions reaches the preset number of evolutions, the initial optimization is stopped and the best individual in the current population is output.
  • the Levinberg-Marquard (LM) algorithm is used to continue the optimization based on the optimization of the differential evolution algorithm.
  • the relevant parameters of the LM algorithm are set, and the MATLAB neural network toolbox is called.
  • the "trainlm" neural network training function, the best weight vector derived from the differential evolution algorithm and the threshold vector are the initial weight vector and the threshold vector, and the LM algorithm is started to continue to optimize the weight and threshold when Levin Berg-Mar
  • the weight and the threshold are stopped to be optimized, and the optimized weight and threshold are output.
  • BP is set according to the sliding window size of the sliding samples of the training samples.
  • the number of input layer nodes of the neural network, and then, according to the output mode of the BP neural network, the number of output layer nodes of the BP neural network is set, and then, according to the formula Calculate the number of hidden layer nodes in the BP neural network, where N H represents the number of hidden layer nodes, N i and N o represent the number of input layer nodes and the number of output layer nodes, respectively, and L is a constant of 1 to 10
  • the BP neural network is constructed according to the number of input layer nodes, the number of hidden layer nodes, and the number of output layer nodes, thereby improving the application range of the method for recovering by the BP neural network in the embodiment of the present invention.
  • step S402 the training parameters of the BP neural network are set according to the pre-processed training samples, the weights and thresholds optimized by the Levenberg-Magnel algorithm, and the BP neural network is trained.
  • the training samples need to be processed in advance, specifically, a clear image is selected, and the gray image corresponding to the image is blurred to form a blurred image, and the clear image is formed.
  • the fuzzy image is normalized to form a set of training sets, and then the blurred image is sampled by a sliding window operation to obtain a training sample.
  • step S403 it is determined whether the result of the BP neural network output reaches a preset requirement.
  • step S404 is performed to stop training the BP neural network; otherwise, execute S401.
  • the BP neural network is continuously optimized by differential evolution algorithm and Levenberg-Magnell algorithm.
  • a grayscale image Lenna is selected as a clear image of the sample training set, as shown in (a) of FIG.
  • the degree of blurring is mildly blurred
  • the blurred image of the sample training set is obtained, as shown in (b) of Fig. 6.
  • the image is normalized and preprocessed, and then the 3 ⁇ 3 sliding window is used to sample the blurred image to obtain the input vector X.
  • the corresponding pixel is extracted from the clear image to obtain the vector T, and the input vector X and the ideal output T are composed of samples.
  • BP neural network was created, and BP neural network was optimized by DE-LM algorithm.
  • BP neural network was trained by using sample training set vector group (X, T). After training, the blurred image of training set was restored. The restored image is shown in Fig. 6.
  • (c) Select a Gaussian blurred image Cameraman as the test image, first normalize it, then use the sliding window to sample, get the input vector, input it into the trained BP neural network for recovery, after the restoration is completed, first process the network output. The data, forming an image matrix, and then denormalizing the matrix, can obtain a clear image.
  • Test image Gaussian blurred image Cameraman As shown in (d) of FIG. 6, the restored image of the test image is as shown in (e) of FIG. 6, and the degree of clarity of FIG. 6(e) with respect to the blurred image of FIG. 6(d) It has improved a lot, texture information is well preserved, and the peak signal-to-noise ratio is also improved a lot.
  • a gray image Lenna is selected as a clear image of the sample training set, and Lenna is motion blurred using MATLAB's imnoise function.
  • the degree of blur is depth blur, forming a blurred image of the sample training set.
  • the blurred image is sampled by a 9 ⁇ 9 sliding window.
  • the BP neural network training method is similar to the application test shown in Figure 6. After the BP neural network training is completed, the blurred image of the training set is restored. A motion blurred image Woman is selected as the test image, and input to the trained BP neural network for restoration to obtain a restored image.
  • a blurred image and a corresponding clear image are selected as a set of training sets, first normalized, and the image data is transformed from [0, 255] to [0, 1]. Then use the sliding window to scan the blurred image from top to bottom and from left to right as the input vector X of the BP neural network, and extract the corresponding pixel points of the clear image from top to bottom and from left to right to form an ideal output.
  • the vector T, the input vector X and the ideal output vector T constitute a sample set vector group (X, T), create a three-layer BP neural network, input the training sample set vector group to the network, and the weights of the BP neural network layers and
  • the threshold is set as the solution object of the differential evolution algorithm.
  • the global optimization is performed.
  • the differential evolution algorithm ends, and the column is used based on the differential evolution algorithm search.
  • the Wenger-Marquard algorithm continues to optimize and adjust the weights and thresholds of the BP neural network.
  • the optimal initialization value of the BP neural network is obtained, the training parameters are set, the BP neural network is trained, the output of the BP neural network is calculated, and the performance is evaluated. If the performance meets the requirements, the weight of the BP neural network is saved and Threshold, training ends, otherwise, continue training, use the trained BP neural network to restore the target image, and denormalize the image data output by the network to obtain the restored image.
  • Embodiment 3 is a diagrammatic representation of Embodiment 3
  • FIG. 8 is a diagram showing the structure of a device for restoring a blurred image according to Embodiment 3 of the present invention. For the convenience of description, only parts related to the embodiment of the present invention are shown, including:
  • the image normalization unit 81 is configured to normalize the target blurred image to obtain a normalized image when receiving the request for restoring the target blurred image.
  • Embodiments of the present invention are applicable to computing devices, such as personal computers, smart phones, tablets, and the like.
  • the blurred image that needs to be restored may be a type of blurred image such as a Gaussian blurred image, a Poisson noise blurred image, a salt and pepper noise blurred image, or a motion blurred image.
  • the normalization of the image is to maintain the invariance of the affine and improve the accuracy of the calculation.
  • the formula can be adopted. Normalize the pixels of the image and transform the image pixels from [0, 255] to [0, 1], where Represents the value of the kth pixel after normalization, x k represents the value of the kth pixel before normalization, and T represents the number of pixels.
  • the target sample sampling unit 82 is configured to perform sliding sampling on the normalized image through the sliding window to obtain a target sampling sample corresponding to the target blurred image.
  • the sliding window is used for sampling.
  • the size of the sliding window is 3 ⁇ 3, 5 ⁇ 5, 7 ⁇ 7, 9 ⁇ 9, etc.
  • the sliding window is selected differently. For example, for slight blur, select 3 ⁇ 3 or 5 ⁇ 5.
  • Sliding window, for depth blur, select 7 ⁇ 7 or 9 ⁇ 9 sliding window the blur degree of the image can be judged according to the image quality evaluation index Brenner gradient function or Tenengrad gradient function, and the lower the Brenner gradient function value or the Tenengrad gradient function value, Indicates that the image is more blurred.
  • the size of the sampling sliding window of the normalized image is performed according to the degree of blurring of the blurred image. Setting, then, according to the Euclidean distance of the edge pixel point and the center pixel point sampled by the sliding window, the size of the sliding window is reduced to obtain a sliding window of a preset shape, and finally, a sliding window passing the preset shape
  • the normalized image is subjected to sliding sampling, thereby reducing the amount of calculation of the sample and increasing the sampling speed.
  • the sampling area of the 7 ⁇ 7 sliding window is composed of 37 pixels
  • the sampling area of the 9 ⁇ 9 sliding window is composed of 69 pixels.
  • the restoration data acquisition unit 83 is configured to perform restoration processing on the target sampling sample by using a preset optimized BP neural network to obtain restoration data of the target blurred image.
  • a differential evolution algorithm (DE) and a Levenberg-Marquardt (LM) are combined to form a new hybrid algorithm, referred to as DE-
  • DE-LM Levenberg-Marquardt
  • the LM algorithm uses the fuzzy image and its corresponding clear image data samples to learn and train the BP neural network optimized by DE-LM algorithm, and then uses the trained BP neural network to recover the target blurred image.
  • the restored image acquiring unit 84 is configured to perform inverse normalization processing on the obtained restored data to obtain a restored image of the target blurred image.
  • the target blurred image is restored by the optimized BP neural network, the restored data is output, the restored data is processed to form an image matrix, and then the image matrix is inverse normalized to obtain the target blurred image. Restore the image, which is a sharp image.
  • the target sample sampling unit 82 includes:
  • a window size setting unit 821 configured to set a size of a sampling sliding window of the normalized image according to a degree of blurring of the blurred image
  • a window size reducing unit 822 configured to reduce the size of the sliding window according to the Euclidean distance between the edge pixel point and the center pixel point sampled by the sliding window to obtain a sliding window of a preset shape
  • the sample sampling sub-unit 823 is configured to perform sliding sampling on the normalized image by a sliding window of a preset shape.
  • each unit of the restoration device of the blurred image may be implemented by a corresponding hardware or software unit, and each unit may be an independent software and hardware unit, or may be integrated into one soft and hardware unit, and is not limited thereto. this invention.
  • Embodiment 4 is a diagrammatic representation of Embodiment 4:
  • FIG. 10 is a diagram showing the structure of a device for restoring a blurred image according to Embodiment 4 of the present invention. For the convenience of description, only parts related to the embodiment of the present invention are shown, including:
  • the first node setting unit 100 is configured to set the number of input layer nodes of the BP neural network according to the sliding sampling window size of the training sample;
  • a second node setting unit 101 configured to set an output layer node number of the BP neural network according to an output mode of the BP neural network
  • a third node setting unit 102 for using a formula according to Calculate the number of hidden layer nodes in the BP neural network, where N H represents the number of hidden layer nodes, N i and N o represent the number of input layer nodes and the number of output layer nodes, respectively, and L is a constant of 1 to 10 ;
  • the neural network construction unit 103 is configured to construct a BP neural network according to the number of input layer nodes, the number of hidden layer nodes, and the number of output layer nodes;
  • the neural network optimization unit 104 is configured to initially optimize the weights and thresholds of each layer of the pre-built three-layer BP neural network by using a differential evolution algorithm, and use the Levenberg-Magner algorithm to initially optimize the weights. And threshold optimization;
  • the neural network training unit 105 is configured to set the training parameters of the BP neural network according to the pre-processed training samples, the weights and thresholds optimized by the Levenberg-Magnel algorithm, and set the BP neural network Train;
  • the output result interpretation unit 106 is configured to stop training the BP neural network when the result of the BP neural network output reaches a preset requirement, to obtain the optimized BP neural network, otherwise, by differential evolution algorithm and Levenberg - The Manuel algorithm continues to optimize the BP neural network.
  • each unit of the restoration device of the blurred image may be implemented by a corresponding hardware or software unit, and each unit may be an independent software and hardware unit, or may be integrated into one soft and hardware unit, and is not limited thereto. this invention.
  • each unit may be implemented by a corresponding hardware or software unit, and each unit may be an independent software and hardware unit, or may be integrated into one soft and hardware unit, and is not limited thereto.
  • Embodiment 5 is a diagrammatic representation of Embodiment 5:
  • FIG. 11 shows the structure of a computing device provided by Embodiment 5 of the present invention. For the convenience of description, only parts related to the embodiment of the present invention are shown.
  • the computing device 11 of an embodiment of the present invention includes a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110.
  • the processor 110 executes the computer program 112 to implement the steps in the above-described method of restoring the blurred image, such as steps S101 to S104 shown in FIG.
  • the processor 110 when executing the computer program 112, implements the functions of the various units in the various apparatus embodiments described above, such as the functions of the units 81-84 shown in FIG.
  • the processing is performed to obtain a restored image of the target blurred image, thereby improving the restoration accuracy and the recovery speed of the blurred image, thereby improving the sharpness of the restored image.
  • the computing device of the embodiments of the present invention may be a personal computer, a smart phone, and a tablet.
  • the steps that are implemented when the processor 110 performs the computer program 112 to implement the method for restoring the blurred image may refer to the description of the foregoing method embodiments, and details are not described herein again.
  • a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement steps in the method for restoring the blurred image, such as Steps S101 to S104 shown in FIG.
  • the computer program when executed by the processor, implements the functions of the various units in the various apparatus embodiments described above, such as the functions of units 81 through 84 shown in FIG.
  • the processing is performed to obtain a restored image of the target blurred image, thereby improving the restoration accuracy and the recovery speed of the blurred image, thereby improving the sharpness of the restored image.
  • the computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium such as a ROM/RAM, a magnetic disk, an optical disk, a flash memory or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Physiology (AREA)
  • Image Processing (AREA)

Abstract

一种模糊图像的复原方法、装置、设备及存储介质，适用于图像处理技术领域，该方法包括：当接收到对目标模糊图像进行复原的请求时，对目标模糊图像进行归一化处理，得到归一化图像（S101），通过滑动窗口对归一化图像进行滑动采样，以得到目标模糊图像对应的目标采样样本（S102），通过预设的优化BP神经网络对目标采样样本进行复原处理，以得到目标模糊图像的复原数据（S103），对得到的复原数据进行反归一化处理，得到目标模糊图像的复原图像（S104），从而提高了模糊图像的复原精确度和复原速度，进而提高了复原图像的清晰度。

Description

一种模糊图像的复原方法、装置、设备及存储介质 技术领域
本发明属于图像处理技术领域，尤其涉及一种模糊图像的复原方法、装置、设备及存储介质。
背景技术
图像是人类获取视觉信息的主要途径，人类所获取的外界信息绝大部分来自视觉系统所接收的图像信息，但是，在图像的获取、传输以及保存过程中，由于各种原因，例如成像系统的不完善、传输介质的影响、物体与成像系统的相对运动、环境噪声等，都无法避免地会造成图像散焦、失真、噪声干扰等退化现象，由于图像的退化，在图像接收端显示的图像已不再是传输的原始图像，图像呈现效果很差，而在例如光学、医学、天文学以及气象学等诸多研究领域中，通常对图像的清晰度以及质量的要求都比较高，因此，必须对退化的图像进行处理，才能恢复成真实的原始图像，同时，高质量的清晰图像也是计算机视觉和机器学习等技术应用基础，为此，图像复原技术应运而生。图像复原通常都是采用最优化的准则来重建模糊图像，提高图像的质量，从而得到较为清晰的图像。图像复原技术受到了非常多学者的关注和研究，并普遍应用到了医疗成像、卫星成像等领域。
按照模糊类型分类，模糊图像可分为噪声模糊、运动模糊等类型，而噪声模糊又可以分为高斯噪声、泊松噪声以及椒盐噪声等类型。模糊复原方法分为非盲复原法和盲复原法两种，这两种方法都存在一些问题，对于非盲复原法来说，需要已知点扩散函数，而该函数通常未知，而对于盲复原法来说则普遍存在着收敛速度慢、计算量大以及结果不确定性大等问题，同时，这两种模糊复原方法一般只能用于某一类型的模糊复原，存在适用范围小的问题。
发明内容
本发明的目的在于提供一种模糊图像的复原方法、装置、计算设备及存储介质，旨在解决由于现有技术无法提供一种有效的模糊图像的复原方法，导致模糊图像的复原不精确、复原速度慢的问题。
一方面，本发明提供了一种模糊图像的复原方法，所述方法包括下述步骤：
当接收到对目标模糊图像进行复原的请求时，对所述目标模糊图像进行归一化处理，得到归一化图像；
通过滑动窗口对所述归一化图像进行滑动采样，以得到所述目标模糊图像对应的目标采样样本；
通过预设的优化BP神经网络对所述目标采样样本进行复原处理，以得到所述目标模糊图像的复原数据；
对所述得到的复原数据进行反归一化处理，得到所述目标模糊图像的复原图像。
另一方面，本发明提供了一种模糊图像的复原装置，所述装置包括：
图像归一化单元，用于当接收到对目标模糊图像进行复原的请求时，对所述目标模糊图像进行归一化处理，得到归一化图像；
目标样本采样单元，用于通过滑动窗口对所述归一化图像进行滑动采样，以得到所述目标模糊图像对应的目标采样样本；
复原数据获取单元，用于通过预设的优化BP神经网络对所述目标采样样本进行复原处理，以得到所述目标模糊图像的复原数据；以及
复原图像获取单元，用于对所述得到的复原数据进行反归一化处理，得到所述目标模糊图像的复原图像。
另一方面，本发明还提供了一种计算设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如前所述方法的步骤。
另一方面，本发明还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现如前所述方法的步骤。
本发明当接收到对目标模糊图像进行复原的请求时，首先，对目标模糊图像进行归一化处理，得到归一化图像，再通过滑动窗口对归一化图像进行滑动采样，以得到目标模糊图像对应的目标采样样本，然后，通过预设的优化BP神经网络对目标采样样本进行复原处理，以得到目标模糊图像的复原数据，最后，对得到的复原数据进行反归一化处理，得到目标模糊图像的复原图像，从而提高了模糊图像的复原精确度和复原速度，进而提高了复原图像的清晰度。
附图说明
图1是本发明实施例一提供的模糊图像的复原方法的实现流程图；
图2是本发明实施例一提供的模糊图像的复原方法中进行滑动窗口采样的采样过程示例图；
图3是本发明实施例一提供的模糊图像的复原方法中优化的滑动窗口类型示例图；
图4是本发明实施例二提供的模糊图像的复原方法中对BP神经网络进行优化及训练的实现流程图；
图5是本发明实施例二提供的对BP神经网络进行优化及训练的方法中交叉操作示例图；
图6是本发明实施例二提供的模糊图像的复原方法的应用测试示例图；
图7是本发明实施例二提供的模糊图像的复原方法的应用测试示例图；
图8是本发明实施例三提供的模糊图像的复原装置的结构示意图；
图9是本发明实施例三提供的模糊图像的复原装置的优化结构示意图；
图10是本发明实施例四提供的模糊图像的复原装置的结构示意图；以及
图11是本发明实施例五提供的计算设备的结构示意图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
以下结合具体实施例对本发明的具体实现进行详细描述：
实施例一：
图1示出了本发明实施例一提供的模糊图像的复原方法的实现流程，为了便于说明，仅示出了与本发明实施例相关的部分，详述如下：
在步骤S101中，当接收到对目标模糊图像进行复原的请求时，对目标模糊图像进行归一化处理，得到归一化图像。
本发明实施例适用于计算设备，例如，个人计算机、智能手机、平板等。需要复原的模糊图像可以为高斯模糊图像、泊松噪声模糊图像、椒盐噪声模糊图像或运动模糊图像等类型的模糊图像。图像的归一化是为了保持仿射的不变性以及提高计算的精度，在本发明实施例中，可通过公式
Figure PCTCN2018073794-appb-000001
将图像的像素点进行归一化处理，把图像像素点由[0，255]变换到[0，1]，其中，
Figure PCTCN2018073794-appb-000002
表示归一化后第k个像素点的数值，x k表示归一化前第k个像素点的数值，T表示像素点的数目。
在步骤S102中，通过滑动窗口对归一化图像进行滑动采样，以得到目标模糊图像对应的目标采样样本。
在本发明实施例中，由于考虑了邻域的像素点的影响，因此采用滑动窗口进行采样。滑动窗口的大小有3×3、5×5、7×7、9×9等类型，对于模糊程度不同的图像，滑动窗口选取不同，例如，对于轻度模糊，选取3×3或者5×5滑动窗口，对于深度模糊，选取7×7或者9×9滑动窗口，图像的模糊程度可根据图像质量评价指标Brenner梯度函数或Tenengrad梯度函数来判断，Brenner梯度函数 值或Tenengrad梯度函数值越低，表明图像的模糊程度越高。
作为示例地，如图2所示，以3×3滑动窗口为例，介绍模糊图像的采样过程。如图2的2a所示，模糊图像大小为M×N，由于采用了3×3滑动窗口进行采样，图像最外的一层像素点没有足够的邻域，无法成为一个中心像素点，所以第一个中心像素点是第二行第二列对应的像素点P 1，其区域是黑色实线框所包含的像素点，然后将其区域自上而下、从左到右展开，形成列一个如图2的2b所示的列向量，接下来，滑动窗口向右平移一个像素点，第二行第三列对应的像素点作为中心像素点，继续采样，展开获得一个九维列向量。滑动窗口从左到右，依次对第二行进行采样，遍历完第二行后，转到第三行，以第三行像素点为中心像素点进行采样，以此类推，一直遍历完整幅图像，最终形成一个九行[(M-1)×(N-1)]列的矩阵X，该矩阵X作为BP神经网络输入的目标采样样本。
由于随着滑动窗口尺寸变大，导致采样的样本变大，从而增加了采样的计算量，因此，优选地，首先，根据模糊图像的模糊程度，对归一化图像的采样滑动窗口的大小进行设置，然后，根据该滑动窗口采样得到的边缘像素点与中心像素点的欧式距离，对该滑动窗口的尺寸进行缩减，以得到预设形状的滑动窗口，最后，通过该预设形状的滑动窗口对归一化图像进行滑动采样，从而减少采样的计算量，提高了采样速度。
作为示例地，例如，对于5×5大小的滑动窗口，去掉窗口四个角上的一个像素点，其余的21个像素点组成采样区域，如图3的3a所示，对于7×7、9×9滑动窗口，去掉每一个角上的三个点，共减少12个点，7×7滑动窗口的采样区域由37个像素点组成，9×9滑动窗口的采样区域由69个像素点组成，分别如图3的3b、3c所示。优化的5×5、7×7、9×9等类型的滑动窗口采样过程如图2所示一致，都是通过滑动采样。
在步骤S103中，通过预设的优化BP神经网络对目标采样样本进行复原处理，以得到目标模糊图像的复原数据。
在本发明实施例中，将差分进化算法(Differential Evolution Algorithm，简称DE)和列文伯格-马奈尔算法(Levenberg-Marquardt，简称LM)相结合形成一种新的混合算法，简称DE-LM算法，运用模糊图像及其对应的清晰图像数据样本对DE-LM算法优化的BP神经网络进行学习训练，然后再用训练好的BP神经网络对目标模糊图像进行复原，BP神经网络的优化过程请参考后续实施例的描述，在此不再赘述。
在步骤S104中，对得到的复原数据进行反归一化处理，得到目标模糊图像的复原图像。
在本发明实施例中，目标模糊图像经过优化的BP神经网络进行复原，输出复原数据，将该复原数据进行处理，形成图像矩阵，再对图像矩阵进行反归一化处理，得到目标模糊图像的复原图像，即清晰图像。
在本发明实施例中，当接收到对目标模糊图像进行复原的请求时，首先，对目标模糊图像进行归一化处理，得到归一化图像，再通过滑动窗口对归一化图像进行滑动采样，以得到目标模糊图像对应的目标采样样本，然后，通过预设的优化BP神经网络对目标采样样本进行复原处理，以得到目标模糊图像的复原数据，最后，对得到的复原数据进行反归一化处理，得到目标模糊图像的复原图像，从而提高了模糊图像的复原精确度和复原速度，进而提高了复原图像的清晰度。
实施例二：
图4示出了本发明实施例二提供的模糊图像的复原方法中对BP神经网络进行优化及训练的实现流程，为了便于说明，仅示出了与本发明实施例相关的部分，详述如下：
在实施例一的步骤S103中，通过预设的优化BP神经网络对目标采样样本进行复原处理，在进行复原之前，需要通过差分进化算法和列文伯格-马奈尔算法对BP神经网络进行优化及训练，详细过程如下：
在步骤S401中，通过差分进化算法对预先构建的三层BP神经网络各层的 权值和阈值进行初次优化，并通过列文伯格-马奈尔算法对初始优化后的权值和阈值进行优化。
在本发明实施例中，差分进化算法是一种基于种群进化并且能够自组织优化的方法，通过差分进化算法对预先构建的三层BP神经网络各层的权值和阈值进行初次优化的过程可通过下述步骤实现：
(1)对差分进化算法的种群进行初始化
在本发明实施例中，种群初始化采用的是从给定的范围内随机选取的方法，具体地，通过公式
Figure PCTCN2018073794-appb-000003
对初始种群
Figure PCTCN2018073794-appb-000004
进行初始化，设置进化代数G＝0，其中，x i(0)为种群中第0代的第i条“染色体”(或个体)，x j,i(0)为种群中第0代的第i条“染色体”的第j个“基因”，基因即是变量，x j,i(0)对应于三层BP神经网络各层的权值和阈值，D为基因的个数即变量的数量，NP为预先设置的种群规模的大小，rand(0,1)是(0,1)区间内服从均匀分布的随机数。
(2)采用差分策略来实现种群中个体变异
在本发明实施例中，差分进化算法先随机选择种群中两个不同的成员向量进行差分处理，再将其差向量缩放后与当前最优的成员向量进行合并，形成新的向量，实现个体变异。在本发明实施例中，优选地，在保存原来缩放因子的基础上，通过公式v i(g+1)＝x best(g)+(λ·rand(0,1)+F)·(x r2(g)+x r1(g))来改进差分策略，从而实现了变异操作的多样化，提高了算法收敛速度，其中，i≠r1≠r2，F为缩放因子，λ表示新增的缩放微调参数，x best(g)为经过第g代进化后得到的最优个体，x i(g)为第g代种群的第i个个体，v i(g+1)为第g代种群
Figure PCTCN2018073794-appb-000005
通过变异后产生的中间体。
进一步优选地，在种群变异过程中，需要先判断“染色体”中各个“基因”是否符合边界条件，若某个“基因”不符合边界条件，则用随机方式生成新的“基因”替代，从而确保解的有效性。
(3)采用二项式交叉方式来实现个体间的交叉操作
在本发明实施例中，采用二项式交叉方式来实现个体间的交叉操作，即将第g代种群{x i(g)}和变异的中间体{v i(g+1)}按照一定的规则进行混合来实现个体间的交叉操作，生成试验“染色体”{u i(g+1)}，公式如下：
Figure PCTCN2018073794-appb-000006
其中，CR为交叉概率，j rand为[1,2,...,D]的随机整数。
作为示例地，如图5所示，具有六个基因位的“染色体”进行交叉操作。为了较好地保持种群的多样性，需要保证任意的变异中间体{v i(g+1)}的“染色体”至少有一个“基因”遗传到下一代，第一个进行交叉操作的基因是随机取出v i(g+1)中的第j rand位等位“基因”作为交叉后的试验“染色体”u i(g+1)第j rand位等位“基因”。后续的交叉操作过程，则是根据随机数与交叉概率CR大小的对比情况来选取x i(g)或者v i(g+1)的等位基因作为u i(g+1)的等位基因。
(4)采用贪婪准则来实现选择操作，寻求更优成员
在本发明实施例中，差分进化算法采用贪婪准则将试验“染色体”u i(g+1)与种群中当前预定的目标成员x i(g)进行比较，来决定试验“染色体”u i(g+1)能否成为下一代种群中的成员。如果是求解优化问题，那么对目标函数优化效果更好的“染色体”将成为下一代种群的成员。经过选择操作以后，下一代的全部成员都应该比当代对应的成员对目标函数优化效果更优或者至少一样优，在本发明实施例中，试验“染色体”只与一个相对应的成员相比较，而不是与现有种群中的所有成员相比较。具体地，下一代种群中的成员x i(g+1)通过公式
Figure PCTCN2018073794-appb-000007
进行选择。
(5)终止条件判断
选择操作完成后形成新的种群，计算每个种群向量对应的目标函数值，并判断目标函数值是否满足预设的目标函数的终止条件，当满足终止条件时，停止初次优化，输出当前种群中的最佳个体。例如，对于待求解的优化问题为： minf(x 1,x 2,...,x D)，其中，
Figure PCTCN2018073794-appb-000008
D是优化问题的维数，
Figure PCTCN2018073794-appb-000009
分别表示第j个解分量x j取值范围的下限和上限，当种群向量对应的目标函数值使得目标函数f(x 1,x 2,...,x D)的值最小时，则停止初次优化，输出当前种群中的最佳个体。
当目标函数值不满足预设的目标函数的终止条件时，进化代数G＝G+1，更新种群，继续进化。当进化次数达到预设的进化次数时，停止初始优化，输出当前种群中的最佳个体。
在本法实施例中，在差分进化算法寻优的基础上使用列文伯格-马夸尔(LM)算法继续寻优，具体地，设置LM算法的相关参数，调用MATLAB神经网络工具箱中的“trainlm”神经网络训练函数，以差分进化算法进化得到的最好权值向量与阈值向量为初始权值向量与阈值向量，启动LM算法继续优化权值与阈值，当列文伯格-马奈尔算法得到的系统适应值满足预先设定的误差或者迭代次数达到预设的迭代次数时，停止对权值和阈值进行优化，输出该优化好的权值和阈值。
在本发明实施例中，通过差分进化算法对预先构建的三层BP神经网络各层的权值和阈值进行初始优化之前，优选地，首先，根据对训练样本进行滑动采样的滑动窗口大小设置BP神经网络的输入层节点个数，然后，根据BP神经网络的输出模式设置BP神经网络的输出层节点个数，之后，根据公式
Figure PCTCN2018073794-appb-000010
计算BP神经网络隐含层节点个数，其中，N H表示隐含层节点个数，N i和N o分别表示输入层节点个数和输出层节点个数，L为1～10的一个常数，最后，根据输入层节点个数、隐含层节点个数以及输出层节点个数构建BP神经网络，从而提高了本发明实施例中通过BP神经网络进行复原的方法的适用范围。
在步骤S402中，根据预先处理的训练样本、通过所述列文伯格-马奈尔算法优化得到的权值和阈值，对BP神经网络的训练参数进行设置，并对BP神经网络进行训练。
在本发明实施例中，设置BP神经网络的训练参数之前，需要预先处理训练样本，具体地，选取一张清晰图像，将该图像对应的灰度图像进行模糊处理，形成模糊图像，对清晰图像与模糊图像进行归一化处理，组成一组训练集，再通过滑动窗口操作对模糊图像进行采样，获得训练样本。
在步骤S403中，判断BP神经网络输出的结果是否达到预设的要求，当BP神经网络输出的结果达到预设的要求时，则执行S404步骤，停止对BP神经网络的训练，否则，执行S401步骤，通过差分进化算法和列文伯格-马奈尔算法对BP神经网络继续进行优化。
在对本发明实施例的应用进行测试时，作为示例地，选取一张灰度图像Lenna作为样本训练集的清晰图像，如图6的(a)所示。使用MATLAB的imnoise函数对Lenna进行高斯模糊处理，模糊程度为轻度模糊，得到样本训练集的模糊图像，如图6的(b)所示。先对图像进行归一化预处理，再采用3×3滑动窗口对模糊图像进行采样，得到输入向量X，从清晰图像提取对应的像素点得到向量T,输入向量X与理想的输出T组成样本集向量组(X,T)。创建BP神经网络，使用DE-LM算法对BP神经网络进行优化，使用样本训练集向量组(X,T)训练BP神经网络，训练结束后，训练集的模糊图像得到复原，复原图像如图6的(c)所示。选取一张高斯模糊图像Cameraman作为测试图像，先进行归一化处理，再采用滑动窗口进行采样，得到输入向量，输入到训练好的BP神经网络中进行复原，复原完成后，先处理网络输出的数据，形成图像矩阵，再对矩阵反归一化，即能获得清晰的图像。测试图像高斯模糊图像Cameraman如图6的(d)所示，测试图像的复原图像如图6的(e)所示，相对于图6(d)的模糊图像，图6(e)的清晰程度提高了很多，纹理信息保留得很好，峰值信噪比也提高了很多。
作为示例地，在对本发明实施例的另一应用进行测试时，如图7所示，选取一张灰度图像Lenna作为样本训练集的清晰图像，使用MATLAB的imnoise函数对Lenna进行运动模糊处理，模糊程度为深度模糊，形成样本训练集的模 糊图像。采用9×9滑动窗口对模糊图像进行采样，BP神经网络训练方法如同图6所示的应用测试，BP神经网络训练完成后，训练集的模糊图像得到复原。选取一张运动模糊图像Woman作为测试图像，输入到训练好的BP神经网络进行复原，得到复原图像。
在本发明实施例中，选取一张模糊图像及其对应的一张清晰图像作为一组训练集，首先进行归一化处理，把图像数据由[0，255]变换到[0，1]，再采用滑动窗口自上而下、从左到右滑动对模糊图像进行采样作为BP神经网络的输入向量X，自上而下、从左到右提取清晰图像的对应的像素点，形成理想的输出向量T，输入向量X与理想的输出向量T组成样本集向量组(X，T)，创建一个三层BP神经网络，将训练样本集向量组输入到网络，将BP神经网络各层权值和阈值设定为差分进化算法的求解对象，初始化后进行全局寻优，当系统适应值达到设定的误差或是迭代次数达到最大时，差分进化算法结束，在差分进化算法搜索的基础上使用列文伯格-马夸尔算法继续寻优，调整BP神经网络的权值和阈值，当系统适应值达到设定的误差或是迭代次数达到最大时，列文伯格-马夸尔算法结束，得到BP神经网络的最佳初始化值，设定好训练参数，训练BP神经网络，计算BP神经网络的输出，并进行性能评估，若其性能达到要求，则保存BP神经网络权值和阈值，训练结束，否则，继续训练，用训练好的BP神经网络对目标图像进行复原，把网络输出的图像数据进行反归一化处理即可得到复原图像。
实施例三：
图8示出了本发明实施例三提供的模糊图像的复原装置的结构，为了便于说明，仅示出了与本发明实施例相关的部分，其中包括：
图像归一化单元81，用于当接收到对目标模糊图像进行复原的请求时，对目标模糊图像进行归一化处理，得到归一化图像。
本发明实施例适用于计算设备，例如，个人计算机、智能手机、平板等。需要复原的模糊图像可以为高斯模糊图像、泊松噪声模糊图像、椒盐噪声模糊 图像或运动模糊图像等类型的模糊图像。图像的归一化是为了保持仿射的不变性以及提高计算的精度，在本发明实施例中，可通过公式
Figure PCTCN2018073794-appb-000011
将图像的像素点进行归一化处理，把图像像素点由[0，255]变换到[0，1]，其中，
Figure PCTCN2018073794-appb-000012
表示归一化后第k个像素点的数值，x k表示归一化前第k个像素点的数值，T表示像素点的数目。
目标样本采样单元82，用于通过滑动窗口对归一化图像进行滑动采样，以得到目标模糊图像对应的目标采样样本。
在本发明实施例中，由于考虑了邻域的像素点的影响，因此采用滑动窗口进行采样。滑动窗口的大小有3×3、5×5、7×7、9×9等类型，对于模糊程度不同的图像，滑动窗口选取不同，例如，对于轻度模糊，选取3×3或者5×5滑动窗口，对于深度模糊，选取7×7或者9×9滑动窗口，图像的模糊程度可根据图像质量评价指标Brenner梯度函数或Tenengrad梯度函数来判断，Brenner梯度函数值或Tenengrad梯度函数值越低，表明图像的模糊程度越高。
由于随着滑动窗口尺寸变大，导致采样的样本变大，从而增加了采样的计算量，因此，优选地，首先，根据模糊图像的模糊程度，对归一化图像的采样滑动窗口的大小进行设置，然后，根据该滑动窗口采样得到的边缘像素点与中心像素点的欧式距离，对该滑动窗口的尺寸进行缩减，以得到预设形状的滑动窗口，最后，通过该预设形状的滑动窗口对归一化图像进行滑动采样，从而减少采样的计算量，提高了采样速度。
作为示例地，例如，对于5×5大小的滑动窗口，去掉窗口四个角上的一个像素点，其余的21个像素点组成采样区域，对于7×7、9×9滑动窗口，去掉每一个角上的三个点，共减少12个点，7×7滑动窗口的采样区域由37个像素点组成，9×9滑动窗口的采样区域由69个像素点组成。
复原数据获取单元83，用于通过预设的优化BP神经网络对目标采样样本进行复原处理，以得到目标模糊图像的复原数据。
在本发明实施例中，将差分进化算法(Differential Evolution Algorithm，简称DE)和列文伯格-马奈尔算法(Levenberg-Marquardt，简称LM)相结合形成一种新的混合算法，简称DE-LM算法，运用模糊图像及其对应的清晰图像数据样本对DE-LM算法优化的BP神经网络进行学习训练，然后再用训练好的BP神经网络对目标模糊图像进行复原。
复原图像获取单元84，用于对得到的复原数据进行反归一化处理，得到目标模糊图像的复原图像。
在本发明实施例中，目标模糊图像经过优化的BP神经网络进行复原，输出复原数据，将该复原数据进行处理，形成图像矩阵，再对图像矩阵进行反归一化处理，得到目标模糊图像的复原图像，即清晰图像。
因此，如图9所示，优选地，目标样本采样单元82包括：
窗口大小设置单元821，用于根据模糊图像的模糊程度，对归一化图像的采样滑动窗口的大小进行设置；
窗口尺寸缩减单元822，用于根据该滑动窗口采样得到的边缘像素点与中心像素点的欧式距离，对该滑动窗口的尺寸进行缩减，以得到预设形状的滑动窗口；以及
样本采样子单元823，用于通过预设形状的滑动窗口对归一化图像进行滑动采样。
在本发明实施例中，模糊图像的复原装置的各单元可由相应的硬件或软件单元实现，各单元可以为独立的软、硬件单元，也可以集成为一个软、硬件单元，在此不用以限制本发明。
实施例四：
图10示出了本发明实施例四提供的模糊图像的复原装置的结构，为了便于说明，仅示出了与本发明实施例相关的部分，其中包括：
第一节点设置单元100，用于根据训练样本的滑动采样窗口大小，设置BP神经网络的输入层节点个数；
第二节点设置单元101，用于根据BP神经网络的输出模式设置BP神经网络的输出层节点个数；
第三节点设置单元102，用于根据公式
Figure PCTCN2018073794-appb-000013
计算BP神经网络隐含层节点个数，其中，N H表示隐含层节点个数，N i和N o分别表示输入层节点个数和输出层节点个数，L为1～10的一个常数；
神经网络构建单元103，用于根据输入层节点个数、隐含层节点个数以及输出层节点个数构建BP神经网络；
神经网络优化单元104，用于通过差分进化算法对预先构建的三层BP神经网络各层的权值和阈值进行初始优化，并通过列文伯格-马奈尔算法对初始优化后的权值和阈值进行优化；
神经网络训练单元105，用于根据预先处理的训练样本、通过所述列文伯格-马奈尔算法优化得到的权值和阈值，对BP神经网络的训练参数进行设置，并对BP神经网络进行训练；以及
输出结果判读单元106，用于当BP神经网络输出的结果达到预设的要求时，停止对BP神经网络的训练，以得到所述优化BP神经网络，否则，通过差分进化算法和列文伯格-马奈尔算法对BP神经网络继续进行优化。
在本发明实施例中，模糊图像的复原装置的各单元可由相应的硬件或软件单元实现，各单元可以为独立的软、硬件单元，也可以集成为一个软、硬件单元，在此不用以限制本发明。各单元的具体实施方式可参考实施例二的描述，在此不再赘述。
实施例五：
图11示出了本发明实施例五提供的计算设备的结构，为了便于说明，仅示出了与本发明实施例相关的部分。
本发明实施例的计算设备11包括处理器110、存储器111以及存储在存储器111中并可在处理器110上运行的计算机程序112。该处理器110执行计算机程序112时实现上述模糊图像的复原方法实施例中的步骤，例如图1所示的步 骤S101至S104。或者，处理器110执行计算机程序112时实现上述各装置实施例中各单元的功能，例如图8所示单元81至84的功能。
在本发明实施例中，当接收到对目标模糊图像进行复原的请求时，首先，对目标模糊图像进行归一化处理，得到归一化图像，再通过滑动窗口对归一化图像进行滑动采样，以得到目标模糊图像对应的目标采样样本，然后，通过预设的优化BP神经网络对目标采样样本进行复原处理，以得到目标模糊图像的复原数据，最后，对得到的复原数据进行反归一化处理，得到目标模糊图像的复原图像，从而提高了模糊图像的复原精确度和复原速度，进而提高了复原图像的清晰度。
本发明实施例的计算设备可以为个人计算机、智能手机以及平板。该计算设备11中处理器110执行计算机程序112时实现模糊图像的复原方法时实现的步骤可参考前述方法实施例的描述，在此不再赘述。
实施例六：
在本发明实施例中，提供了一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序被处理器执行时实现上述模糊图像的复原方法实施例中的步骤，例如，图1所示的步骤S101至S104。或者，该计算机程序被处理器执行时实现上述各装置实施例中各单元的功能，例如图8所示单元81至84的功能。
在本发明实施例中，当接收到对目标模糊图像进行复原的请求时，首先，对目标模糊图像进行归一化处理，得到归一化图像，再通过滑动窗口对归一化图像进行滑动采样，以得到目标模糊图像对应的目标采样样本，然后，通过预设的优化BP神经网络对目标采样样本进行复原处理，以得到目标模糊图像的复原数据，最后，对得到的复原数据进行反归一化处理，得到目标模糊图像的复原图像，从而提高了模糊图像的复原精确度和复原速度，进而提高了复原图像的清晰度。
本发明实施例的计算机可读存储介质可以包括能够携带计算机程序代码的 任何实体或装置、记录介质，例如，ROM/RAM、磁盘、光盘、闪存等存储器。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种模糊图像的复原方法，其特征在于，所述方法包括下述步骤：
    当接收到对目标模糊图像进行复原的请求时，对所述目标模糊图像进行归一化处理，得到归一化图像；
    通过滑动窗口对所述归一化图像进行滑动采样，以得到所述目标模糊图像对应的目标采样样本；
    通过预设的优化BP神经网络对所述目标采样样本进行复原处理，以得到所述目标模糊图像的复原数据；
    对所述得到的复原数据进行反归一化处理，得到所述目标模糊图像的复原图像。
  2. 如权利要求1所述的方法，其特征在于，通过预设的优化BP神经网络对所述目标采样样本进行复原处理的步骤之前，包括：
    通过差分进化算法对预先构建的三层BP神经网络各层的权值和阈值进行初始优化，并通过列文伯格-马奈尔算法对所述初始优化后的所述权值和所述阈值进行优化；
    根据预先处理的训练样本、通过所述列文伯格-马奈尔算法优化得到的所述权值和所述阈值，对所述BP神经网络的训练参数进行设置，并对所述BP神经网络进行训练；
    当所述BP神经网络输出的结果达到预设的要求时，停止对所述BP神经网络的训练，以得到所述优化BP神经网络，否则，通过所述差分进化算法和所述列文伯格-马奈尔算法对所述BP神经网络继续进行优化。
  3. 如权利要求2所述的方法，其特征在于，通过差分进化算法对预先构建的三层BP神经网络各层的权值和阈值进行初始优化的步骤之前，包括：
    根据所述训练样本的滑动采样窗口大小，设置所述BP神经网络的输入层节点个数；
    根据所述BP神经网络的输出模式设置所述BP神经网络的输出层节点个 数；
    根据公式
    Figure PCTCN2018073794-appb-100001
    计算所述BP神经网络隐含层节点个数，其中，N H表示隐含层节点个数，N i和N o分别表示输入层节点个数和输出层节点个数，L为1～10的一个常数；
    根据所述输入层节点个数、所述隐含层节点个数以及所述输出层节点个数构建所述BP神经网络。
  4. 如权利要求1所述的方法，其特征在于，通过滑动窗口对所述归一化图像进行滑动采样的步骤包括：
    根据所述模糊图像的模糊程度，对所述归一化图像的采样滑动窗口的大小进行设置；
    根据所述滑动窗口采样得到的边缘像素点与中心像素点的欧式距离，对所述滑动窗口的尺寸进行缩减，以得到预设形状的滑动窗口；
    通过所述预设形状的滑动窗口对所述归一化图像进行滑动采样。
  5. 一种模糊图像的复原装置，其特征在于，所述装置包括：
    图像归一化单元，用于当接收到对目标模糊图像进行复原的请求时，对所述目标模糊图像进行归一化处理，得到归一化图像；
    目标样本采样单元，用于通过滑动窗口对所述归一化图像进行滑动采样，以得到所述目标模糊图像对应的目标采样样本；
    复原数据获取单元，用于通过预设的优化BP神经网络对所述目标采样样本进行复原处理，以得到所述目标模糊图像的复原数据；以及
    复原图像获取单元，用于对所述得到的复原数据进行反归一化处理，得到所述目标模糊图像的复原图像。
  6. 如权利要求5所述的装置，其特征在于，所述装置还包括：
    神经网络优化单元，用于通过差分进化算法对预先构建的三层BP神经网络各层的权值和阈值进行初始优化，并通过列文伯格-马奈尔算法对所述初始优化后的所述权值和所述阈值进行优化；
    神经网络训练单元，用于根据预先处理的训练样本、通过所述列文伯格-马奈尔算法优化得到的所述权值和所述阈值，对所述BP神经网络的训练参数进行设置，并对所述BP神经网络进行训练；以及
    输出结果判断单元，用于当所述BP神经网络输出的结果达到预设的要求时，停止对所述BP神经网络的训练，以得到所述优化BP神经网络，否则，通过所述差分进化算法和所述列文伯格-马奈尔算法对所述BP神经网络继续进行优化。
  7. 如权利要求6所述的装置，其特征在于，所述装置还包括：
    第一节点设置单元，用于根据所述训练样本的滑动采样窗口大小，设置所述BP神经网络的输入层节点个数；
    第二节点设置单元，用于根据所述BP神经网络的输出模式设置所述BP神经网络的输出层节点个数；
    第三节点设置单元，用于根据公式
    Figure PCTCN2018073794-appb-100002
    计算所述BP神经网络隐含层节点个数，其中，N H表示隐含层节点个数，N i和N o分别表示输入层节点个数和输出层节点个数，L为1～10的一个常数；以及
    神经网络构建单元，用于根据所述输入层节点个数、所述隐含层节点个数以及所述输出层节点个数构建所述BP神经网络。
  8. 如权利要求5所述的装置，其特征在于，所述目标样本采样单元包括：
    窗口大小设置单元，用于根据所述模糊图像的模糊程度，对所述归一化图像的采样滑动窗口的大小进行设置；
    窗口尺寸缩减单元，用于根据所述滑动窗口采样得到的边缘像素点与中心像素点的欧式距离，对所述滑动窗口的尺寸进行缩减，以得到预设形状的滑动窗口；以及
    样本采样子单元，用于通过所述预设形状的滑动窗口对所述归一化图像进行滑动采样。
  9. 一种计算设备，包括存储器、处理器以及存储在所述存储器中并可在所 述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至4任一项所述方法的步骤。
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至4任一项所述方法的步骤。
PCT/CN2018/073794 2018-01-11 2018-01-23 一种模糊图像的复原方法、装置、设备及存储介质 WO2019136772A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810027495.5A CN108364262A (zh) 2018-01-11 2018-01-11 一种模糊图像的复原方法、装置、设备及存储介质
CN201810027495.5 2018-01-11

Publications (1)

Publication Number Publication Date
WO2019136772A1 true WO2019136772A1 (zh) 2019-07-18

Family

ID=63011043

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/073794 WO2019136772A1 (zh) 2018-01-11 2018-01-23 一种模糊图像的复原方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN108364262A (zh)
WO (1) WO2019136772A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111310385A (zh) * 2020-01-22 2020-06-19 南昌航空大学 一种基于贪婪变异搜索的cmode优化夹具定位方法
CN112100940A (zh) * 2020-09-17 2020-12-18 浙江大学 光纤预制棒一次拉伸工艺参数预测方法及装置
CN113674176A (zh) * 2021-08-23 2021-11-19 北京市商汤科技开发有限公司 图像修复方法及装置、电子设备和存储介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109886891B (zh) * 2019-02-15 2022-01-11 北京市商汤科技开发有限公司 一种图像复原方法及装置、电子设备、存储介质
CN112241670B (zh) * 2019-07-18 2024-03-01 杭州海康威视数字技术股份有限公司 图像处理方法及装置
CN110796584A (zh) * 2019-10-28 2020-02-14 国网内蒙古东部电力有限公司电力科学研究院 运动模糊图像建模方法、装置、存储介质及巡检机器人
CN110838094B (zh) * 2019-11-06 2023-07-04 杭州迪英加科技有限公司 病理切片染色风格转换方法和电子设备
CN113132755B (zh) * 2019-12-31 2022-04-01 北京大学 可扩展人机协同图像编码方法及系统、解码器训练方法
CN112184597A (zh) * 2020-11-05 2021-01-05 温州大学大数据与信息技术研究院 一种图像复原装置及方法
CN112381733B (zh) * 2020-11-13 2022-07-01 四川大学 面向图像恢复的多尺度神经网络结构搜索方法及网络应用
CN112944611A (zh) * 2021-03-19 2021-06-11 珠海格力电器股份有限公司 空调器的控制方法、装置、存储介质及处理器
CN116029911A (zh) * 2021-10-26 2023-04-28 北京字跳网络技术有限公司 一种模糊图像生成方法、网络模型训练方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101266683A (zh) * 2008-05-06 2008-09-17 南京航空航天大学 一种图像复原方法
CN105404902A (zh) * 2015-10-27 2016-03-16 清华大学 基于脉冲神经网络的图像特征描述和记忆方法
CN106127702A (zh) * 2016-06-17 2016-11-16 兰州理工大学 一种基于深度学习的图像去雾算法
CN106127696A (zh) * 2016-06-13 2016-11-16 西安电子科技大学 一种基于bp神经网络拟合运动场的图像去除反射方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740646A (zh) * 2016-01-13 2016-07-06 湖南工业大学 一种基于bp神经网络的蛋白质二级结构预测方法
CN106250812B (zh) * 2016-07-15 2019-08-20 汤一平 一种基于快速r-cnn深度神经网络的车型识别方法
CN107066995A (zh) * 2017-05-25 2017-08-18 中国矿业大学 一种基于卷积神经网络的遥感图像桥梁检测方法
CN107392865B (zh) * 2017-07-01 2020-08-07 广州深域信息科技有限公司 一种人脸图像的复原方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101266683A (zh) * 2008-05-06 2008-09-17 南京航空航天大学 一种图像复原方法
CN105404902A (zh) * 2015-10-27 2016-03-16 清华大学 基于脉冲神经网络的图像特征描述和记忆方法
CN106127696A (zh) * 2016-06-13 2016-11-16 西安电子科技大学 一种基于bp神经网络拟合运动场的图像去除反射方法
CN106127702A (zh) * 2016-06-17 2016-11-16 兰州理工大学 一种基于深度学习的图像去雾算法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MASTER, CHINA MASTER'S THESES FULL-TEXT DATABASE, 15 June 2007 (2007-06-15), pages 26, ISSN: 1671-6779 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111310385A (zh) * 2020-01-22 2020-06-19 南昌航空大学 一种基于贪婪变异搜索的cmode优化夹具定位方法
CN111310385B (zh) * 2020-01-22 2022-08-30 南昌航空大学 一种基于贪婪变异搜索的cmode优化夹具定位方法
CN112100940A (zh) * 2020-09-17 2020-12-18 浙江大学 光纤预制棒一次拉伸工艺参数预测方法及装置
CN113674176A (zh) * 2021-08-23 2021-11-19 北京市商汤科技开发有限公司 图像修复方法及装置、电子设备和存储介质
CN113674176B (zh) * 2021-08-23 2024-04-16 北京市商汤科技开发有限公司 图像修复方法及装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN108364262A (zh) 2018-08-03

Similar Documents

Publication Publication Date Title
WO2019136772A1 (zh) 一种模糊图像的复原方法、装置、设备及存储介质
CN108596258B (zh) 一种基于卷积神经网络随机池化的图像分类方法
CN108510485B (zh) 一种基于卷积神经网络的无参照图像质量评估方法
McCann et al. Convolutional neural networks for inverse problems in imaging: A review
CN106845529B (zh) 基于多视野卷积神经网络的影像特征识别方法
CN108229381B (zh) 人脸图像生成方法、装置、存储介质和计算机设备
CN109754078A (zh) 用于优化神经网络的方法
CN109711426B (zh) 一种基于gan和迁移学习的病理图片分类装置及方法
Hung et al. Robust soft-decision interpolation using weighted least squares
Ren et al. Single image super-resolution via adaptive high-dimensional non-local total variation and adaptive geometric feature
CN109359608B (zh) 一种基于深度学习模型的人脸识别方法
Li et al. Blind image deconvolution through support vector regression
CN107392865B (zh) 一种人脸图像的复原方法
CN110349185B (zh) 一种rgbt目标跟踪模型的训练方法及装置
US20210342977A1 (en) Method And Apparatus For Image Restoration, Storage Medium And Terminal
Kanemura et al. Superresolution with compound Markov random fields via the variational EM algorithm
Lyu et al. DeGAN: Mixed noise removal via generative adversarial networks
CN110648292A (zh) 一种基于深度卷积网络的高噪声图像去噪方法
Chen et al. A soft double regularization approach to parametric blind image deconvolution
CN112580728B (zh) 一种基于强化学习的动态链路预测模型鲁棒性增强方法
CN111224905A (zh) 一种大规模物联网中基于卷积残差网络的多用户检测方法
He et al. Finger vein image deblurring using neighbors-based binary-gan (nb-gan)
Liu et al. Modal regression-based graph representation for noise robust face hallucination
CN106934398A (zh) 基于超像素聚类和稀疏表示的图像去噪方法
CN115860113B (zh) 一种自对抗神经网络模型的训练方法及相关装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18899713

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 07.10.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18899713

Country of ref document: EP

Kind code of ref document: A1