WO2023019685A1 - 一种图像处理方法、系统、装置及存储介质 - Google Patents

一种图像处理方法、系统、装置及存储介质 Download PDF

Info

Publication number
WO2023019685A1
WO2023019685A1 PCT/CN2021/120521 CN2021120521W WO2023019685A1 WO 2023019685 A1 WO2023019685 A1 WO 2023019685A1 CN 2021120521 W CN2021120521 W CN 2021120521W WO 2023019685 A1 WO2023019685 A1 WO 2023019685A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
signal
image processing
neural network
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/120521
Other languages
English (en)
French (fr)
Inventor
戴文睿
郑紫阳
李成林
邹君妮
熊红凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiao Tong University
Original Assignee
Shanghai Jiao Tong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiao Tong University filed Critical Shanghai Jiao Tong University
Publication of WO2023019685A1 publication Critical patent/WO2023019685A1/zh
Priority to US18/472,248 priority Critical patent/US11995801B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present invention relates to the field of signal processing, in particular to an image processing method, system, device and storage medium.
  • Image sparse reconstruction can solve the problems of partial loss of image signals and low image compression efficiency during the compression process due to the large amount of image information. , in order to reconstruct the image required by the application.
  • Chinese patent application CN201811059265.3, CN109345453A discloses a system and method for image super-resolution reconstruction using standardization group sparse regularization, using bilinear interpolation method to obtain the initial estimated value of image super-resolution reconstruction ; Then, using the standardized sparse prior of the image block group, through multiple iterations of the image block group PCA domain adaptive soft threshold filtering and regularized least squares in the image pixel domain, the quality of super-resolution image reconstruction is effectively improved.
  • Chinese patent application CN202110108699.3 and CN112884851A disclose a deep compressed sensing network that expands an iterative optimization algorithm.
  • the deep compressed sensing network application It is a method of adaptively compressing and sampling images at the encoding end, and using an optimization algorithm to guide the design of a deep neural network structure at the decoding end, thereby accurately reconstructing the original image.
  • the reconstruction network module is divided into two parts: the first part expands the Fast Iterative Soft Threshold Shrinkage Optimization Algorithm (FISTA) into a deep neural network form, and the second part uses a set of convolution filters and nonlinear activation functions to reduce the block-wise compressive sensing band. coming block artifacts.
  • FISTA Fast Iterative Soft Threshold Shrinkage Optimization Algorithm
  • the present invention provides an image processing method, system, terminal and storage medium, which introduces unlimited network learning parameters on the basis of iterative soft threshold shrinkage algorithm, and can reconstruct the image required by the application , to meet the current needs of image sparse reconstruction.
  • an image processing method for image sparse reconstruction, image denoising, compressed sensing image reconstruction or image restoration including:
  • the measurement signal is used as the input of the differentiable deep network model, and the sparse signal is used as the target of the differentiable deep network model, and the reverse gradient propagation algorithm is used to optimize the result.
  • the differentiable deep network model is used to obtain a reconstructed sparse signal, and the reconstructed sparse signal is approximated to the sparse signal to obtain a final image, the final image is the product of the dictionary and the reconstructed sparse signal, And make the final image approximate to the training image.
  • the training images are processed to obtain measurement signals, including any of the following:
  • the image processing is image restoration: obtain training images the training images with a 0-1 diagonal sampling matrix multiplied to obtain the measured signal
  • the measurement signal is an image whose missing pixels are to be restored, and the 0-1 diagonal sampling matrix It is a diagonal matrix, and the diagonal elements are either 0 or 1.
  • the establishment of a linear inverse problem of image processing under a norm constraint includes any of the following:
  • x is the reconstructed sparse signal
  • ⁇ is the sampling matrix
  • D is the dictionary
  • ⁇ is the parameter of the constrained regular term
  • ⁇ 1 is a norm of the vector
  • said using an iterative soft threshold shrinkage algorithm to solve the linear inverse problem includes using a standard iterative soft threshold shrinkage algorithm to solve the linear inverse problem, or using a learned iterative soft threshold shrinkage algorithm to solve the linear inverse problem questions, among them:
  • the method of solving the linear inverse problem using a standard iterative soft threshold shrinkage algorithm is specifically: Wherein x n represents the sparse signal obtained by solving the linear inverse problem during the n iteration, AT represents the transposition of matrix A, I represents the identity matrix, and t n represents the step size during the n iteration; Represents the near-end projection operator under a norm constraint
  • the iterative soft-threshold contraction algorithm using learning is used to solve the linear inverse problem, specifically:
  • x n represents the reconstructed sparse signal obtained by solving the linear inverse problem at the nth iteration
  • W n represents the coefficient matrix that needs to be trained by the neural network at the nth iteration
  • ⁇ n represents the nth iteration threshold parameter.
  • the establishment of a differentiable deep network model based on convex combination includes:
  • the loss function of the differentiable deep network model is defined as the mean square error L MSE of the sparse signal reconstruction error in the training data set:
  • N is the number of sparse signals in the training data set
  • x i is the i-th signal according to the deep neural network
  • the reconstructed sparse signal obtained from the measured signal bi .
  • the output x n- 1 of the n-1th block is used as the input to the nth block, and the output x n of the nth block is updated through a convex combination block, specifically including:
  • the first proximal projection according to the input x n-1 , based on the iterative soft threshold shrinkage algorithm to solve, output the first proximal projection v n ;
  • Deep neural network mapping input the first proximal projection v n , map through the deep neural network, and output u n , calculated as in Indicates the deep neural network introduced in the nth iteration, and P n indicates the network parameters that need to be trained by the neural network;
  • the second proximal projection according to the input u n , solve based on the iterative soft threshold shrinkage algorithm, and output the second proximal projection w n ;
  • an image processing system comprising:
  • Training data set building module acquire training images, process the training images to obtain measurement signals; decompose the training images into the product of dictionary and sparse signal, and use the measurement signal and the sparse signal as training sample pairs , to build a training data set;
  • Differentiable deep neural network building module under a norm constraint, establish the linear inverse problem of image processing, use the iterative soft threshold shrinkage algorithm to solve the linear inverse problem, according to the iterative soft threshold shrinkage algorithm and the convexity of the deep neural network Combination, building a differentiable deep neural network of convex combinations;
  • Processing module using the training sample pairs in the training data set, using the measurement signal as the input of the differentiable deep network model, and the sparse signal as the target of the differentiable deep network model, using reverse gradient propagation The algorithm optimizes the differentiable deep network model to obtain a reconstructed sparse signal, and makes the reconstructed sparse signal approximate to the sparse signal to obtain a final image, the final image is the dictionary and the reconstructed sparse signal and make the final image approximate to the training image.
  • an image processing device including a memory, a processor, and a computer program stored in the memory and operable on the processor; the processor can be used to execute the computer program Execute the image processing method described above.
  • a computer-readable storage medium on which a computer program is stored, and the program can be used to execute the above image processing method when executed by a processor.
  • a computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program is operable to cause a computer to perform the image processing method.
  • a chip system including a processor, the processor is coupled to a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor Implement the image processing method described above.
  • a computer device comprising:
  • the processor is used to load and execute the program stored in the memory, so as to realize each step of the image processing method.
  • the present invention has at least one of the following beneficial effects:
  • the image processing method, device and storage medium provided by the present invention not only ensure the accuracy of reconstruction, but also explain the network architecture from the perspective of optimization, and ensure the convergence of network training.
  • the image processing method, device and storage medium provided by the present invention ensure the convergence of network training and at the same time, do not limit the introduced neural network structure, and have higher reconstruction accuracy .
  • the image processing method, device and storage medium provided by the present invention have improved operation speed and reconstruction accuracy.
  • Fig. 1 is the flowchart of the image sparse reconstruction method based on deep neural network provided by a preferred embodiment of the present invention
  • Fig. 2 is the flowchart of setting up network model to solve linear inverse problem in a preferred embodiment of the present invention
  • the image processing method provided by the embodiment of the present invention can be used for image sparse reconstruction, image denoising, compressed sensing image reconstruction or image restoration, wherein, a general linear optimization inverse is established under the one-norm regular constraint of the sparse signal Problem: Based on the iterative soft threshold shrinkage algorithm, a differentiable deep network model based on convex combination is established to solve the problem; a deep neural network with any structure is introduced in the solution step to accelerate the solution step, reducing the number of iterations required for the algorithm to converge.
  • an iterative soft threshold shrinkage algorithm is used to establish a differentiable deep network model based on convex combination to solve the general linear inverse problem, and it is allowed to introduce a deep neural network with a free structure to improve performance, which can be obtained from the measurement signal of the image , recover the sparse signal, and then reconstruct the image according to the sparse signal.
  • the image denoising, compressed sensing image reconstruction and image restoration methods in other embodiments can be regarded as the specific application of the generalized method of this image sparse reconstruction method. The difference between them is that different measurement signals and sampling matrices are defined according to different applications. and linear transformation to form different training sets to train differentiable deep network models for corresponding image denoising, compressed sensing image reconstruction and image restoration applications.
  • FIG. 1 is a flow chart of a method for image sparse reconstruction based on a deep neural network provided by a preferred embodiment of the present invention.
  • the image sparse reconstruction method provided by this preferred embodiment may include the following steps:
  • the measurement signal is a signal that can actually be observed or measured.
  • the training images are known, and the measurement signals are obtained by sampling the training images.
  • the specific meaning of the measurement signal is different in different tasks. For example, in image denoising, it is the actual noisy image, in image restoration, it is the image to be restored with lost pixels, and in compressed sensing reconstruction, it is the measured sampling value. .
  • x is the reconstructed sparse signal
  • ⁇ is the sampling matrix
  • D is the dictionary
  • ⁇ is the parameter of the constrained regular term
  • ⁇ 1 is a norm of the vector.
  • the reconstructed sparse signal x is the output of the differentiable deep network model. sparse signal Since directly from the training images Only for training differentiable deep network models.
  • the above embodiments of the present invention not only ensure the reconstruction accuracy, but also can explain the network architecture from the perspective of optimization, and ensure the convergence of network training.
  • S3 may include the following steps:
  • the linear inverse problem is solved using the standard iterative soft-threshold shrinkage algorithm, specifically: where x n represents the sparse signal obtained by solving the linear inverse problem at the nth iteration, A T represents the transpose of matrix A, I represents the identity matrix, and t n represents the step size at the nth iteration; Represents the near-end projection operator under a norm constraint
  • x n represents the sparse signal obtained by solving the linear inverse problem at the nth iteration
  • W n represents the coefficient matrix that needs to be trained by the neural network at the nth iteration
  • ⁇ n represents the threshold parameter at the nth iteration.
  • S32 constructs a corresponding differentiable deep neural network model according to the iterative soft threshold shrinkage algorithm selected in S31, which can achieve the same level with less time complexity and computational complexity compared with traditional iterative optimization algorithms.
  • the accuracy Compared with the standard deep neural network model, it has theoretical convergence and interpretability based on optimization theory.
  • the output x n-1 of the n-1th block is used as the input to the nth block, and the nth block is updated through convex combination blocks
  • the output x n can include the following steps:
  • the first proximal projection input x n-1 , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the first proximal projection v n ;
  • the calculation steps are: Among them, ⁇ n and t n represent the parameters and step size of the constrained regular term obtained by neural network training at the nth iteration respectively; in the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • deep neural network mapping input the first proximal projection v n , map through the deep neural network, and output u n , the calculation steps are in Indicates the deep neural network introduced in the nth iteration, and P n indicates the network parameters that need to be trained by the neural network.
  • the structure of the deep neural network in S322 is the same in each convex combination block, and any composition structure can be applied, including but not limited to convolutional layer, fully connected layer, nonlinear activation layer, pooling layer, block Normalization layer, layer-skip connection, etc.; the learnable parameters are obtained by learning the mean square error of the sparse signal reconstruction minimized by the backpropagation algorithm, which can accelerate the convergence of the algorithm, making the algorithm compared with the traditional or learned iterative soft
  • the threshold shrinkage algorithm can achieve the same accuracy with less time complexity and computational complexity.
  • the second proximal projection input u n , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the second proximal projection w n ;
  • the calculation steps are: In the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • the measurement signal b is used as the input of the differentiable deep network model, and the sparse signal
  • the output x K of the Kth block is used as the x obtained from the solution
  • the loss function of the differentiable deep network model is optimized by using the reverse gradient propagation algorithm, so that x K is close to the target signal Among them
  • the loss function of the differentiable deep network model is defined as the mean square error of the sparse signal reconstruction error in the training data set Among them, N is the number of sparse signals in the training data set, is the i-th signal in the training data set, For the deep neural network according to the i-th signal The output of the K-th block is obtained from the measured signal bi .
  • S33 utilizes end-to-end network training to obtain network parameters with optimal performance. Compared with hyperparameters that need to be manually selected in traditional iterative optimization algorithms, S33 can be adaptively updated to obtain higher reconstruction performance.
  • S3 introduces a deep neural network model with any reasonable and feasible structure, and constructs a differentiable deep neural network based on convex combination, which can be compared with traditional iterative optimization algorithms in less time complexity and computational complexity. It achieves the same accuracy; compared with the standard deep neural network model, it has theoretical convergence and interpretability based on optimization theory.
  • the deep neural network structure used by the convex combination block is the same, and the learnable parameters are obtained by learning the mean square error of the sparse signal reconstruction minimized by the backpropagation algorithm, which can accelerate the convergence of the algorithm and make the algorithm relatively Compared with the traditional or learned iterative soft threshold shrinkage algorithm, the same accuracy can be achieved with less time complexity and computational complexity.
  • the deep neural network used by the convex combination block can be applied to any composition structure, including but not limited to convolutional layer, fully connected layer, nonlinear activation layer, pooling layer, block normalization layer, skip layer connection, etc.
  • the image sparse reconstruction method based on the deep neural network provided by the above embodiments of the present invention can be applied to solving general linear inverse problems and related practical application scenarios, such as image denoising, compressed sensing, and image restoration.
  • image denoising e.g., image denoising
  • compressed sensing e.g., compressed sensing
  • image restoration e.g., image restoration
  • the image sparse reconstruction method based on deep neural network includes the following three main steps:
  • Step 1 obtain several sparse training images with a dimension of 500 because has sparsity, so the dictionary D can be made equal to the identity matrix, at this time the training image with sparse signals equal.
  • Step 2 obtain several sparse training images with a dimension of 500 because has sparsity, so the dictionary D can be made equal to the identity matrix, at this time the training image with sparse signals equal.
  • Step 3 obtain several sparse training images with a dimension of 500 because has sparsity, so the dictionary D can be made equal to the identity matrix, at this time the training image with sparse signals equal.
  • Combine the measured signal b with the sparse signal as training samples Create a training data set ⁇ ;
  • Step 2 under the one-norm regular constraint of reconstructing the sparse signal x, establish a general linear optimization inverse problem:
  • x is the reconstructed sparse signal
  • the linear transformation matrix A is equal to the sampling matrix ⁇
  • ⁇ is the parameter of the constrained regular term
  • ⁇ 1 is a norm of the vector.
  • Step 3 use the learned iterative soft threshold shrinkage algorithm to solve the above linear optimization inverse problem, establish a differentiable deep network model based on convex combination, and use the training samples in the training data set to pair
  • the measurement signal b is used as the input of the differentiable deep network model, and the sparse signal
  • step 3 use the learned iterative soft threshold shrinkage algorithm to solve the above linear optimization inverse problem, specifically:
  • x n represents the sparse signal obtained by solving the linear inverse problem at the nth iteration
  • W n represents the coefficient matrix that needs to be trained by the neural network at the nth iteration
  • ⁇ n represents the threshold parameter at the nth iteration.
  • step 3 a differentiable deep network model based on convex combination is established, specifically: the sparse signal x 0 when not iterated is obtained by initialization, and 16 convex combination blocks are constructed according to the iterative soft threshold shrinkage algorithm and the convex combination of the deep neural network
  • the output x n- 1 of the n-1th block is used as the input to the nth block, and the output x n corresponding to the nth block is updated and obtained through the convex combination of blocks, which specifically includes the following steps:
  • deep neural network mapping input the first proximal projection v n , map through the deep neural network, and output u n , the calculation steps are in Indicates the deep neural network introduced in the nth iteration, and P n indicates the network parameters that need to be trained by the neural network.
  • the introduced deep neural network contains three convolutional layers with dimensions (9, 1, 16), (9, 16, 16), (9, 16, 1), where the meaning of the dimensions is (The size of the convolution kernel, the feature dimension of the input signal, and the feature dimension of the output signal).
  • each convolutional layer there is a nonlinear activation unit ReLU.
  • the input signal of the first convolutional layer is additively connected to the output signal of the third convolutional layer.
  • step 3 use the training sample pairs in the training data set
  • the measurement signal b is used as the input of the differentiable deep network model, and the sparse signal
  • the measurement signal b is used as the input of the differentiable deep network model, and the sparse signal
  • the output x 16 of the 16th block is used as the x obtained from the solution, and the loss function of the differentiable deep network model is optimized by using the reverse gradient propagation algorithm, so that x 16 is close to the target signal Among them, the loss function of the differentiable deep network model is defined as the mean square error of the sparse signal reconstruction error in the training data set Among them, N is the number of sparse signals in the training data set, is the i-th signal in the training data set, For the deep neural network according to the i-th signal The output of the
  • this embodiment randomly generates different 500-dimensional sparse image signals For model training, until the model reaches the convergence condition, randomly generate 1000 different 500-dimensional sparse image signals for model testing.
  • each corresponding sparse signal with the measured signal b′ as a test sample pair for testing network models.
  • the network structure is designed, and after the model training is completed, the measurement signal b' of 1000 sparse test signals is input into the network to obtain the output x' of the network.
  • NMSE normalized mean square error
  • This embodiment compares other three sparse signal reconstruction methods based on the iterative soft threshold shrinkage algorithm, which are "Theoretical Linear Convergence of Unfolded ISTA” published at the Conference and Workshop on Neural Information Processing Systems (NeurIPS) in 2018. and Its Practical Weights and Thresholds” proposed the LISTA-CP and LISTA-CPSS methods, and "ALISTA: Analytic weights are as good as learned weights in LISTA” published at the International Conference on Learning Representations (ICLR) conference in 2019 The ALISTA method proposed in .
  • the specific comparative test results are shown in Table 1.
  • the evaluation standard is the normalized mean square error (NMSE), the unit is dB, and the lower the value, the better the effect.
  • NMSE normalized mean square error
  • Table 1 NMSE (dB) comparison of LISTA-CP, LISTA-CPSS, ALISTA and the method proposed by the present invention (HLISTA) on each differentiable deep network block during sparse signal reconstruction
  • the present invention provides an image denoising method, which may include the following steps:
  • the linear transformation matrix is a dictionary D
  • ie ⁇ is the parameter of the constrained regular term
  • ⁇ 1 is a norm of the vector.
  • S102 may include the following steps:
  • the linear inverse problem is solved using the standard iterative soft-threshold shrinkage algorithm, specifically: where x n represents the sparse signal obtained by solving the linear inverse problem at the nth iteration, representation matrix The transposition of , I represents the identity matrix, and t n represents the step size of the nth iteration; Represents the near-end projection operator under a norm constraint
  • x n represents the sparse signal obtained by solving the linear inverse problem at the nth iteration
  • W n represents the coefficient matrix that needs to be trained by the neural network at the nth iteration
  • ⁇ n represents the threshold parameter at the nth iteration.
  • the output x n-1 of the n-1th block is used as the input to the nth block, and the output x n of the nth block is updated through convex combination blocks, which may include the following steps:
  • the first proximal projection input x n-1 , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the first proximal projection v n ;
  • the calculation steps are: Among them, ⁇ n and t n represent the parameters and step size of the constrained regular term obtained by neural network training at the nth iteration respectively; in the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • deep neural network mapping input the first proximal projection v n , map through the deep neural network, and output u n , the calculation steps are in Indicates the deep neural network introduced in the nth iteration, and P n indicates the network parameters that need to be trained by the neural network.
  • the structure of the deep neural network in S10222 is the same in each convex combination block, and any composition structure can be applied, including but not limited to convolutional layer, fully connected layer, nonlinear activation layer, pooling layer, block Normalization layer, layer-skip connection, etc.; the learnable parameters are obtained by learning the mean square error of the sparse signal reconstruction minimized by the backpropagation algorithm, which can accelerate the convergence of the algorithm, making the algorithm compared with the traditional or learned iterative soft
  • the threshold shrinkage algorithm can achieve the same accuracy with less time complexity and computational complexity.
  • the second proximal projection input u n , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the second proximal projection w n ;
  • the calculation steps are: In the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • the loss function of the differentiable deep network model is defined as the mean square error of the sparse signal reconstruction error in the training data set Among them, N is the number of sparse signals in the training data set, is the i-th signal in the training data set, for deep neural networks with Corresponding to the output of the Kth block.
  • the de-Gaussian noise of natural images is realized, and compared with the "ALISTA: Analytic weights are as good as learned weights in LISTA" published at the International Conference on Learning Representations (ICLR) conference in 2019.
  • ALISTA method The evaluation criterion is the peak signal-to-noise ratio (PSNR), and the unit is dB.
  • PSNR peak signal-to-noise ratio
  • the experimental results show that under the experimental conditions of Gaussian noise signal-to-noise ratio of 30 decibels, the PSNR of the natural image reconstructed by the system of this embodiment is about 1dB higher than that of the natural image obtained by the ALISTA method, that is, the effect of removing Gaussian noise of the system of this embodiment is excellent. in the ALISTA method.
  • a compressed sensing image reconstruction method which may include the following steps:
  • the linear transformation matrix is the random sampling matrix
  • D the dictionary D
  • ⁇ the parameter of the constrained regular term
  • ⁇ 1 a norm of the vector
  • S202 may include the following steps:
  • the linear inverse problem is solved using the standard iterative soft-threshold shrinkage algorithm, specifically: where x n represents the sparse signal obtained by solving the linear inverse problem at the nth iteration, representation matrix The transposition of , I represents the identity matrix, and t n represents the step size of the nth iteration; Represents the near-end projection operator under a norm constraint
  • x n represents the sparse signal obtained by solving the linear inverse problem at the nth iteration
  • W n represents the coefficient matrix that needs to be trained by the neural network at the nth iteration
  • ⁇ n represents the threshold parameter at the nth iteration.
  • the output x n-1 of the n-1th block is used as the input to the nth block, and the output x n of the nth block is updated through convex combination blocks, which may include the following steps:
  • the first proximal projection input x n-1 , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the first proximal projection v n ;
  • the calculation steps are: Among them, ⁇ n and t n represent the parameters and step size of the constrained regular term obtained by neural network training at the nth iteration respectively; in the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • deep neural network mapping input the first proximal projection v n , map through the deep neural network, and output u n , the calculation steps are in Indicates the deep neural network introduced in the nth iteration, and P n indicates the network parameters that need to be trained by the neural network.
  • the structure of the deep neural network in S20222 is the same in each convex combination block, and any composition structure can be applied, including but not limited to convolutional layer, fully connected layer, nonlinear activation layer, pooling layer, block Normalization layer, layer-skip connection, etc.; the learnable parameters are obtained by learning the mean square error of the sparse signal reconstruction minimized by the backpropagation algorithm, which can accelerate the convergence of the algorithm, making the algorithm compared with the traditional or learned iterative soft
  • the threshold shrinkage algorithm can achieve the same accuracy with less time complexity and computational complexity.
  • the second proximal projection input u n , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the second proximal projection w n ;
  • the calculation steps are: In the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • the loss function of the differentiable deep network model is defined as the mean square error of the sparse signal reconstruction error in the training data set Among them, N is the number of sparse signals in the training data set, is the i-th signal in the training data set, for deep neural networks with Corresponding to the output of the Kth block.
  • the compressed sensing reconstruction of natural images is realized, and three other sparse signal reconstruction methods based on iterative soft threshold shrinkage algorithm are compared, which are presented at the Conference and Workshop on Neural Information Processing Systems in 2018.
  • the LISTA-CP and LISTA-CPSS methods proposed in "Theoretical Linear Convergence of Unfolded ISTA and Its Practical Weights and Thresholds” published at the (NeurIPS) conference, and the 2019 International Conference on Learning Representations (ICLR) conference
  • the ALISTA method proposed in "ALISTA: Analytic weights are as good as learned weights in LISTA".
  • the evaluation criterion is the peak signal-to-noise ratio (PSNR), and the unit is dB.
  • PSNR peak signal-to-noise ratio
  • the experimental results show that under the experimental conditions with a sampling rate of 50%, the PSNR of the natural image reconstructed by the system of this embodiment is higher than that of the natural image obtained by the LISTA-CP method, LISTA-CPSS method and ALISTA method by 0.95 and 0.90 respectively. and 0.98dB.
  • an image restoration method is also provided, which may include the following steps:
  • the linear transformation matrix is a 0-1 diagonal sampling matrix
  • D the dictionary D
  • ⁇ the parameter of the constrained regular term
  • ⁇ 1 is a norm of the vector.
  • S302 may include the following steps:
  • the output x n-1 of the n-1th block is used as the input to the nth block, and the output x n of the nth block is updated through convex combination blocks, which may include the following steps:
  • the first proximal projection input x n-1 , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the first proximal projection v n ;
  • the calculation steps are: Among them, ⁇ n and t n represent the parameters and step size of the constrained regular term obtained by neural network training at the nth iteration respectively; in the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • deep neural network mapping input the first proximal projection v n , map through the deep neural network, and output u n , the calculation steps are in Indicates the deep neural network introduced in the nth iteration, and P n indicates the network parameters that need to be trained by the neural network.
  • the structure of the deep neural network in S30222 is the same in each convex combination block, and any composition structure can be applied, including but not limited to convolutional layer, fully connected layer, nonlinear activation layer, pooling layer, block Normalization layer, layer-skip connection, etc.; the learnable parameters are obtained by learning the mean square error of the sparse signal reconstruction minimized by the backpropagation algorithm, which can accelerate the convergence of the algorithm, making the algorithm compared with the traditional or learned iterative soft
  • the threshold shrinkage algorithm can achieve the same accuracy with less time complexity and computational complexity.
  • the second proximal projection input u n , solve it based on a standard or learned iterative soft threshold shrinkage algorithm, and output the second proximal projection w n ;
  • the calculation steps are: In the learning-based iterative soft threshold shrinkage algorithm, the calculation steps are in Represents the coefficient matrix that needs to be trained by the neural network at the nth iteration.
  • the loss function of the differentiable deep network model is defined as the mean square error of the sparse signal reconstruction error in the training data set Among them, N is the number of sparse signals in the training data set, is the i-th signal in the training data set, for deep neural networks with Corresponding to the output of the Kth block.
  • an image processing system which is used to implement the image processing method in the above embodiment, and realize image sparse reconstruction, image denoising, compressed sensing image reconstruction or Image restoration and other purposes.
  • the image processing system includes:
  • Training data set building module obtain training images, process the training images to obtain measurement signals; decompose the training images into the product of dictionary and sparse signals, use the measurement signals and sparse signals as training sample pairs, and establish training data sets;
  • Differentiable deep neural network building module under the constraint of a norm, establish the linear inverse problem of image processing, use the iterative soft threshold shrinkage algorithm to solve the linear inverse problem, according to the convex combination of the iterative soft threshold shrinkage algorithm and the deep neural network, establish a convex Combined Differentiable Deep Neural Networks;
  • Processing module using the training sample pairs in the training data set, the measurement signal is used as the input of the differentiable deep network model, the sparse signal is used as the target of the differentiable deep network model, and the differentiable deep network model is optimized by using the reverse gradient propagation algorithm to obtain the re Construct the sparse signal, and make the reconstructed sparse signal approximate to the sparse signal to obtain the final image.
  • the final image is the product of the dictionary and the reconstructed sparse signal, and make the final image approximate to the training image.
  • an image processing device including a memory, a processor, and a computer program stored in the memory and operable on the processor.
  • the processor executes the program, it can be used to execute the above-mentioned embodiment. any method.
  • the memory is used to store programs; the memory may include a volatile memory (English: volatile memory), such as a random-access memory (English: random-access memory, abbreviated as RAM), such as a static random-access memory (English: static random-access memory, abbreviation: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviation: DDR SDRAM), etc.; memory can also include non-volatile Non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory).
  • the memory is used to store computer programs (such as application programs and functional modules for implementing the above methods), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above-mentioned computer programs, computer instructions, data, etc. can be invoked by the processor.
  • the above-mentioned computer programs, computer instructions, etc. may be partitioned and stored in one or more memories. And the above-mentioned computer programs, computer instructions, data, etc. can be invoked by the processor.
  • the processor is configured to execute the computer program stored in the memory, so as to implement each step in the method involved in the foregoing embodiments. For details, refer to the related descriptions in the foregoing method embodiments.
  • the processor and the memory can be independent structures or an integrated structure integrated together.
  • the processor and the memory are independent structures, the memory and the processor can be coupled and connected through a bus.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, it can be used to execute the image processing method in any one of the above-mentioned embodiments.
  • a computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the image Approach.
  • a chip system including a processor, the processor is coupled to a memory, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor When realizing the described image processing method.
  • a computer device including: a communication interface, a memory, and a processor, wherein: the memory is used to store a program for implementing the image processing method in any of the above-mentioned embodiments; the processor is used to The program stored in the memory is loaded and executed, so as to realize each step of the image processing method in any one of the above-mentioned embodiments.
  • the communication interface may be used to receive data sent by other devices, and may include an interface for communication between a computer device and other devices, or a communication interface used for communication between components within the computer device.
  • the communication interface may be used to receive data sent by other devices, and may include an interface for communication between a computer device and other devices, or a communication interface used for communication between components within the computer device.
  • the computer device may be a server, a computer, and the like. Therefore, the structural composition of the computer device is not limited to the communication interface, memory, and processor, but may also include other hardware devices, such as other storage devices, etc., which may be determined according to the functions of the computer device.
  • the image processing method, system, device and readable medium provided by the present invention can ensure the reconstruction accuracy and at the same time, can theoretically prove the convergence from the perspective of optimization; compared with the existing The differential deep neural network ensures the convergence and does not limit the introduced neural network structure, and has higher reconstruction accuracy; compared with the traditional iterative optimization algorithm, it improves the operation speed and reconstruction accuracy; the reconstruction method It can be applied to solve general linear inverse problems, and can also be applied to related practical scenarios, such as image denoising, compressed sensing, image restoration, etc.
  • the system provided by the present invention and its various devices can be completely programmed with logic gates, logic gates, The same functions can be realized in the form of switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system provided by the present invention and its various devices can be considered as a hardware component, and the devices included in it for realizing various functions can also be regarded as the structure in the hardware component; Means for implementing various functions can be regarded as either a software module implementing a method or a structure within a hardware component.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Operations Research (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)

Abstract

本发明提供了一种图像处理方法、系统、装置及存储介质，用于图像稀疏重构、图像去噪、压缩感知图像重构或图像复原，包括：在稀疏信号的一范数正则约束下建立一般性的线性优化逆问题；在标准的或学习的迭代软阈值收缩算法基础上，建立基于凸组合的可微分深度网络模型求解该问题；在求解步骤中引入任意结构的深度神经网络加速求解步骤，减少算法达到收敛所需的迭代次数。本发明将传统迭代优化算法和任意结构的深度神经网络结合，提高图像重构性能，并保证快速收敛，满足图像稀疏重构的当前需求。

Description

一种图像处理方法、系统、装置及存储介质 技术领域
本发明涉及信号处理领域，具体地，涉及一种图像处理方法、系统、装置及存储介质。
背景技术
随着媒体技术的发展，海量数据在实时传输、分析、存储方面都面临着巨大的挑战。比如媒体图像在各方面大量应用发展，对图像分辨率以及成像速度等提出更高需求，图像稀疏重构能解决由于图像信息量大而导致压缩过程中图像信号部分丢失、图像压缩效率低的问题，以便重构出应用所需要的图像。
针对上述的技术需求，已有研究人员提出了解决方法，比如中国专利CN201611029446.2，CN106780636A，提出了一种图像的稀疏重构方法及装置，根据信号的压缩感知理论，将图像的重构问题转换为优化问题，并通过图像的压缩矩阵，确定了该优化问题中的待优化目标，对该优化目标进行多次的优化，根据优化结果获得该图像的稀疏系数矩阵，得到重构后的图像，从而不仅实现了图像的重构，而且通过提高整个优化过程的收敛速度和收敛能力，有效地降低了图像重构过程中软硬件资源的消耗，提高了图像的重构效率和重构精。再比如，中国专利申请CN201811059265.3，CN109345453A，公开了一种利用标准化组稀疏规则化的图像超分辨率重构系统及方法，采用双线性插值方法得到图像超分辨率重构的初始估计值；然后再利用图像块组的标准化稀疏先验，通过多次迭代图像块组PCA域自适应软阈值滤波与图像像素域的规则化最小二乘，有效提升超分辨率图像重构的质量。
传统迭代优化方法通常具有收敛性保证，然而优化结构过于僵硬，且性能相较于深度神经网络差距较大。随着深度神经网络的发展，对于深度神经网络的应用也越来越多，比如中国专利申请CN202110108699.3，CN112884851A，公开了一种展开迭代优化算法的深度压缩感知网络，该深度压缩感知网络应用于编码端图像自适应压缩采样，并在解码端利用优化算法引导深度神经网络结构设计，从而精确重建原始图像的方法。重构网络模块分为两部分：第一部分将快速迭代软阈值收缩优化算法(FISTA)展开为深度神经网络形式，第二部分使用一组卷积滤波器和非线性激活函数减少分块压缩感知带来的块伪影现象。
近年来，有学者结合深度学习及传统迭代优化方法，设计在优化理论指导下的可解释的 可微分深度神经网络模型。但是，当前可微分神经网络无法兼顾模型灵活性和理论收敛性，其中一些具备理论收敛性的方法对引入的网络参数有严格的维度和运算限制，而允许引入灵活的网络参数的方法却并不能从优化的角度论证方法的收敛性。因此，很难满足图像稀疏重构的当前需求。
发明内容
本发明针对现有技术的不足，提供了一种图像处理方法、系统、终端及存储介质，在迭代软阈值收缩算法的基础上引入无限制的网络学习参数，能重构出应用所需要的图像，满足图像稀疏重构的当前需求。
根据本发明的第一方面，提供一种图像处理方法，用于图像稀疏重构、图像去噪、压缩感知图像重构或图像复原，包括：
获取训练图像，将所述训练图像进行处理，获得测量信号；
将所述训练图像分解为字典与稀疏信号的乘积，将所述测量信号与所述稀疏信号作为训练样本对，建立训练数据集；
在一范数约束下，建立图像处理的线性逆问题；利用迭代软阈值收缩算法求解所述线性逆问题，根据所述迭代软阈值收缩算法与深度神经网络的凸组合，建立凸组合的可微分深度神经网络；
利用所述训练数据集中的训练样本对，将所述测量信号作为所述可微分深度网络模型的输入，所述稀疏信号作为所述可微分深度网络模型的目标，采用反向梯度传播算法优化所述可微分深度网络模型，得到重构稀疏信号，并使所述重构稀疏信号逼近于所述稀疏信号，得到最终图像，所述最终图像为所述字典与所述重构稀疏信号的乘积，且使所述最终图像逼近于所述训练图像。
可选地，将所述训练图像进行处理，获得测量信号，包括如下任一种：
-当所述图像处理为图像稀疏重构时：获取训练图像
Figure PCTCN2021120521-appb-000001
将所述训练图像
Figure PCTCN2021120521-appb-000002
由采样矩阵Φ进行采样，获得测量信号
Figure PCTCN2021120521-appb-000003
-当所述图像处理为图像去噪时：获取训练图像
Figure PCTCN2021120521-appb-000004
将所述训练图像
Figure PCTCN2021120521-appb-000005
与噪声矩阵相加，获得测量信号
Figure PCTCN2021120521-appb-000006
该测量信号
Figure PCTCN2021120521-appb-000007
为带噪图像；
-当所述图像处理为压缩感知图像重构时：获取训练图像
Figure PCTCN2021120521-appb-000008
将所述训练图像
Figure PCTCN2021120521-appb-000009
与一个随机采样矩阵
Figure PCTCN2021120521-appb-000010
相乘，获得测量信号
Figure PCTCN2021120521-appb-000011
该测量信号
Figure PCTCN2021120521-appb-000012
为采样值；
-当所述图像处理为图像复原时：获取训练图像
Figure PCTCN2021120521-appb-000013
将所述训练图像
Figure PCTCN2021120521-appb-000014
与一个0-1对角采 样矩阵
Figure PCTCN2021120521-appb-000015
相乘，获得测量信号
Figure PCTCN2021120521-appb-000016
该测量信号为丢失像素点待复原的图像，所述0-1对角采样矩阵
Figure PCTCN2021120521-appb-000017
是一个对角矩阵，且对角元素非0即1。
可选地，所述在一范数约束下，建立图像处理的线性逆问题，包括如下任一种：
-当所述图像处理为图像稀疏重构时：在重构稀疏信号x的一范数正则约束下，建立一般性的线性优化逆问题：
Figure PCTCN2021120521-appb-000018
其中，x是重构稀疏信号，A＝ΦD是一个线性变换矩阵，Φ是采样矩阵，D是字典，λ是约束正则项的参数，‖·‖ 1是向量的一范数；
-当所述图像处理为图像去噪时：在一范数约束下，建立图像去噪的线性逆问题：
Figure PCTCN2021120521-appb-000019
其中x是重构稀疏信号，线性变换矩阵
Figure PCTCN2021120521-appb-000020
为字典D，即
Figure PCTCN2021120521-appb-000021
λ是约束正则项的参数，‖·‖ 1是向量的一范数；
-当所述图像处理为压缩感知图像重构时：在一范数约束下，建立压缩感知图像重构的线性逆问题：
Figure PCTCN2021120521-appb-000022
其中x是重构稀疏信号，线性变换矩阵
Figure PCTCN2021120521-appb-000023
是随机采样矩阵
Figure PCTCN2021120521-appb-000024
与字典D的乘积，即
Figure PCTCN2021120521-appb-000025
λ是约束正则项的参数，‖·‖ 1是向量的一范数；
-当所述图像处理为图像复原时：在一范数约束下，建立图像复原的线性逆问题：
Figure PCTCN2021120521-appb-000026
其中x是重构稀疏信号，线性变换矩阵
Figure PCTCN2021120521-appb-000027
是0-1对角采样矩阵
Figure PCTCN2021120521-appb-000028
与字典D的乘积，即
Figure PCTCN2021120521-appb-000029
λ是约束正则项的参数，‖·‖ 1是向量的一范数。
可选地，所述利用迭代软阈值收缩算法求解所述线性逆问题，包括利用标准的迭代软阈值收缩算法求解所述线性逆问题，或，利用学习的迭代软阈值收缩算法求解所述线性逆问题，其中：
所述利用标准的迭代软阈值收缩算法求解所述线性逆问题，具体为：
Figure PCTCN2021120521-appb-000030
Figure PCTCN2021120521-appb-000031
其中x n表示在第n次迭代时求解所述线性逆问题得到的稀疏信号，A T表示矩阵A的转置，I表示单位矩阵，t n表示第n次迭代时的步长；
Figure PCTCN2021120521-appb-000032
表示在一范数约束||·|| 1和超参数λt n下的近端投影算子，具体表示为求解优化问题：
Figure PCTCN2021120521-appb-000033
其中r为近端投影算子表示的优化问题的求解目标变量，a为阈值参数；
所述利用学习的迭代软阈值收缩算法求解所述线性逆问题，具体为：
Figure PCTCN2021120521-appb-000034
Figure PCTCN2021120521-appb-000035
其中x n表示在第n次迭代时求解所述线性逆问题得到的 重构稀疏信号，W n表示在第n次迭代时需要神经网络训练得到的系数矩阵，θ n表示在第n次迭代时的阈值参数。
可选地，所述建立基于凸组合的可微分深度网络模型，包括：
初始化得到未迭代时的稀疏信号x 0，根据迭代软阈值收缩算法与深度神经网络的凸组合，构建由K个凸组合块构成的可微分深度网络模型，分别对应所述迭代软阈值收缩算法的第n＝1至n＝K次迭代，其中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新并得到第n个块对应输出x n，n＝1,…,K。
可选地，所述可微分深度网络模型，其损失函数定义为训练数据集中稀疏信号重构误差的均方误差L MSE：
Figure PCTCN2021120521-appb-000036
上式中，N为训练数据集中稀疏信号的数目，
Figure PCTCN2021120521-appb-000037
为训练数据集中第i个信号，x i为所述深度神经网络根据第i个信号
Figure PCTCN2021120521-appb-000038
的测量信号b i获得的重构稀疏信号。
可选地，所述将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新得到第n个块的输出x n，具体包括：
第一近端投影：根据输入x n-1，基于所述迭代软阈值收缩算法求解，输出第一近端投影v n；
深度神经网络映射：输入第一近端投影v n，经过深度神经网络进行映射，输出u n，计算为
Figure PCTCN2021120521-appb-000039
其中
Figure PCTCN2021120521-appb-000040
表示第n次迭代时引入的深度神经网络，P n表示需要神经网络训练得到的网络参数；
第二近端投影：根据输入u n，基于所述迭代软阈值收缩算法求解，输出第二近端投影w n；
凸组合计算：输入第一近端投影v n和第二近端投影w n，输出第n次迭代的结果x n，计算步骤为x n＝α nv n+(1-α n)w n，其中α n为介于0和1之间的需要神经网络训练得到的凸组合参数。
根据本发明的第二方面，提供一种图像处理系统，包括：
训练数据集建立模块：获取训练图像，将所述训练图像进行处理，获得测量信号；将所述训练图像分解为字典与稀疏信号的乘积，将所述测量信号与所述稀疏信号作为训练样本对，建立训练数据集；
可微分深度神经网络建立模块：在一范数约束下，建立图像处理的线性逆问题，利用迭代软阈值收缩算法求解所述线性逆问题，根据所述迭代软阈值收缩算法与深度神经网络的凸 组合，建立凸组合的可微分深度神经网络；
处理模块：利用所述训练数据集中的训练样本对，将所述测量信号作为所述可微分深度网络模型的输入，所述稀疏信号作为所述可微分深度网络模型的目标，采用反向梯度传播算法优化所述可微分深度网络模型，得到重构稀疏信号，并使所述重构稀疏信号逼近于所述稀疏信号，得到最终图像，所述最终图像为所述字典与所述重构稀疏信号的乘积，且使所述最终图像逼近于所述训练图像。
根据本发明的第三方面，提供一种图像处理装置，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序；所述处理器可用于执行所述计算机程序时可用于执行上述所述的图像处理方法。
根据本发明的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时可用于执行上述的图像处理方法。
根据本发明的第五方面，提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序可操作来使计算机执行所述的图像处理方法。
根据本发明的第六方面，提供一种芯片系统，包括处理器，所述处理器与存储器的耦合，所述存储器存储有程序指令，当所述存储器存储的程序指令被所述处理器执行时实现所述的图像处理方法。
根据本发明的第七方面，提供一种计算机设备，包括：
通信接口；
存储器，用于储存实现所述的图像处理方法的程序；
处理器，用于加载并执行所述存储器存储的程序，以实现所述的图像处理方法的各个步骤。
本发明与现有技术相比，具有如下至少一项的有益效果：
本发明提供的图像处理方法、装置及存储介质，相较于标准的深度神经网络方法，保证了重构精度的同时，可以从优化的角度解释网络架构，保证了网络训练的收敛性。
本发明提供的图像处理方法、装置及存储介质，相较于现有的可微分深度神经网络方法，保证了网络训练的收敛性同时，不限制引入的神经网络结构，具有更高的重构精度。
本发明提供的图像处理方法、装置及存储介质，相较于传统的迭代优化算法，提高了运算速度和重构精度。
附图说明
通过阅读参照以下附图对非限制性实施例所作的详细描述，本发明的其它特征、目的和优点将会变得更明显：
图1为本发明一优选实施例提供的基于深度神经网络的图像稀疏重构方法的流程图；
图2为本发明一优选实施例中建立网络模型求解线性逆问题的流程图；
图3为本发明一优选实施例中第n个凸组合块(n＝1,…,K)的设计方法示意图。
具体实施方式
下面结合具体实施例对本发明进行详细说明。以下实施例将有助于本领域的技术人员进一步理解本发明，但不以任何形式限制本发明。应当指出的是，对本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进。这些都属于本发明的保护范围。以下没有详细说明的部分可以采用现有技术实现。
本发明实施例提供的图像处理方法，可以用于图像稀疏重构、图像去噪、压缩感知图像重构或图像复原，其中，在稀疏信号的一范数正则约束下建立一般性的线性优化逆问题；在迭代软阈值收缩算法基础上，建立基于凸组合的可微分深度网络模型求解该问题；在求解步骤中引入任意结构的深度神经网络加速求解步骤，减少算法达到收敛所需的迭代次数。其中，图像稀疏重构时，采用基于迭代软阈值收缩算法，建立基于凸组合的可微分深度网络模型求解一般线性逆问题，且允许引入自由结构的深度神经网络提升性能，能够从图像的测量信号，恢复出稀疏信号，再根据稀疏信号重构出图像。其他实施例中的图像去噪、压缩感知图像重构和图像复原方法，可以看作这种图像稀疏重构方法一般化方法的具体应用，它们区别在于根据不同应用定义不同的测量信号、采样矩阵和线性变换，构成不同的训练集来训练可微分深度网络模型，用于对应的图像去噪、压缩感知图像重构和图像复原应用。
图1为本发明一优选实施例提供的基于深度神经网络的图像稀疏重构方法的流程图。
如图1所示，该优选实施例提供的图像稀疏重构方法，可以包括如下步骤：
S1，获取训练图像
Figure PCTCN2021120521-appb-000041
将训练图像
Figure PCTCN2021120521-appb-000042
由采样矩阵Φ进行采样，获得测量信号
Figure PCTCN2021120521-appb-000043
将训练图像
Figure PCTCN2021120521-appb-000044
分解为字典D与稀疏信号
Figure PCTCN2021120521-appb-000045
的乘积，将测量信号b与稀疏信号
Figure PCTCN2021120521-appb-000046
作为训练样本对
Figure PCTCN2021120521-appb-000047
建立训练数据集。
测量信号是实际能够观测或者测量得到的信号。在训练时，训练图像已知，测量信号通过采样训练图像获得。具体的，在不同任务中测量信号的具体意义不同，例如图像去噪中为实际得到的有噪声图像，图像复原中为丢失像素点待复原的图像，压缩感知重构中为测量到 的采样值。
S2，在重构稀疏信号x的一范数正则约束下，建立一般性的线性优化逆问题：
Figure PCTCN2021120521-appb-000048
其中，x是重构稀疏信号，A＝ΦD是一个线性变换矩阵，Φ是采样矩阵，D是字典，λ是约束正则项的参数，‖·‖ 1是向量的一范数。
S3，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解上述线性优化逆问题，建立基于凸组合的可微分深度网络模型，利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000049
将测量信号b作为可微分深度网络模型的输入，稀疏信号
Figure PCTCN2021120521-appb-000050
作为可微分深度网络模型的目标，采用反向梯度传播算法优化可微分深度网络模型，得到x并使其逼近于稀疏信号
Figure PCTCN2021120521-appb-000051
得到重构图像y＝Dx并使y逼近于训练图像
Figure PCTCN2021120521-appb-000052
本实施例中，重构稀疏信号x是可微分深度网络模型的输出。稀疏信号
Figure PCTCN2021120521-appb-000053
由于直接来自训练图像
Figure PCTCN2021120521-appb-000054
仅用于训练可微分深度网络模型。
本发明上述实施例相较于标准的深度神经网络的图像重构方法，保证了重构精度的同时，可以从优化的角度解释网络架构，保证了网络训练的收敛性。
如图2所示，作为一优选实施例，S3可以包括如下步骤：
S31，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解线性优化逆问题。
具体的：
利用标准的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000055
Figure PCTCN2021120521-appb-000056
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，A T表示矩阵A的转置，I表示单位矩阵，t n表示第n次迭代时的步长；
Figure PCTCN2021120521-appb-000057
表示在一范数约束||·|| 1和超参数λt n下的近端投影算子，具体表示为求解优化问题：
Figure PCTCN2021120521-appb-000058
Figure PCTCN2021120521-appb-000059
其中r表示上述优化问题中的求解目标，a表示阈值参数。
利用学习的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000060
Figure PCTCN2021120521-appb-000061
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，W n表示在第n次迭代时需要神经网络训练得到的系数矩阵，θ n表示在第n次迭代时的阈值参数。
S32，初始化得到未迭代时的稀疏信号x 0，根据迭代软阈值收缩算法与深度神经网络的凸组合，构建由K个凸组合块构成的可微分深度网络模型，分别对应的标准的或学习的迭代软阈值收缩算法的第n＝1至n＝K次迭代。其中，将第n-1个块的输出x n-1作为对第n个块 的输入，通过凸组合块，更新并得到第n个块对应输出x n，n＝1,…,K。
作为一优选实施例，S32根据S31中选择的迭代软阈值收缩算法，构建相应的可微分深度神经网络模型，相较于传统迭代优化算法可以在更少的时间复杂度和计算复杂度下达到相同的精度；相较于标准的深度神经网络模型具有理论上的可收敛性和基于优化理论的可解释性。
如图3所示，在该过程中，作为一优选实施例，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新得到第n个块的输出x n，可以包括如下步骤：
S321，第一近端投影：输入x n-1，基于标准的或学习的迭代软阈值收缩算法求解，输出第一近端投影v n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000062
Figure PCTCN2021120521-appb-000063
其中λ n和t n分别表示在第n次迭代时需要神经网络训练得到的约束正则项的参数和步长；基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000064
Figure PCTCN2021120521-appb-000065
其中
Figure PCTCN2021120521-appb-000066
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S322，深度神经网络映射：输入第一近端投影v n，经过深度神经网络进行映射，输出u n，计算步骤为
Figure PCTCN2021120521-appb-000067
其中
Figure PCTCN2021120521-appb-000068
表示第n次迭代时引入的深度神经网络，P n表示需要神经网络训练得到的网络参数。
作为一优选实施例，S322中的深度神经网络的结构在各个凸组合块中相同，可适用任意组成结构，包括但不限于卷积层、全连接层、非线性激活层、池化层、块归一化层、跳层连接等；可学习参数通过反向传播算法最小化的稀疏信号重构的均方误差学习得到，可以加速算法的收敛，使算法相较于传统的或学习的迭代软阈值收缩算法，可以在更少的时间复杂度和计算复杂度下达到相同的精度。
S323，第二近端投影：输入u n，基于标准的或学习的迭代软阈值收缩算法求解，输出第二近端投影w n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000069
Figure PCTCN2021120521-appb-000070
基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000071
Figure PCTCN2021120521-appb-000072
其中
Figure PCTCN2021120521-appb-000073
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S324，凸组合计算：输入第一近端投影v n和第二近端投影w n，输出第n次迭代的结果x n，计算步骤为x n＝α nv n+(1-α n)w n，其中α n为介于0和1之间的需要神经网络训练得到的凸组合参数。
S33，利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000074
将测量信号b作为可微分深度网络模型的输入，将稀疏信号
Figure PCTCN2021120521-appb-000075
作为可微分深度网络模型的目标，将第K个块的输出x K作为求解得到的 x，采用反向梯度传播算法优化可微分深度网络模型的损失函数，使x K逼近于目标信号
Figure PCTCN2021120521-appb-000076
其中，将可微分深度网络模型的损失函数定义为训练数据集中稀疏信号重构误差的均方误差
Figure PCTCN2021120521-appb-000077
其中，N为训练数据集中稀疏信号的数目，
Figure PCTCN2021120521-appb-000078
为训练数据集中第i个信号，
Figure PCTCN2021120521-appb-000079
为深度神经网络根据第i个信号
Figure PCTCN2021120521-appb-000080
的测量信号b i获得的第K个块的输出。
上述实施例中，S33利用端到端地网络训练获得最优性能的网络参数，相较于传统迭代优化算法中需要人工选取的超参数，可以自适应的更新以获取更高的重构性能。
上述实施例中，S3引入了任意合理可行的结构的深度神经网络模型，构建了基于凸组合的可微分深度神经网络，相较于传统迭代优化算法可以在更少的时间复杂度和计算复杂度下达到相同的精度；相较于标准的深度神经网络模型具有理论上的可收敛性和基于优化理论的可解释性。
本发明上述实施例中，凸组合块使用的深度神经网络结构相同，可学习参数通过反向传播算法最小化的稀疏信号重构的均方误差学习得到，可以加速算法的收敛，使算法相较于传统的或学习的迭代软阈值收缩算法，可以在更少的时间复杂度和计算复杂度下达到相同的精度。凸组合块使用的的深度神经网络可适用任意组成结构，包括但不限于卷积层、全连接层、非线性激活层、池化层、块归一化层、跳层连接等。
本发明上述实施例提供的基于深度神经网络的图像稀疏重构方法，可应用于一般的线性逆问题求解及其相关实际应用场景，如图像去噪、压缩感知、图像复原等。为了更好理解，以下结合具体应用实例对本发明上述实施例提供的技术方案进一步详细描述如下。
在该具体应用实例中，实现稀疏信号重建，具体的，基于深度神经网络的图像稀疏重构方法包括以下3个主要步骤：
步骤1，获得若干稀疏的维度为500的训练图像
Figure PCTCN2021120521-appb-000081
因
Figure PCTCN2021120521-appb-000082
具有稀疏性，所以可令字典D等于单位矩阵，此时训练图像
Figure PCTCN2021120521-appb-000083
与稀疏信号
Figure PCTCN2021120521-appb-000084
相等。将训练图像
Figure PCTCN2021120521-appb-000085
由给定的维度为(250，500)的采样矩阵Φ进行采样，获得测量信号
Figure PCTCN2021120521-appb-000086
将测量信号b与稀疏信号
Figure PCTCN2021120521-appb-000087
作为训练样本对
Figure PCTCN2021120521-appb-000088
建立训练数据集Γ；
步骤2，在重构稀疏信号x的一范数正则约束下，建立一般性的线性优化逆问题：
Figure PCTCN2021120521-appb-000089
其中，x是重构稀疏信号，线性变换矩阵A与采样矩阵Φ相等，λ是约束正则项的参数，‖·‖ 1是向量的一范数。
步骤3，利用学习的迭代软阈值收缩算法求解上述线性优化逆问题，建立基于凸组合的 可微分深度网络模型，利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000090
将测量信号b作为可微分深度网络模型的输入，稀疏信号
Figure PCTCN2021120521-appb-000091
作为可微分深度网络模型的目标，采用反向梯度传播算法优化可微分深度网络模型，得到x并使其逼近于稀疏信号
Figure PCTCN2021120521-appb-000092
得到重构图像y＝x并使y逼近于训练图像
Figure PCTCN2021120521-appb-000093
步骤3中，利用学习的迭代软阈值收缩算法求解上述线性优化逆问题，具体为：
Figure PCTCN2021120521-appb-000094
Figure PCTCN2021120521-appb-000095
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，W n表示在第n次迭代时需要神经网络训练得到的系数矩阵，θ n表示在第n次迭代时的阈值参数。
步骤3中，建立基于凸组合的可微分深度网络模型，具体为：初始化得到未迭代时的稀疏信号x 0，根据迭代软阈值收缩算法与深度神经网络的凸组合，构建由16个凸组合块构成的可微分深度网络模型，分别对应的标准的或学习的迭代软阈值收缩算法的第n＝1至n＝16次迭代。其中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新并得到第n个块对应输出x n，n＝1,…,16。
上述的将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新并得到第n个块对应输出x n，具体包括如下步骤：
S321，第一近端投影：输入x n-1，基于学习的迭代软阈值收缩算法，输出
Figure PCTCN2021120521-appb-000096
Figure PCTCN2021120521-appb-000097
其中
Figure PCTCN2021120521-appb-000098
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S322，深度神经网络映射：输入第一近端投影v n，经过深度神经网络进行映射，输出u n，计算步骤为
Figure PCTCN2021120521-appb-000099
其中
Figure PCTCN2021120521-appb-000100
表示第n次迭代时引入的深度神经网络，P n表示需要神经网络训练得到的网络参数。
作为一具体应用实例，引入的深度神经网络包含三个维度分别为(9，1，16)、(9，16，16)、(9，16，1)的卷积层，其中维度的含义为(卷积核的尺寸，输入信号的特征维度，输出信号的特征维度)。每个卷积层后有一个非线性激活单元ReLU。第一个卷积层的输入信号和第三个卷积层的输出信号进行加法连接。
S323，第二近端投影：输入u n，基于学习的迭代软阈值收缩算法，输出
Figure PCTCN2021120521-appb-000101
Figure PCTCN2021120521-appb-000102
其中
Figure PCTCN2021120521-appb-000103
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S324，凸组合计算：输入第一近端投影v n和第二近端投影w n，输出第n次迭代的结果x n，计算步骤为x n＝α nv n+(1-α n)w n，其中α n为介于0和1之间的需要神经网络训练得到 的凸组合参数。
步骤3的利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000104
将测量信号b作为可微分深度网络模型的输入，稀疏信号
Figure PCTCN2021120521-appb-000105
作为可微分深度网络模型的目标，根据线性变换矩阵A＝Φ求解的线性优化逆问题，具体为：利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000106
将测量信号b作为可微分深度网络模型的输入，将稀疏信号
Figure PCTCN2021120521-appb-000107
作为可微分深度网络模型的目标，将第16个块的输出x 16作为求解得到的x，采用反向梯度传播算法优化可微分深度网络模型的损失函数，使x 16逼近于目标信号
Figure PCTCN2021120521-appb-000108
其中，将可微分深度网络模型的损失函数定义为训练数据集中稀疏信号重构误差的均方误差
Figure PCTCN2021120521-appb-000109
其中，N为训练数据集中稀疏信号的数目，
Figure PCTCN2021120521-appb-000110
为训练数据集中第i个信号，
Figure PCTCN2021120521-appb-000111
为深度神经网络根据第i个信号
Figure PCTCN2021120521-appb-000112
的测量信号b i获得的第16个块的输出。
在数值试验中，本实施例随机生成不同的500维的稀疏图像信号
Figure PCTCN2021120521-appb-000113
用于模型训练，直至模型达到收敛条件，随机生成1000个不同的500维的稀疏图像信号
Figure PCTCN2021120521-appb-000114
用于模型测试。对于每个稀疏信号
Figure PCTCN2021120521-appb-000115
和
Figure PCTCN2021120521-appb-000116
用维度为(250，500)的高斯随机矩阵Φ作为采样矩阵，得到测量信号b和b′；在图像信号稀疏的情况下，令字典D等于单位矩阵，此时图像信号
Figure PCTCN2021120521-appb-000117
与稀疏信号
Figure PCTCN2021120521-appb-000118
相等，
Figure PCTCN2021120521-appb-000119
与
Figure PCTCN2021120521-appb-000120
相等；将每个对应的稀疏信号
Figure PCTCN2021120521-appb-000121
与测量信号b作为一个训练样本对
Figure PCTCN2021120521-appb-000122
用于网络模型的训练，将每个对应的稀疏信号
Figure PCTCN2021120521-appb-000123
与测量信号b′作为一个测试样本对
Figure PCTCN2021120521-appb-000124
用于网络模型的测试。按照本实施例的基于深度神经网络的图像稀疏重构方法，设计网络结构，待模型训练完毕，将1000个稀疏测试信号的测量信号b′输入网络，得到网络的输出x′。计算测试的稀疏信号
Figure PCTCN2021120521-appb-000125
和深度神经网络输出的x′的归一化均方误差(NMSE)，以此作为性能评估准则，评估本实施例的基于任意神经网络结构的可收敛稀疏信号重构方法的性能。
本实施例比较了其他三种基于迭代软阈值收缩算法展开的稀疏信号重构方法，分别是在2018年的Conference and Workshop on Neural Information Processing Systems(NeurIPS)会议上发表的“Theoretical Linear Convergence of Unfolded ISTA and Its Practical Weights and Thresholds”中提出的LISTA-CP和LISTA-CPSS方法，以及在2019年的International Conference on Learning Representations(ICLR)会议上发表的“ALISTA:Analytic weights are as good as learned weights in LISTA”中提出的ALISTA方法。具体的对比试验结果如表1所示，评估标准为归一化均方误差(NMSE)，单位为dB，数值越低效果越好。
表1实验结果表明，本实施例系统重建出来的信号在重构质量上明显优于另外三种方法得到的信号。另外三种方法均是现有的可微分深度神经网络模型，具有收敛性的保证，但 是由于限制引入的神经网络结构，因此性能收到限制。本实施例在提高重构精度的同时，依然可以保证理论的收敛性。
表1：LISTA-CP、LISTA-CPSS、ALISTA和本发明提出的方法(HLISTA)在稀疏信号重构时每一个可微分深度网络块上的NMSE(dB)比较
Figure PCTCN2021120521-appb-000126
在另一实施例中，本发明提供了一种图像去噪方法，可以包括如下步骤：
S100，获取训练图像
Figure PCTCN2021120521-appb-000127
将训练图像
Figure PCTCN2021120521-appb-000128
与噪声矩阵相加，获得测量信号
Figure PCTCN2021120521-appb-000129
该测量信号
Figure PCTCN2021120521-appb-000130
为带噪图像；将训练图像
Figure PCTCN2021120521-appb-000131
分解为字典D与稀疏信号
Figure PCTCN2021120521-appb-000132
的乘积；将每对稀疏信号
Figure PCTCN2021120521-appb-000133
和带噪图像
Figure PCTCN2021120521-appb-000134
作为一个训练样本对，建立训练数据集。
S101，在一范数约束下，建立图像去噪的线性逆问题：
Figure PCTCN2021120521-appb-000135
其中x是重构稀疏信号，线性变换矩阵
Figure PCTCN2021120521-appb-000136
为字典D，即
Figure PCTCN2021120521-appb-000137
λ是约束正则项的参数，‖·‖ 1是向量的一范数。
S102，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解上述图像去噪的线性逆问题，建立凸组合的可微分深度神经网络，利用训练数据集中的训练样本对，将测量信号
Figure PCTCN2021120521-appb-000138
作为可微分深度网络模型的输入，稀疏信号
Figure PCTCN2021120521-appb-000139
作为可微分深度网络模型的目标，采用反向梯度传播算法优化可微分深度网络模型，得到x并使其逼近于稀疏信号
Figure PCTCN2021120521-appb-000140
得到 重构图像y＝Dx并使y逼近于训练图像
Figure PCTCN2021120521-appb-000141
作为一实施例，S102可以包括如下步骤：
S1021，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解线性优化逆问题，包括：
利用标准的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000142
Figure PCTCN2021120521-appb-000143
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，
Figure PCTCN2021120521-appb-000144
表示矩阵
Figure PCTCN2021120521-appb-000145
的转置，I表示单位矩阵，t n表示第n次迭代时的步长；
Figure PCTCN2021120521-appb-000146
表示在一范数约束||·|| 1和超参数λt n下的近端投影算子，具体表示为求解优化问题：
Figure PCTCN2021120521-appb-000147
Figure PCTCN2021120521-appb-000148
其中r表示上述优化问题中的求解目标，a表示阈值参数；
利用学习的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000149
Figure PCTCN2021120521-appb-000150
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，W n表示在第n次迭代时需要神经网络训练得到的系数矩阵，θ n表示在第n次迭代时的阈值参数。
S1022，初始化得到未迭代时的稀疏信号x 0，根据迭代软阈值收缩算法与深度神经网络的凸组合，构建由K个凸组合块构成的可微分深度网络模型，分别对应的标准的或学习的迭代软阈值收缩算法的第n＝1至n＝K次迭代。其中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新并得到第n个块对应输出x n，n＝1,…,K。
在该过程中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新得到第n个块的输出x n，可以包括如下步骤：
S10221，第一近端投影：输入x n-1，基于标准的或学习的迭代软阈值收缩算法求解，输出第一近端投影v n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000151
Figure PCTCN2021120521-appb-000152
其中λ n和t n分别表示在第n次迭代时需要神经网络训练得到的约束正则项的参数和步长；基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000153
Figure PCTCN2021120521-appb-000154
其中
Figure PCTCN2021120521-appb-000155
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S10222，深度神经网络映射：输入第一近端投影v n，经过深度神经网络进行映射，输出u n，计算步骤为
Figure PCTCN2021120521-appb-000156
其中
Figure PCTCN2021120521-appb-000157
表示第n次迭代时引入的深度神经网络，P n表示需要神经网络训练得到的网络参数。
作为一优选实施例，S10222中的深度神经网络的结构在各个凸组合块中相同，可适用任意组成结构，包括但不限于卷积层、全连接层、非线性激活层、池化层、块归一化层、跳 层连接等；可学习参数通过反向传播算法最小化的稀疏信号重构的均方误差学习得到，可以加速算法的收敛，使算法相较于传统的或学习的迭代软阈值收缩算法，可以在更少的时间复杂度和计算复杂度下达到相同的精度。
S10223，第二近端投影：输入u n，基于标准的或学习的迭代软阈值收缩算法求解，输出第二近端投影w n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000158
Figure PCTCN2021120521-appb-000159
基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000160
Figure PCTCN2021120521-appb-000161
其中
Figure PCTCN2021120521-appb-000162
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S10224，凸组合计算：输入第一近端投影v n和第二近端投影w n，输出第n次迭代的结果x n，计算步骤为x n＝α nv n+(1-α n)w n，其中α n为介于0和1之间的需要神经网络训练得到的凸组合参数。
S1023，利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000163
将测量信号
Figure PCTCN2021120521-appb-000164
作为可微分深度网络模型的输入，将稀疏信号
Figure PCTCN2021120521-appb-000165
作为可微分深度网络模型的目标，将第K个块的输出x K作为求解得到的x，采用反向梯度传播算法优化可微分深度网络模型的损失函数，使x K逼近于目标信号
Figure PCTCN2021120521-appb-000166
其中，将可微分深度网络模型的损失函数定义为训练数据集中稀疏信号重构误差的均方误差
Figure PCTCN2021120521-appb-000167
其中，N为训练数据集中稀疏信号的数目，
Figure PCTCN2021120521-appb-000168
为训练数据集中第i个信号，
Figure PCTCN2021120521-appb-000169
为深度神经网络与
Figure PCTCN2021120521-appb-000170
对应的第K个块的输出。
在该具体应用实例中，实现自然图像的去高斯噪声，并比较了在2019年的International Conference on Learning Representations(ICLR)会议上发表的“ALISTA:Analytic weights are as good as learned weights in LISTA”中提出的ALISTA方法。评估标准为峰值信噪比(PSNR)，单位为dB。实验结果表明，在高斯噪声信噪比30分贝的实验条件下，本实施例系统重建出来的自然图像的PSNR高于ALISTA方法获得的自然图像约1dB，即本实施例系统去高斯噪声的效果优于ALISTA方法。
在本发明另一实施例中，还提供一种压缩感知图像重构方法，可以包括如下步骤：
S200，获取训练图像
Figure PCTCN2021120521-appb-000171
将训练图像
Figure PCTCN2021120521-appb-000172
与一个随机采样矩阵
Figure PCTCN2021120521-appb-000173
相乘，获得测量信号
Figure PCTCN2021120521-appb-000174
该测量信号
Figure PCTCN2021120521-appb-000175
为采样值；将训练图像
Figure PCTCN2021120521-appb-000176
分解为字典D与稀疏信号
Figure PCTCN2021120521-appb-000177
的乘积，将每对稀疏信号
Figure PCTCN2021120521-appb-000178
和采样值
Figure PCTCN2021120521-appb-000179
作为一个训练样本对，建立训练数据集。
S201，在一范数约束下，建立压缩感知图像重构的线性逆问题：
Figure PCTCN2021120521-appb-000180
其中x是重构稀疏信号，线性变换矩阵
Figure PCTCN2021120521-appb-000181
是随机采样矩阵
Figure PCTCN2021120521-appb-000182
与字典D的乘积，即
Figure PCTCN2021120521-appb-000183
λ是约束正则项的参数，‖·‖ 1是向量的一范数。
S202，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解上述压缩感知图像重构的线性逆问题，建立凸组合的可微分深度神经网络，利用训练数据集中的训练样本对，将测量信号
Figure PCTCN2021120521-appb-000184
作为可微分深度网络模型的输入，稀疏信号
Figure PCTCN2021120521-appb-000185
作为可微分深度网络模型的目标，采用反向梯度传播算法优化可微分深度网络模型，得到x并使其逼近于稀疏信号
Figure PCTCN2021120521-appb-000186
得到重构图像y＝Dx并使y逼近于训练图像
Figure PCTCN2021120521-appb-000187
作为一实施例，S202可以包括如下步骤：
S2021，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解线性优化逆问题，包括：
利用标准的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000188
Figure PCTCN2021120521-appb-000189
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，
Figure PCTCN2021120521-appb-000190
表示矩阵
Figure PCTCN2021120521-appb-000191
的转置，I表示单位矩阵，t n表示第n次迭代时的步长；
Figure PCTCN2021120521-appb-000192
表示在一范数约束||·|| 1和超参数λt n下的近端投影算子，具体表示为求解优化问题：
Figure PCTCN2021120521-appb-000193
Figure PCTCN2021120521-appb-000194
其中r表示上述优化问题中的求解目标，a表示阈值参数；
利用学习的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000195
Figure PCTCN2021120521-appb-000196
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，W n表示在第n次迭代时需要神经网络训练得到的系数矩阵，θ n表示在第n次迭代时的阈值参数。
S2022，初始化得到未迭代时的稀疏信号x 0，根据迭代软阈值收缩算法与深度神经网络的凸组合，构建由K个凸组合块构成的可微分深度网络模型，分别对应的标准的或学习的迭代软阈值收缩算法的第n＝1至n＝K次迭代。其中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新并得到第n个块对应输出x n，n＝1,…,K。
在该过程中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新得到第n个块的输出x n，可以包括如下步骤：
S20221，第一近端投影：输入x n-1，基于标准的或学习的迭代软阈值收缩算法求解，输出第一近端投影v n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000197
Figure PCTCN2021120521-appb-000198
其中λ n和t n分别表示在第n次迭代时需要神经网络训练得到的约束正则项的参数和步长；基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000199
Figure PCTCN2021120521-appb-000200
其中
Figure PCTCN2021120521-appb-000201
表示在第n次迭代时需要神经网络训练得到 的系数矩阵。
S20222，深度神经网络映射：输入第一近端投影v n，经过深度神经网络进行映射，输出u n，计算步骤为
Figure PCTCN2021120521-appb-000202
其中
Figure PCTCN2021120521-appb-000203
表示第n次迭代时引入的深度神经网络，P n表示需要神经网络训练得到的网络参数。
作为一优选实施例，S20222中的深度神经网络的结构在各个凸组合块中相同，可适用任意组成结构，包括但不限于卷积层、全连接层、非线性激活层、池化层、块归一化层、跳层连接等；可学习参数通过反向传播算法最小化的稀疏信号重构的均方误差学习得到，可以加速算法的收敛，使算法相较于传统的或学习的迭代软阈值收缩算法，可以在更少的时间复杂度和计算复杂度下达到相同的精度。
S20223，第二近端投影：输入u n，基于标准的或学习的迭代软阈值收缩算法求解，输出第二近端投影w n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000204
Figure PCTCN2021120521-appb-000205
基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000206
Figure PCTCN2021120521-appb-000207
其中
Figure PCTCN2021120521-appb-000208
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S20224，凸组合计算：输入第一近端投影v n和第二近端投影w n，输出第n次迭代的结果x n，计算步骤为x n＝α nv n+(1-α n)w n，其中α n为介于0和1之间的需要神经网络训练得到的凸组合参数。
S2023，利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000209
将测量信号
Figure PCTCN2021120521-appb-000210
作为可微分深度网络模型的输入，将稀疏信号
Figure PCTCN2021120521-appb-000211
作为可微分深度网络模型的目标，将第K个块的输出x K作为求解得到的x，采用反向梯度传播算法优化可微分深度网络模型的损失函数，使x K逼近于目标信号
Figure PCTCN2021120521-appb-000212
其中，将可微分深度网络模型的损失函数定义为训练数据集中稀疏信号重构误差的均方误差
Figure PCTCN2021120521-appb-000213
其中，N为训练数据集中稀疏信号的数目，
Figure PCTCN2021120521-appb-000214
为训练数据集中第i个信号，
Figure PCTCN2021120521-appb-000215
为深度神经网络与
Figure PCTCN2021120521-appb-000216
对应的第K个块的输出。
在该具体应用实例中，实现自然图像的压缩感知重构，并比较了其他三种基于迭代软阈值收缩算法展开的稀疏信号重构方法，分别是在2018年的Conference and Workshop on Neural Information Processing Systems(NeurIPS)会议上发表的“Theoretical Linear Convergence of Unfolded ISTA and Its Practical Weights and Thresholds”中提出的LISTA-CP和LISTA-CPSS方法，以及在2019年的International Conference on Learning Representations(ICLR)会议上发表的“ALISTA:Analytic weights are as good as learned weights in LISTA”中提出的ALISTA方 法。评估标准为峰值信噪比(PSNR)，单位为dB。实验结果表明，在采样率为百分之50的实验条件下，本实施例系统重建出来的自然图像的PSNR分别高于LISTA-CP方法、LISTA-CPSS方法和ALISTA方法获得的自然图像0.95、0.90和0.98dB。
在本发明另一实施例中，还提供一种图像复原方法，可以包括如下步骤：
S300，获取训练图像
Figure PCTCN2021120521-appb-000217
将训练图像
Figure PCTCN2021120521-appb-000218
与一个0-1对角采样矩阵
Figure PCTCN2021120521-appb-000219
相乘，获得测量信号
Figure PCTCN2021120521-appb-000220
Figure PCTCN2021120521-appb-000221
该测量信号为丢失像素点待复原的图像，0-1对角采样矩阵
Figure PCTCN2021120521-appb-000222
是一个对角矩阵，且对角元素非0即1；将训练图像
Figure PCTCN2021120521-appb-000223
分解为字典D与稀疏信号
Figure PCTCN2021120521-appb-000224
的乘积；将每对稀疏信号
Figure PCTCN2021120521-appb-000225
和丢失像素点待复原的图像
Figure PCTCN2021120521-appb-000226
作为一个训练样本对，建立训练数据集。
S301，在一范数约束下，建立图像复原的线性逆问题：
Figure PCTCN2021120521-appb-000227
其中x是重构稀疏信号，线性变换矩阵
Figure PCTCN2021120521-appb-000228
是0-1对角采样矩阵
Figure PCTCN2021120521-appb-000229
与字典D的乘积，即
Figure PCTCN2021120521-appb-000230
λ是约束正则项的参数，‖·‖ 1是向量的一范数。
S302，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解上述图像复原的线性逆问题，建立凸组合的可微分深度神经网络，利用训练数据集中的训练样本对，将测量信号
Figure PCTCN2021120521-appb-000231
作为可微分深度网络模型的输入，稀疏信号
Figure PCTCN2021120521-appb-000232
作为可微分深度网络模型的目标，采用反向梯度传播算法优化可微分深度网络模型，得到x并使其逼近于稀疏信号
Figure PCTCN2021120521-appb-000233
得到重构图像y＝Dx并使y逼近于训练图像
Figure PCTCN2021120521-appb-000234
作为一实施例，S302可以包括如下步骤：
S3021，利用标准的迭代软阈值收缩算法，或，利用学习的迭代软阈值收缩算法求解线性优化逆问题，包括：
S30211，利用标准的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000235
Figure PCTCN2021120521-appb-000236
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，
Figure PCTCN2021120521-appb-000237
表示矩阵
Figure PCTCN2021120521-appb-000238
的转置，I表示单位矩阵，t n表示第n次迭代时的步长；
Figure PCTCN2021120521-appb-000239
表示在一范数约束||·|| 1和超参数λt n下的近端投影算子，具体表示为求解优化问题：
Figure PCTCN2021120521-appb-000240
Figure PCTCN2021120521-appb-000241
其中r表示上述优化问题中的求解目标，a表示阈值参数；
S30212，利用学习的迭代软阈值收缩算法求解线性逆问题，具体为：
Figure PCTCN2021120521-appb-000242
Figure PCTCN2021120521-appb-000243
其中x n表示在第n次迭代时求解线性逆问题得到的稀疏信号，W n表示在第n次迭代时需要神经网络训练得到的系数矩阵，θ n表示在第n次迭代时的阈值参数。
S3022，初始化得到未迭代时的稀疏信号x 0，根据迭代软阈值收缩算法与深度神经网络的凸组合，构建由K个凸组合块构成的可微分深度网络模型，分别对应的标准的或学习的迭代软阈值收缩算法的第n＝1至n＝K次迭代。其中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新并得到第n个块对应输出x n，n＝1,…,K。
在该过程中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新得到第n个块的输出x n，可以包括如下步骤：
S30221，第一近端投影：输入x n-1，基于标准的或学习的迭代软阈值收缩算法求解，输出第一近端投影v n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000244
Figure PCTCN2021120521-appb-000245
其中λ n和t n分别表示在第n次迭代时需要神经网络训练得到的约束正则项的参数和步长；基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000246
Figure PCTCN2021120521-appb-000247
其中
Figure PCTCN2021120521-appb-000248
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S30222，深度神经网络映射：输入第一近端投影v n，经过深度神经网络进行映射，输出u n，计算步骤为
Figure PCTCN2021120521-appb-000249
其中
Figure PCTCN2021120521-appb-000250
表示第n次迭代时引入的深度神经网络，P n表示需要神经网络训练得到的网络参数。
作为一优选实施例，S30222中的深度神经网络的结构在各个凸组合块中相同，可适用任意组成结构，包括但不限于卷积层、全连接层、非线性激活层、池化层、块归一化层、跳层连接等；可学习参数通过反向传播算法最小化的稀疏信号重构的均方误差学习得到，可以加速算法的收敛，使算法相较于传统的或学习的迭代软阈值收缩算法，可以在更少的时间复杂度和计算复杂度下达到相同的精度。
S30223，第二近端投影：输入u n，基于标准的或学习的迭代软阈值收缩算法求解，输出第二近端投影w n；基于标准的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000251
Figure PCTCN2021120521-appb-000252
基于学习的迭代软阈值收缩算法时，计算步骤为
Figure PCTCN2021120521-appb-000253
Figure PCTCN2021120521-appb-000254
其中
Figure PCTCN2021120521-appb-000255
表示在第n次迭代时需要神经网络训练得到的系数矩阵。
S30224，凸组合计算：输入第一近端投影v n和第二近端投影w n，输出第n次迭代的结果x n，计算步骤为x n＝α nv n+(1-α n)w n，其中α n为介于0和1之间的需要神经网络训练得到的凸组合参数。
S3023，利用训练数据集中的训练样本对
Figure PCTCN2021120521-appb-000256
将测量信号
Figure PCTCN2021120521-appb-000257
作为可微分深度网络模型 的输入，将稀疏信号
Figure PCTCN2021120521-appb-000258
作为可微分深度网络模型的目标，将第K个块的输出x K作为求解得到的x，采用反向梯度传播算法优化可微分深度网络模型的损失函数，使x K逼近于目标信号
Figure PCTCN2021120521-appb-000259
其中，将可微分深度网络模型的损失函数定义为训练数据集中稀疏信号重构误差的均方误差
Figure PCTCN2021120521-appb-000260
其中，N为训练数据集中稀疏信号的数目，
Figure PCTCN2021120521-appb-000261
为训练数据集中第i个信号，
Figure PCTCN2021120521-appb-000262
为深度神经网络与
Figure PCTCN2021120521-appb-000263
对应的第K个块的输出。
在该具体应用实例中，实现医学核磁共振图像的复原，并比较了在2018年的IEEE Conference on Computer Vision and Pattern Recognition(CVPR)会议上发表的“ISTA-Net:Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing”中提出的ISTA-Net方法。评估标准为峰值信噪比(PSNR)，单位为dB。实验结果表明，在丢失像素点百分之20的实验条件下，本实施例系统重建出来的医学图像的PSNR高于ISTA-Net方法获得的医学图像约1.5dB。
基于上述相同的技术构思，本发明另一实施例中，提供一种图像处理系统，用于实现上述实施例中的图像处理方法，实现图像稀疏重构、图像去噪、压缩感知图像重构或图像复原等目的。具体的，图像处理系统包括：
训练数据集建立模块：获取训练图像，将训练图像进行处理，获得测量信号；将训练图像分解为字典与稀疏信号的乘积，将测量信号与稀疏信号作为训练样本对，建立训练数据集；
可微分深度神经网络建立模块：在一范数约束下，建立图像处理的线性逆问题，利用迭代软阈值收缩算法求解线性逆问题，根据迭代软阈值收缩算法与深度神经网络的凸组合，建立凸组合的可微分深度神经网络；
处理模块：利用训练数据集中的训练样本对，将测量信号作为可微分深度网络模型的输入，稀疏信号作为可微分深度网络模型的目标，采用反向梯度传播算法优化可微分深度网络模型，得到重构稀疏信号，并使重构稀疏信号逼近于稀疏信号，得到最终图像，最终图像为字典与重构稀疏信号的乘积，且使最终图像逼近于训练图像。
本发明在另一实施例中，还提供一种图像处理装置，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，处理器执行程序时可用于执行上述实施例中任一项的方法。
可选地，存储器，用于存储程序；存储器，可以包括易失性存储器(英文：volatile memory)，例如随机存取存储器(英文：random-access memory，缩写：RAM)，如静态随机存取存储器(英文：static random-access memory，缩写：SRAM)，双倍数据率同步动态随机存取存储器(英文：Double Data Rate Synchronous Dynamic Random Access Memory，缩写：DDR SDRAM) 等；存储器也可以包括非易失性存储器(英文：non-volatile memory)，例如快闪存储器(英文：flash memory)。存储器用于存储计算机程序(如实现上述方法的应用程序、功能模块等)、计算机指令等，上述的计算机程序、计算机指令等可以分区存储在一个或多个存储器中。并且上述的计算机程序、计算机指令、数据等可以被处理器调用。
上述的计算机程序、计算机指令等可以分区存储在一个或多个存储器中。并且上述的计算机程序、计算机指令、数据等可以被处理器调用。
处理器，用于执行存储器存储的计算机程序，以实现上述实施例涉及的方法中的各个步骤。具体可以参见前面方法实施例中的相关描述。
处理器和存储器可以是独立结构，也可以是集成在一起的集成结构。当处理器和存储器是独立结构时，存储器、处理器可以通过总线耦合连接。
在本发明一实施例中，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时可用于执行上述实施例中任一项的图像处理方法。
在本发明一实施例中，还提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序可操作来使计算机执行所述的图像处理方法。
在本发明一实施例中，还提供一种芯片系统，包括处理器，所述处理器与存储器的耦合，所述存储器存储有程序指令，当所述存储器存储的程序指令被所述处理器执行时实现所述的图像处理方法。
在本发明一实施例中，还提供一种计算机设备，包括：通信接口、存储器和处理器，其中：存储器，用于储存实现上述任一项实施例中的图像处理方法的程序；处理器用于加载并执行所述存储器存储的程序，以实现上述任一项实施例中的图像处理方法的各个步骤。
通信接口、存储器和处理器的数量可以为至少一个，且通信接口、存储器和处理器相互之间可以通过通信总线进行通信。通信接口可以用来接收其他设备发送的数据，可以包括计算机设备与其他设备进行通信的接口，也可以包括该计算机设备内部各部件之间通信所使用的通信接口。处理器和存储器的具体说明可以参照以上实施例，不再说明。
在实际应用中，该计算机设备可以是服务器、电脑等。因此，计算机设备的结构组成并不局限于通信接口、存储器和处理器，还可以包括其他硬件设备，如其他存储设备等等，可以根据该计算机设备所具有的功能确定。
本发明提供的图像处理方法、系统、装置及可读介质，相较于标准的深度神经网络，保证了重构精度的同时，可以从优化的角度理论证明收敛性；相较于现有的可微分深度神经网 络，保证了收敛性的同时，不限制引入的神经网络结构，具有更高的重构精度；相较于传统的迭代优化算法，提高了运算速度和重构精度；该重构方法可应用于一般的线性逆问题求解，同时可以应用于相关实际场景，如图像去噪、压缩感知、图像复原等。
需要说明的是，本发明提供的方法中的步骤，可以利用系统中对应的模块、装置、单元等予以实现，本领域技术人员可以参照方法的技术方案实现系统的组成，即，方法中的实施例可理解为构建系统的优选例，在此不予赘述。
本领域技术人员知道，除了以纯计算机可读程序代码方式实现本发明提供的系统及其各个装置以外，完全可以通过将方法步骤进行逻辑编程来使得本发明提供的系统及其各个装置以逻辑门、开关、专用集成电路、可编程逻辑控制器以及嵌入式微控制器等的形式来实现相同功能。所以，本发明提供的系统及其各项装置可以被认为是一种硬件部件，而对其内包括的用于实现各种功能的装置也可以视为硬件部件内的结构；也可以将用于实现各种功能的装置视为既可以是实现方法的软件模块又可以是硬件部件内的结构。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的系统、装置、计算机设备而言，由于其与实施例公开的方法对应，所以描述的比较简单，相关之处参见方法部分说明即可。
以上对本发明的具体实施例进行了描述。需要理解的是，本发明并不局限于上述特定实施方式，本领域技术人员可以在权利要求的范围内做出各种变形或修改，这并不影响本发明的实质内容。

Claims (13)

  1. 一种图像处理方法，用于图像稀疏重构、图像去噪、压缩感知图像重构或图像复原，其特征在于，包括：
    获取训练图像，将所述训练图像进行处理，获得测量信号；
    将所述训练图像分解为字典与稀疏信号的乘积，将所述测量信号与所述稀疏信号作为训练样本对，建立训练数据集；
    在一范数约束下，建立图像处理的线性逆问题；利用迭代软阈值收缩算法求解所述线性逆问题，根据所述迭代软阈值收缩算法与深度神经网络的凸组合，建立凸组合的可微分深度神经网络；
    利用所述训练数据集中的训练样本对，将所述测量信号作为所述可微分深度网络模型的输入，所述稀疏信号作为所述可微分深度网络模型的目标，采用反向梯度传播算法优化所述可微分深度网络模型，得到重构稀疏信号，并使所述重构稀疏信号逼近于所述稀疏信号，得到最终图像，所述最终图像为所述字典与所述重构稀疏信号的乘积，且使所述最终图像逼近于所述训练图像。
  2. 根据权利要求1所述的图像处理方法，其特征在于，将所述训练图像进行处理，获得测量信号，包括如下任一种：
    -当所述图像处理为图像稀疏重构时：获取训练图像
    Figure PCTCN2021120521-appb-100001
    将所述训练图像
    Figure PCTCN2021120521-appb-100002
    由采样矩阵Φ进行采样，获得测量信号
    Figure PCTCN2021120521-appb-100003
    -当所述图像处理为图像去噪时：获取训练图像
    Figure PCTCN2021120521-appb-100004
    将所述训练图像
    Figure PCTCN2021120521-appb-100005
    与噪声矩阵相加，获得测量信号
    Figure PCTCN2021120521-appb-100006
    该测量信号
    Figure PCTCN2021120521-appb-100007
    为带噪图像；
    -当所述图像处理为压缩感知图像重构时：获取训练图像
    Figure PCTCN2021120521-appb-100008
    将所述训练图像
    Figure PCTCN2021120521-appb-100009
    与一个随机采样矩阵
    Figure PCTCN2021120521-appb-100010
    相乘，获得测量信号
    Figure PCTCN2021120521-appb-100011
    该测量信号
    Figure PCTCN2021120521-appb-100012
    为采样值；
    -当所述图像处理为图像复原时：获取训练图像
    Figure PCTCN2021120521-appb-100013
    将所述训练图像
    Figure PCTCN2021120521-appb-100014
    与一个0-1对角采样矩阵
    Figure PCTCN2021120521-appb-100015
    相乘，获得测量信号
    Figure PCTCN2021120521-appb-100016
    该测量信号为丢失像素点待复原的图像，所述0-1对角采样矩阵
    Figure PCTCN2021120521-appb-100017
    是一个对角矩阵，且对角元素非0即1。
  3. 根据权利要求2所述的图像处理方法，其特征在于，所述在一范数约束下，建立图像处理的线性逆问题，包括如下任一种：
    -当所述图像处理为图像稀疏重构时：在重构稀疏信号x的一范数正则约束下，建立一般性的线性优化逆问题：
    Figure PCTCN2021120521-appb-100018
    其中，x是重构稀疏信号，A＝ΦD是一个线性变换矩阵，Φ是采样矩阵，D是字典，λ是约束正则项的参数，‖·‖ 1是向量的一范数；
    -当所述图像处理为图像去噪时：在一范数约束下，建立图像去噪的线性逆问题：
    Figure PCTCN2021120521-appb-100019
    其中x是重构稀疏信号，线性变换矩阵
    Figure PCTCN2021120521-appb-100020
    为字典D，即
    Figure PCTCN2021120521-appb-100021
    Figure PCTCN2021120521-appb-100022
    λ是约束正则项的参数，‖·‖ 1是向量的一范数；
    -当所述图像处理为压缩感知图像重构时：在一范数约束下，建立压缩感知图像重构的线性逆问题：
    Figure PCTCN2021120521-appb-100023
    其中x是重构稀疏信号，线性变换矩阵
    Figure PCTCN2021120521-appb-100024
    是随机采样矩阵
    Figure PCTCN2021120521-appb-100025
    与字典D的乘积，即
    Figure PCTCN2021120521-appb-100026
    λ是约束正则项的参数，‖·‖ 1是向量的一范数；
    -当所述图像处理为图像复原时：在一范数约束下，建立图像复原的线性逆问题：
    Figure PCTCN2021120521-appb-100027
    其中x是重构稀疏信号，线性变换矩阵
    Figure PCTCN2021120521-appb-100028
    是0-1对角采样矩阵
    Figure PCTCN2021120521-appb-100029
    与字典D的乘积，即
    Figure PCTCN2021120521-appb-100030
    λ是约束正则项的参数，‖·‖ 1是向量的一范数。
  4. 根据权利要求1所述的图像处理方法，其特征在于，所述利用迭代软阈值收缩算法求解所述线性逆问题，包括利用标准的迭代软阈值收缩算法求解所述线性逆问题，或，利用学习的迭代软阈值收缩算法求解所述线性逆问题，其中：
    所述利用标准的迭代软阈值收缩算法求解所述线性逆问题，具体为：
    Figure PCTCN2021120521-appb-100031
    Figure PCTCN2021120521-appb-100032
    其中x n表示在第n次迭代时求解所述线性逆问题得到的稀疏信号，A T表示矩阵A的转置，I表示单位矩阵，t n表示第n次迭代时的步长；
    Figure PCTCN2021120521-appb-100033
    表示在一范数约束||·|| 1和超参数λt n下的近端投影算子，具体表示为求解优化问题：
    Figure PCTCN2021120521-appb-100034
    其中r为近端投影算子表示的优化问题的求解目标变量，a为阈值参数；
    所述利用学习的迭代软阈值收缩算法求解所述线性逆问题，具体为：
    Figure PCTCN2021120521-appb-100035
    Figure PCTCN2021120521-appb-100036
    其中x n表示在第n次迭代时求解所述线性逆问题得到的重构稀疏信号，W n表示在第n次迭代时需要神经网络训练得到的系数矩阵，θ n表示在第n次迭代时的阈值参数。
  5. 根据权利要求1所述的图像处理方法，其特征在于，所述建立基于凸组合的可微分深度网络模型，包括：
    初始化得到未迭代时的稀疏信号x 0，根据迭代软阈值收缩算法与深度神经网络的凸组合，构建由K个凸组合块构成的可微分深度网络模型，分别对应所述迭代软阈值收缩算法的第n＝1至n＝K次迭代，其中，将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新并得到第n个块对应输出x n，n＝1,…,K。
  6. 根据权利要求5所述的图像处理方法，其特征在于，所述可微分深度网络模型，其损失函数定义为训练数据集中稀疏信号重构误差的均方误差L MSE：
    Figure PCTCN2021120521-appb-100037
    上式中，N为训练数据集中稀疏信号的数目，
    Figure PCTCN2021120521-appb-100038
    为训练数据集中第i个信号，x i为所述深度神经网络根据第i个信号
    Figure PCTCN2021120521-appb-100039
    的测量信号b i获得的重构稀疏信号。
  7. 根据权利要求5所述的图像处理方法，其特征在于，所述将第n-1个块的输出x n-1作为对第n个块的输入，通过凸组合块，更新得到第n个块的输出x n，具体包括：
    第一近端投影：根据输入x n-1，基于所述迭代软阈值收缩算法求解，输出第一近端投影v n；
    深度神经网络映射：输入第一近端投影v n，经过深度神经网络进行映射，输出u n，计算为
    Figure PCTCN2021120521-appb-100040
    其中
    Figure PCTCN2021120521-appb-100041
    表示第n次迭代时引入的深度神经网络，P n表示需要神经网络训练得到的网络参数；
    第二近端投影：根据输入u n，基于所述迭代软阈值收缩算法求解，输出第二近端投影w n；
    凸组合计算：输入第一近端投影v n和第二近端投影w n，输出第n次迭代的结果x n，计算步骤为x n＝α nv n+(1-α n)w n，其中α n为介于0和1之间的需要神经网络训练得到的凸组合参数。
  8. 一种图像处理系统，其特征在于，包括：
    训练数据集建立模块：获取训练图像，将所述训练图像进行处理，获得测量信号；将所述训练图像分解为字典与稀疏信号的乘积，将所述测量信号与所述稀疏信号作为训练样本对，建立训练数据集；
    可微分深度神经网络建立模块：在一范数约束下，建立图像处理的线性逆问题，利用迭代软阈值收缩算法求解所述线性逆问题，根据所述迭代软阈值收缩算法与深度神经网络的凸组合，建立凸组合的可微分深度神经网络；
    处理模块：利用所述训练数据集中的训练样本对，将所述测量信号作为所述可微分深度网络模型的输入，所述稀疏信号作为所述可微分深度网络模型的目标，采用反向梯 度传播算法优化所述可微分深度网络模型，得到重构稀疏信号，并使所述重构稀疏信号逼近于所述稀疏信号，得到最终图像，所述最终图像为所述字典与所述重构稀疏信号的乘积，且使所述最终图像逼近于所述训练图像。
  9. 一种图像处理装置，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时可用于执行权利要求1-7任一项所述的图像处理方法。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时可用于执行权利要求1-7任一项所述的图像处理方法。
  11. 一种计算机程序产品，其特征在于，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序可操作来使计算机执行如权利要求1-7任一项所述的图像处理方法。
  12. 一种芯片系统，包括处理器，所述处理器与存储器的耦合，所述存储器存储有程序指令，其特征在于，当所述存储器存储的程序指令被所述处理器执行时实现权利要求1-7任一项所述的图像处理方法。
  13. 一种计算机设备，包括：通信接口；其特征在于，还包括：
    存储器，用于储存实现如权利要求1-7任一项所述的图像处理方法的程序；
    处理器，用于加载并执行所述存储器存储的程序，以实现如权利要求1-7任一项所述的图像处理方法的各个步骤。
PCT/CN2021/120521 2021-08-17 2021-09-26 一种图像处理方法、系统、装置及存储介质 Ceased WO2023019685A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/472,248 US11995801B2 (en) 2021-08-17 2023-09-22 Image processing method, system, device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110940469.3 2021-08-17
CN202110940469.3A CN113674172B (zh) 2021-08-17 2021-08-17 一种图像处理方法、系统、装置及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/472,248 Continuation US11995801B2 (en) 2021-08-17 2023-09-22 Image processing method, system, device and storage medium

Publications (1)

Publication Number Publication Date
WO2023019685A1 true WO2023019685A1 (zh) 2023-02-23

Family

ID=78543155

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/120521 Ceased WO2023019685A1 (zh) 2021-08-17 2021-09-26 一种图像处理方法、系统、装置及存储介质

Country Status (3)

Country Link
US (1) US11995801B2 (zh)
CN (1) CN113674172B (zh)
WO (1) WO2023019685A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116243313A (zh) * 2023-03-08 2023-06-09 北京理工大学 一种基于距离分区的sar快速智能稀疏自聚焦技术
CN116309175A (zh) * 2023-03-24 2023-06-23 齐鲁工业大学(山东省科学院) 基于多正则化项的即插即用图像复原方法及系统
CN116484174A (zh) * 2023-03-16 2023-07-25 哈尔滨工业大学 一种基于可学习分数阶小波变换的信号稀疏分解与重建系统及方法
CN116630738A (zh) * 2023-04-25 2023-08-22 安徽工程大学 一种基于深度卷积稀疏表示重建网络的能谱ct成像方法
CN117093950A (zh) * 2023-08-21 2023-11-21 电子科技大学长三角研究院(衢州) 一种无参数融合lasso模型求解方法
CN117576217A (zh) * 2024-01-12 2024-02-20 电子科技大学 一种基于单实例图像重建的物体位姿估计方法
CN118608631A (zh) * 2024-08-08 2024-09-06 武汉卓目科技股份有限公司 基于深度压缩感知网络的图像压缩感知重建方法及装置
CN119337055A (zh) * 2024-09-02 2025-01-21 上海交通大学 基于混合去噪先验的信号重建方法、系统、介质及终端
CN119359819A (zh) * 2024-09-30 2025-01-24 山东大学 冷冻电子显微镜图像基准标记对齐中的投影参数标定方法

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114648657B (zh) * 2022-02-14 2024-11-15 南京邮电大学 一种基于稀疏约束重构胶囊网络的图像识别方法、装置及介质
CN114596228B (zh) * 2022-03-04 2025-02-11 南京邮电大学 基于深度3d卷积稀疏编码的多光谱图像去噪算法
CN114584150B (zh) * 2022-03-04 2024-06-21 南京邮电大学 一种利用深度神经网络实现的传感器网络信号补全方法
CN116405100B (zh) * 2023-05-29 2023-08-22 武汉能钠智能装备技术股份有限公司 一种基于先验知识的失真信号还原方法
CN117932227B (zh) * 2024-02-01 2025-09-12 南京邮电大学 一种信号支持度未知的带限图信号的自适应重构方法
GB2637757A (en) * 2024-02-02 2025-08-06 Sony Interactive Entertainment Inc Image transmission in gaming systems
CN117892068B (zh) * 2024-03-15 2024-06-07 江南大学 一种倒装芯片超声信号去噪方法及装置
CN118264627B (zh) * 2024-03-27 2025-11-18 浪潮通信信息系统有限公司 数据压缩传输方法及数据解压缩方法
WO2025222143A1 (en) * 2024-04-18 2025-10-23 Regents Of The University Of Minnesota Systems and methods for unsupervised learning for inverse problems using compressibility
CN118096556B (zh) * 2024-04-23 2024-08-27 华东交通大学 一种图像重构方法、系统、设备及介质
CN118731891B (zh) * 2024-08-30 2024-11-15 中国人民解放军国防科技大学 基于结构化短时稀疏表示的时频表征增强方法及装置
CN119295584A (zh) * 2024-09-26 2025-01-10 安徽大学 基于端到端深度学习的相位掩膜约束的图像重建方法、装置、设备及存储介质
CN119559388B (zh) * 2024-10-15 2025-08-01 南京航空航天大学 一种基于稀疏优化算法启发神经网络的图像分割方法
CN119478088A (zh) * 2024-10-30 2025-02-18 江苏优众微纳半导体科技有限公司 使用分路模式和图像重建联合优化基于超表面的彩色成像系统的方法
CN119180882B (zh) * 2024-11-22 2025-03-07 中国科学院自动化研究所 数理信息驱动的磁粒子成像梯度重建方法、模型训练方法及系统
CN119202908B (zh) * 2024-11-26 2025-04-29 西安交通大学 基于可学习稀疏正则字典学习的机械异常检测方法与系统
CN119295593B (zh) * 2024-12-11 2025-03-25 天津医科大学 一种基于分数生成模型的同轴相衬ct重建方法及系统
CN119832096B (zh) * 2024-12-12 2026-04-10 杭州师范大学 一种基于引入注意力监督深度展开网络的压缩感知重建方法及其系统
CN119304894B (zh) * 2024-12-18 2025-03-25 华中科技大学 一种基于跨物种优化的机械臂稀疏运动规划方法及系统
CN119810218B (zh) * 2025-01-10 2025-11-28 浙江工业大学 一种基于量化比特擦除的鲁棒性压缩感知方法
CN119902206B (zh) * 2025-01-21 2025-12-26 南京航空航天大学 一种基于回波模拟的分离稀疏sar成像方法、装置及介质
CN120263914B (zh) * 2025-06-03 2025-08-15 江西财经大学 稀疏光场子孔径图像加密与重建的隐私保护方法与系统
CN120804528B (zh) * 2025-09-12 2025-11-28 江南大学 基于自适应dun的倒装芯片超声信号去噪方法及系统
CN120855502B (zh) * 2025-09-25 2026-01-30 中能建建筑集团有限公司 沙漠光伏电站逆变器间协同抗扰控制方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100246920A1 (en) * 2009-03-31 2010-09-30 Iowa State University Research Foundation, Inc. Recursive sparse reconstruction
CN110678899A (zh) * 2017-06-02 2020-01-10 三菱电机株式会社 成像系统和图像处理方法
US20210158548A1 (en) * 2019-11-26 2021-05-27 Microsoft Technology Licensing, Llc Using machine learning to selectively overlay image content
CN112884851A (zh) * 2021-01-27 2021-06-01 武汉大学 一种展开迭代优化算法的深度压缩感知网络
CN113034414A (zh) * 2021-03-22 2021-06-25 上海交通大学 一种图像重构方法、系统、装置及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106780372B (zh) * 2016-11-30 2019-06-18 华南理工大学 一种基于广义树稀疏的权重核范数磁共振成像重建方法
WO2018106805A1 (en) * 2016-12-09 2018-06-14 William Marsh Rice University Signal recovery via deep convolutional networks
CN108289220B (zh) 2018-01-15 2020-11-27 深圳市奥拓电子股份有限公司 虚拟图像处理方法、图像处理系统及存储介质
CN109064406A (zh) * 2018-08-26 2018-12-21 东南大学 一种正则化参数自适应的稀疏表示图像重建方法
CN109683161B (zh) * 2018-12-20 2023-09-26 南京航空航天大学 一种基于深度admm网络的逆合成孔径雷达成像的方法
CN112907449B (zh) * 2021-02-22 2023-06-09 西南大学 一种基于深度卷积稀疏编码的图像超分辨率重建方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100246920A1 (en) * 2009-03-31 2010-09-30 Iowa State University Research Foundation, Inc. Recursive sparse reconstruction
CN110678899A (zh) * 2017-06-02 2020-01-10 三菱电机株式会社 成像系统和图像处理方法
US20210158548A1 (en) * 2019-11-26 2021-05-27 Microsoft Technology Licensing, Llc Using machine learning to selectively overlay image content
CN112884851A (zh) * 2021-01-27 2021-06-01 武汉大学 一种展开迭代优化算法的深度压缩感知网络
CN113034414A (zh) * 2021-03-22 2021-06-25 上海交通大学 一种图像重构方法、系统、装置及存储介质

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116243313A (zh) * 2023-03-08 2023-06-09 北京理工大学 一种基于距离分区的sar快速智能稀疏自聚焦技术
CN116484174A (zh) * 2023-03-16 2023-07-25 哈尔滨工业大学 一种基于可学习分数阶小波变换的信号稀疏分解与重建系统及方法
CN116309175A (zh) * 2023-03-24 2023-06-23 齐鲁工业大学(山东省科学院) 基于多正则化项的即插即用图像复原方法及系统
CN116630738A (zh) * 2023-04-25 2023-08-22 安徽工程大学 一种基于深度卷积稀疏表示重建网络的能谱ct成像方法
CN117093950A (zh) * 2023-08-21 2023-11-21 电子科技大学长三角研究院(衢州) 一种无参数融合lasso模型求解方法
CN117576217A (zh) * 2024-01-12 2024-02-20 电子科技大学 一种基于单实例图像重建的物体位姿估计方法
CN117576217B (zh) * 2024-01-12 2024-03-26 电子科技大学 一种基于单实例图像重建的物体位姿估计方法
CN118608631A (zh) * 2024-08-08 2024-09-06 武汉卓目科技股份有限公司 基于深度压缩感知网络的图像压缩感知重建方法及装置
CN119337055A (zh) * 2024-09-02 2025-01-21 上海交通大学 基于混合去噪先验的信号重建方法、系统、介质及终端
CN119359819A (zh) * 2024-09-30 2025-01-24 山东大学 冷冻电子显微镜图像基准标记对齐中的投影参数标定方法

Also Published As

Publication number Publication date
CN113674172A (zh) 2021-11-19
US11995801B2 (en) 2024-05-28
CN113674172B (zh) 2023-11-28
US20240029204A1 (en) 2024-01-25

Similar Documents

Publication Publication Date Title
CN113674172B (zh) 一种图像处理方法、系统、装置及存储介质
CN112884851B (zh) 基于展开迭代优化算法的深度压缩感知网络的构建方法
Shi et al. Image compressed sensing using convolutional neural network
CN109903228B (zh) 一种基于卷积神经网络的图像超分辨率重建方法
CN113177882B (zh) 一种基于扩散模型的单帧图像超分辨处理方法
CN117115042B (zh) 基于协方差优化扩散模型的图像处理方法、系统及装置
CN108171762A (zh) 一种深度学习的压缩感知同类图像快速重构系统与方法
CN108492249B (zh) 基于小卷积递归神经网络的单帧超分辨重建方法
CN111714124A (zh) 磁共振电影成像方法、装置、成像设备及存储介质
CN110111251B (zh) 一种结合深度监督自编码和感知迭代反投影的图像超分辨率重建方法
CN114140442A (zh) 一种基于频域和图像域退化感知的深度学习稀疏角度ct重建方法
CN113222812B (zh) 一种基于信息流加强深度展开网络的图像重建方法
CN109756740B (zh) 基于最优测量矩阵的半张量图像压缩方法和图像恢复方法
CN114245117B (zh) 多采样率复用重建网络方法、装置、设备及存储介质
Hu et al. A restoration network as an implicit prior
CN118898558A (zh) 基于一致性模型先验的图像处理方法、系统、介质及终端
CN118823505A (zh) 基于状态空间模型的连续高光谱重建方法及系统
CN119228929A (zh) 一种基于生成先验扩散的图像压缩感知重建方法
CN114494006A (zh) 图像重建模型的训练方法、装置、电子设备及存储介质
CN117745852A (zh) 一种基于Transformer与大核卷积的自然图像压缩感知方法
CN115270892B (zh) 一种信号重建方法、系统、装置及存储介质
CN110728732A (zh) 一种图像重建方法、装置、设备及介质
CN114359419A (zh) 一种基于注意力机制的图像压缩感知重建方法
CN106447610B (zh) 图像重建方法及装置
CN112150566A (zh) 一种基于特征融合的密集残差网络图像压缩感知重建方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21953921

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21953921

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21953921

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24-09-2025)

122 Ep: pct application non-entry in european phase

Ref document number: 21953921

Country of ref document: EP

Kind code of ref document: A1