CN116862787A - Image restoration method, system, equipment and storage medium based on tensor singular value decomposition - Google Patents

Image restoration method, system, equipment and storage medium based on tensor singular value decomposition Download PDF

Info

Publication number
CN116862787A
CN116862787A CN202310725633.8A CN202310725633A CN116862787A CN 116862787 A CN116862787 A CN 116862787A CN 202310725633 A CN202310725633 A CN 202310725633A CN 116862787 A CN116862787 A CN 116862787A
Authority
CN
China
Prior art keywords
image
tensor
singular value
value decomposition
image restoration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310725633.8A
Other languages
Chinese (zh)
Inventor
马淑丽
杜慧茜
王宏艳
储飞黄
方胜良
胡豪杰
范有臣
李志飞
董尧尧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Original Assignee
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peoples Liberation Army Strategic Support Force Aerospace Engineering University filed Critical Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority to CN202310725633.8A priority Critical patent/CN116862787A/en
Publication of CN116862787A publication Critical patent/CN116862787A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00Road transport of goods or passengers
    • Y02T10/10Internal combustion engine [ICE] based vehicles
    • Y02T10/40Engine management systems

Abstract

The invention provides an image restoration method based on tensor singular value decomposition. In order to mine potential tensor low tube (Tubal) rank priori information of the image data, a data rearrangement scheme is designed, so that image data elements are rearranged into a data structure suitable for tensor singular value decomposition, and the scheme can ensure that rearranged data has tensor low tube rank. And then, constructing an image restoration optimization model based on tensor singular value decomposition by restraining tensor low tube rank property and combining image local smoothness. And finally, developing a model optimization solving algorithm based on the ADMM method. Experimental results show that the method can effectively repair various typical image damage forms, and is superior to a low-rank matrix filling method and an image repair method based on Tucker tensor decomposition in visual and numerical indexes.

Description

Image restoration method, system, equipment and storage medium based on tensor singular value decomposition
Technical Field
The invention relates to the technical field of image processing, in particular to an image restoration method, an image restoration system, image restoration equipment and a storage medium based on tensor singular value decomposition.
Background
The image restoration technology relates to the technology for carrying out optimized restoration on data under the conditions of local damage, pulse interference, random noise, malicious matting, text coverage, advertisement coverage and the like of image data. The conventional image restoration technology generally performs optimization restoration by using methods such as low rank property of a transformation domain, sparsity of the transformation domain, optimized filtering or interpolation of a matrix according to matrix structural features of an image. The method for repairing the image by directly utilizing the low rank property of the matrix has poor repairing effect under the pollution conditions of image blocks, parallel lines, stripes and the like.
In recent years, a variety of new tensor decomposition frameworks have emerged in the mathematical arts for processing high-dimensional data. The novel tensor singular value decomposition (t-SVD) framework breaks through the traditional CANDECOMP/PARAFAC (CP) and Tucker tensor decomposition ideas, and further improves the computational complexity and the low rank approximation effectiveness of tensors. In video restoration application, the restoration accuracy of a low-tube (tubal) rank method defined by a tensor singular value decomposition framework is obviously improved compared with a method directly utilizing matrix low rank and a method utilizing Tucker low rank. However, RGB image data cannot be directly image-processed using a tensor singular value decomposition framework due to dimensional limitations. When the conventional CP or Tucker decomposition is applied to RGB image restoration, the RGB image is generally directly used as a third-order tensor, and potential tensor low-rank prior information of the image is not deeply mined, so that the restoration effect is often poor.
Disclosure of Invention
Aiming at the problems, in order to further improve the image restoration precision, the invention provides an image restoration method based on tensor singular value decomposition. And then, performing image restoration modeling and optimization algorithm solving by using tensor singular value decomposition.
The technical solution for realizing the purpose of the invention is as follows:
an image restoration method based on tensor singular value decomposition is characterized by comprising the following steps:
step 1: inputting a polluted RGB image to obtain an image size m multiplied by n multiplied by 3 and a polluted area pixel position set omega;
step 2: rearranging the image data elements obtained in the step 1 by adopting an image rearranging scheme to obtain new third-order tensor data;
step 3: constructing an image restoration model based on tensor singular value decomposition low-tube rank constraint based on new third-order tensor data;
step 4: setting the maximum iteration times, the iteration error tolerance and experience parameters, initializing variables, and solving the image restoration model based on an iterative optimization algorithm adopting an alternate direction multiplier method;
step 5: and outputting the repaired image.
Further, the specific operation steps of the step 2 include:
step 21: copying edge pixel points of the polluted RGB image to enable the length m and the width n of the image to meet the geometric multiplication relation of 2;
step 22: extracting four pixel points of upper left, lower left, upper right and lower right at every other row and every other column, and dividing an image into four small images;
step 23: repeating step 22 for each small image until the last obtained number n of sub-images 3 On the same order of magnitude as the length and width of the subgraph, with 10 being the base;
step 24: will n 3 The sub-graphs are arranged into a size n 1 ×n 2 ×n 3 Is a balanced third-order tensor.
Further, the image restoration model constructed in the step 3 is as follows:
wherein phi is an image rearrangement operator, lambda>0 and beta>0 is the balance parameter, and the balance parameter is the balance parameter, I.I TNN Representing tensor kernel norms, τ, defined under the t-SVD decomposition framework Ω (X) represents:
further, the specific operation steps of the step 4 include:
step 41: introducing an auxiliary variable z=dx, b=Φx, converting the image restoration model into an augmented lagrangian function minimization problem:
wherein ρ is 1 >0 and ρ 2 >0 is the balance coefficient to be introduced, I.I F Representing the Frobenius norm, Λ and L representing the amount of residual introduced;
step 42: when the number of iterations s is satisfied<s max And eta<η max The distributed iteration solves the problem (2), and at the s-th iteration, the following sub-problems are solved in turn:
and (3) calculating:
Λ s =Λ s-1 +ΦX s -Z s
L s =L s-1 +DX s -Z s
step 43: the iteration number s=s+1, and returns to step 42 until the iteration is ended, and a restored image is output.
The image restoration system based on tensor singular value decomposition is characterized by comprising an image input module, an image rearrangement module, an image restoration module and an image output module;
the image input module is used for inputting pollution RGB images to be repaired;
the image rearrangement module is used for rearranging the input image data elements by adopting an image rearrangement scheme to obtain new third-order tensor data;
the image restoration module is used for establishing an image restoration model according to the third-order tensor data obtained by the image rearrangement module, solving the image restoration model by adopting an iterative optimization algorithm of an alternate direction multiplier method, and restoring the image;
the image output module is used for outputting the repaired image.
An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.
A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-4.
A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-4.
The beneficial effects are that:
firstly, the invention provides an interlaced data rearrangement scheme, so that under the condition of not changing the total number of the image data element values, the image data is converted into a data structure suitable for tensor singular value decomposition, and the new data structure has low tube rank, thereby providing a new idea of using a tensor singular value decomposition frame for the image data restoration technology;
secondly, the invention provides a novel image restoration model based on tensor singular value decomposition and provides an algorithm for solving the model. The method solves the problem that the repair effect is often poor under the pollution conditions of image blocks, stripes and the like when the matrix low rank property is directly utilized for image repair to a certain extent. And finally, comparing and analyzing the experiment, and under the condition of various image damage forms, the provided repairing method is more convenient than the traditional tensor decomposition method and the direct matrix low-rank repairing method, and can further improve the visual precision and numerical index of image repairing.
Drawings
FIGS. 1 a-1 f are schematic diagrams of image contamination;
FIG. 2 is a schematic diagram of a process for extracting four types of data elements from an interlaced array;
FIG. 3 is a diagram of a Lena image divided into four small Lena images;
FIG. 4 is a plot of pixel values from large to small for four small Lena images;
FIG. 5 is a schematic diagram of an image rearrangement scheme;
FIG. 6 is a graph of a balance tensor with low tube rank verification obtained after a rearrangement scheme of an image;
FIG. 7 is a graph comparing the repair effect of three methods.
Detailed Description
In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Aiming at the problem of image restoration of low-dimensional image data under the pollution conditions of stripes, blocks, random pulses, random character coverage and the like, the invention provides an image restoration method based on tensor singular value decomposition. First, a data rearrangement scheme is proposed, aiming at mining potential low-rank prior information of images. Then, image restoration modeling and optimization algorithm design based on tensor singular value decomposition are carried out. And finally, executing the program to recover the high-precision image. Referring to fig. 1 a-1 f, fig. 1a, 1c, 1e are original diagrams, and fig. 1b, 1d, 1f are diagrams of impulse pollution, stripe pollution, and block pollution, respectively.
The invention provides an image restoration method based on tensor singular value decomposition, which comprises the following steps:
step 1: inputting a pollution image, acquiring the image size of m multiplied by n multiplied by 3, and acquiring a pollution pixel point position set omega;
step 2: inputting the maximum iteration number s max Iterative error tolerance η tol Empirical parameters lambda, beta, p 1 、ρ 2 Setting initial iteration number s=1, initializing X 0 Auxiliary variable Z 0 And B 0 Residual variable lambda 0 And L 0 Is an all zero tensor;
step 3: considering that the tube rank approximation of the balance tensor is more effective, converting an unbalanced image into balanced third-order tensor data by using a rearrangement scheme, namely rearranging image data elements to obtain a new third-order data structure;
specifically, the rearrangement scheme is:
step 31: the edge pixels are duplicated so that the image length m and width n satisfy a geometric multiplication relationship of 2.
Step 32: four-division image operation: extracting four types of pixel points of upper left, lower left, upper right and lower right at intervals of every other row and every other column, wherein the four types of pixel points of upper left, lower left, upper right and lower right respectively correspond to elements in the blue, green, yellow and red square grids as shown in fig. 2;
four sub-images can be obtained from one image after four types of data elements are extracted by interlacing and separating columns, and the Lena image can be divided into 4 small Lena images as shown in fig. 3. Because of the local similarity of the images, the four small images obtained are similar, and the four curves of the four small Lena images arranged from large to small as shown in fig. 4 have almost overlapped into one curve, which is also the reason for the low tube rank of the rearranged "balance" tensor obtained in the subsequent step.
Step 33: then, each obtained sub-graph is continuously subjected to four types of pixel points extracted from every other line and every other column, namely, four sub-graphs can be obtained from each sub-graph, and the like, through multi-level four-sub-graph operation, the number n of sub-graphs of the last stage is obtained 3 Length n to last level subgraph 1 And width n 2 When the same order of magnitude is 10-based, then the continued quarter image operation ends.
All the sub-pictures of the last stage are arranged according to the sequence and are coded into n 1 ×n 2 ×n 3 Is a third order tensor of (c). The size n of this tensor 1 、n 2 、n 3 Is more balanced than the original image tensor by length m, width n and depth 3 (because n 1 、n 2 And n 3 On the same order of magnitude, while m, n and 3 orders of magnitude differ significantly). This "balanced" third order tensor is more suitable for t-SVD decomposition than the "unbalanced" mxn x 3 third order tensor. As shown in fig. 5, a 256×256×3Lena image of "unbalanced" size may be operated on by a three-level quartering image to obtain a 64×64×48-sized third-order tensor of "balanced". At this time, the operator Φ is defined to represent the procedure of steps 31 to 33.
To illustrate that the resulting balanced third-order tensor has tensor low tube rank, a singular value curve is plotted as shown in FIG. 6, where the ordinate delta j The expression is as follows,
let X denote an image of size mxn×3, then T is obtained according to the following two equations:
Q=fft(ΦX,[],3)
[U,T(:,:,j),V]=SVD(Q(:,:,j)),j=1,2,…,n 3
i.e. the size is first of all n 1 ×n 2 ×n 3 The "balanced" third-order tensor Φx of Q is 1D-FFT along the third dimension to obtain tensor Q, and then singular value decomposition is performed on each slice Q (: j) of Q to obtain a diagonal matrix T (: j), where j=1, 2, …, n 3 Finally, a tensor T is formed. The abscissa in fig. 6 represents i=1, 2, …, min (n 1 ,n 2 ). In drawing fig. 6, the image X is selected as a Lena image of 256×256×3 size.
According to the tube rank definition under the tensor t-SVD decomposition framework, the curve of FIG. 6 verifies that the "balanced" third-order tensor obtained after rearrangement has low tube rank.
Step 4: constructing an image restoration model based on tensor singular value decomposition low-tube rank constraint;
combining low-tube rank and total variation domain sparsity, introducing TV (total variation) constraint terms, and obtaining a repair model:
wherein lambda is>0 and beta>0 is the balance parameter τ Ω (X) represents
||·|| TNN Representing tensor kernel norms (tensor nuclear norm, TNN) defined under the t-SVD decomposition framework, |·|| TNN The definition of (c) is calculated as follows,
wherein the method comprises the steps ofRepresenting +.>A new tensor obtained by a 1-dimensional Fourier transform (1D-FFT), i.e.>||·|| * Representing nuclear norms>Representing tensor +.>Block diagonalization, i.e.
Wherein the method comprises the steps ofIndicating fixation->A matrix of slices obtained at the kth position of the third dimension of (a), i.e
Step 5: executing an iterative optimization algorithm based on an alternate direction multiplier method, and solving a model (1);
the auxiliary variables z=dx and b=Φx are introduced, where l X TV =||DX 1 D is a difference operator, i.e. dx= (D x X,D y X), wherein
||·|| 1 Representing the sum of absolute values of all elements, i.e.
Converting model (1) to an augmented lagrangian minimization problem:
wherein ρ is 1 >0 and ρ 2 >0 is the balance coefficient to be introduced, I.I F The Frobenius norm is represented, and Λ and L represent the introduced residual amount.
When s is less than or equal to s max Eta is less than or equal to eta tol The distributed iteration solves the problem (2), and the following sub-problems are solved in sequence in the s-th iteration:
and (3) calculating: Λ type s =Λ s-1 +ΦX s -Z s ,L s =L s-1 +DX s -Z s
Specifically, solving problem (3) to obtain X s
Wherein F represents a two-dimensional Fourier transform (2D-FFT), F -1 Inverse transform of F, phi -1 The inverse operation of phi is indicated,D -1 representing an inverse differential operator, ++>
Wherein the method comprises the steps of
Solving problem (4) to obtain Z s The shrinkage threshold algorithm can be utilized to solve in one step,
where abs (.) represents absolute values and.×represents matrix dot product.
Solving problem (5) to obtain B s The following problem can be solved by using a shrinkage singular value thresholding algorithm to obtain B s
The above solution algorithm is shown in table 1.
TABLE 1 solving algorithm based on ADMM method
Step 6: outputting the repaired image: and displaying and storing.
Each column in fig. 7 is a pollution map, a matrix low rank method repair map, a tensor Tucker decomposition method repair map, and an inventive method repair map, respectively. As can be seen from fig. 7, in the forms of salt and pepper pulse pollution, blocky and parallel line stripe pollution, the proposed repairing method can further improve the image repairing precision compared with the traditional tensor decomposition method and the direct matrix low rank repairing method. The method solves the problem that the repair effect is often poor under the pollution conditions of image blocks, stripes and the like when the matrix low rank property is directly utilized for image repair to a certain extent.
What is not described in detail in this specification is prior art known to those skilled in the art. Although the present invention has been described with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described, or equivalents may be substituted for elements thereof, and any modifications, equivalents, improvements and changes may be made without departing from the spirit and principles of the present invention.

Claims (8)

1. An image restoration method based on tensor singular value decomposition is characterized by comprising the following steps:
step 1: inputting a polluted RGB image to obtain an image size m multiplied by n multiplied by 3 and a polluted area pixel position set omega;
step 2: rearranging the image data elements obtained in the step 1 by adopting an image rearranging scheme to obtain new third-order tensor data;
step 3: constructing an image restoration model based on tensor singular value decomposition low-tube rank constraint based on new third-order tensor data;
step 4: setting the maximum iteration times, the iteration error tolerance and experience parameters, initializing variables, and solving the image restoration model based on an iterative optimization algorithm adopting an alternate direction multiplier method;
step 5: and outputting the repaired image.
2. The method for image restoration based on tensor singular value decomposition according to claim 1, wherein the specific operation steps of step 2 include:
step 21: copying edge pixel points of the polluted RGB image to enable the length m and the width n of the image to meet the geometric multiplication relation of 2;
step 22: extracting four pixel points of upper left, lower left, upper right and lower right at every other row and every other column, and dividing an image into four small images;
step 23: repeating step 22 for each small image until the last obtained number n of sub-images 3 On the same order of magnitude as the length and width of the subgraph, with 10 being the base;
step 24: will n 3 The sub-graphs are arranged into a size n 1 ×n 2 ×n 3 Is a balanced third-order tensor.
3. The image restoration method based on tensor singular value decomposition according to claim 1, wherein the image restoration model constructed in the step 3 is:
wherein phi is an image rearrangement operator, lambda>0 and beta>0 is the balance parameter, and the balance parameter is the balance parameter, I.I. | TNN Representing tensor kernel norms, τ, defined under the t-SVD decomposition framework Ω (X) represents:
4. a method of image restoration based on tensor singular value decomposition according to claim 3, wherein the specific operation steps of step 4 include:
step 41: introducing an auxiliary variable z=dx, b=Φx, converting the image restoration model into an augmented lagrangian function minimization problem:
wherein ρ is 1 >0 and ρ 2 >0 is the balance coefficient to be introduced, I.I. | F Representing the Frobenius norm, Λ and L representing the amount of residual introduced;
step 42: when the number of iterations s is satisfied<s max And eta<η max The distributed iteration solves the problem (2), and at the s-th iteration, the following sub-problems are solved in turn:
and (3) calculating:
Λ s =Λ s-1 +ΦX s -Z s
L s =L s-1 +DX s -Z s
step 43: the iteration number s=s+1, and returns to step 42 until the iteration is ended, and a restored image is output.
5. The image restoration system based on tensor singular value decomposition is characterized by comprising an image input module, an image rearrangement module, an image restoration module and an image output module;
the image input module is used for inputting pollution RGB images to be repaired;
the image rearrangement module is used for rearranging the input image data elements by adopting an image rearrangement scheme to obtain new third-order tensor data;
the image restoration module is used for establishing an image restoration model according to the third-order tensor data obtained by the image rearrangement module, solving the image restoration model by adopting an iterative optimization algorithm of an alternate direction multiplier method, and restoring the image;
the image output module is used for outputting the repaired image.
6. An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.
7. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-4.
8. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-4.
CN202310725633.8A 2023-06-19 2023-06-19 Image restoration method, system, equipment and storage medium based on tensor singular value decomposition Pending CN116862787A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310725633.8A CN116862787A (en) 2023-06-19 2023-06-19 Image restoration method, system, equipment and storage medium based on tensor singular value decomposition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310725633.8A CN116862787A (en) 2023-06-19 2023-06-19 Image restoration method, system, equipment and storage medium based on tensor singular value decomposition

Publications (1)

Publication Number Publication Date
CN116862787A true CN116862787A (en) 2023-10-10

Family

ID=88229547

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310725633.8A Pending CN116862787A (en) 2023-06-19 2023-06-19 Image restoration method, system, equipment and storage medium based on tensor singular value decomposition

Country Status (1)

Country Link
CN (1) CN116862787A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117252788A (en) * 2023-11-20 2023-12-19 华南理工大学 Image restoration method, device and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117252788A (en) * 2023-11-20 2023-12-19 华南理工大学 Image restoration method, device and storage medium
CN117252788B (en) * 2023-11-20 2024-03-05 华南理工大学 Image restoration method, device and storage medium

Similar Documents

Publication Publication Date Title
Liu et al. Learning affinity via spatial propagation networks
Elad et al. Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization
Jia et al. Non-local robust quaternion matrix completion for large-scale color image and video inpainting
Ding et al. Perceptually aware image inpainting
CN109191404B (en) Hyperspectral image restoration method based on E-3DTV regularization
CN108510013B (en) Background modeling method for improving robust tensor principal component analysis based on low-rank core matrix
Wen et al. FRIST—Flipping and rotation invariant sparsifying transform learning and applications
CN110139046B (en) Tensor-based video frame synthesis method
Xu et al. Fast tensor nuclear norm for structured low-rank visual inpainting
CN116862787A (en) Image restoration method, system, equipment and storage medium based on tensor singular value decomposition
CN112465713A (en) Tensor low-rank model non-smooth three-dimensional image completion method based on manifold optimization
CN116757930A (en) Remote sensing image super-resolution method, system and medium based on residual separation attention mechanism
CN109741258B (en) Image super-resolution method based on reconstruction
Lucet New sequential exact Euclidean distance transform algorithms based on convex analysis
CN113888491A (en) Multilevel hyperspectral image progressive and hyper-resolution method and system based on non-local features
Kuznetsov et al. Remote sensing image inpainting with generative adversarial networks
Wang et al. Hyperspectral unmixing via plug-and-play priors
CN103413351A (en) Three-dimensional face rapid-rebuilding method based on compressed sensing theory
Fang et al. Learning explicit smoothing kernels for joint image filtering
CN116310452B (en) Multi-view clustering method and system
Danelakis et al. Blind image deconvolution using a banded matrix method
Takahashi et al. Rank minimization approach to image inpainting using null space based alternating optimization
CN116797456A (en) Image super-resolution reconstruction method, system, device and storage medium
Hesabi et al. A modified patch propagation-based image inpainting using patch sparsity
CN114494021A (en) Image reconstruction method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination