CN112241938A - Image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation - Google Patents
Image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation Download PDFInfo
- Publication number
- CN112241938A CN112241938A CN202010846940.8A CN202010846940A CN112241938A CN 112241938 A CN112241938 A CN 112241938A CN 202010846940 A CN202010846940 A CN 202010846940A CN 112241938 A CN112241938 A CN 112241938A
- Authority
- CN
- China
- Prior art keywords
- image
- decomposition
- hank
- algorithm
- tensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000000354 decomposition reaction Methods 0.000 title claims abstract description 24
- 230000009466 transformation Effects 0.000 title claims abstract description 11
- 230000000007 visual effect Effects 0.000 claims abstract description 11
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 4
- OVBPIULPVIDEAO-LBPRGKRZSA-N folic acid Chemical compound C=1N=C2NC(N)=NC(=O)C2=NC=1CNC1=CC=C(C(=O)N[C@@H](CCC(O)=O)C(O)=O)C=C1 OVBPIULPVIDEAO-LBPRGKRZSA-N 0.000 claims description 4
- 230000000903 blocking effect Effects 0.000 claims description 3
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 claims description 2
- 230000000295 complement effect Effects 0.000 claims description 2
- 150000001875 compounds Chemical class 0.000 claims description 2
- 230000001419 dependent effect Effects 0.000 claims description 2
- 230000009977 dual effect Effects 0.000 claims description 2
- 238000009499 grossing Methods 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 238000011479 proximal gradient method Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 5
- 230000008901 benefit Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000011049 filling Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation comprises the following steps: 1) inputting image to be restoredDetermining an area to be repaired of an image; 2) constructing a high-order tensor Hankelization and discrete total variation model; 3) combining the step 2) to construct an image restoration model of smooth Tak decomposition and high-order tensor Hank transformation, restoring the color image, and finally reconstructing and outputting a high-quality visual data imageThe invention has the advantages that: the efficiency of image processing and the accuracy of image restoration are both considered.
Description
Technical Field
The invention relates to the field of image processing, in particular to an image restoration method.
Background
With the rapid development of modern network technology, computer communication and sampling technology, the data to be analyzed mostly has a very complex structure. Generally, the image data acquisition process is influenced by various external factors, which results in poor visual quality, such as damage to hardware equipment, light and interference of electromagnetic waves. In this case, too, it may be impossible to directly retrieve the relevant image data due to device or time limitations. Therefore, repairing various blurred, low-resolution, partial-pixel-loss and other images to obtain high-quality visual data is a research content with practical application value.
Image inpainting is a typical image processing ill-posed problem that can be expressed as a missing value estimation problem. The core problem of missing value estimation is how to establish the relationship between known elements and unknown elements, and adding other prior information can effectively solve the image restoration problem, such as local smooth prior, non-local self-similar prior, sparse prior, low rank prior, sparse gradient prior, and the like. In recent years, many scholars have proposed different image restoration algorithms, which are mainly classified into three categories: 1) image restoration based on a variational differential equation; 2) image inpainting based on texture synthesis; 3) and (3) a mixing method. Bertalmia et al first proposed a differential equation-based image restoration method that restores an image by diffusing information of an undamaged region into the interior of a region to be restored by diffusing the boundaries of the region to be restored in different directions. This method has a good repairing effect only on the damages of a small area in the image. Chan et al propose a Total Variation (TV) algorithm, which has the greatest advantage of effectively overcoming the problem of linear filtering that smoothes image edges while suppressing noise, but the greatest drawback of the TV algorithm is that the "discontinuity" principle in human vision cannot be satisfied. The Curvature-Drive Diffusion (CDD) algorithm is an improved algorithm for TV algorithm, and aims to solve the problem of visual discontinuity in TV algorithm. Criminisi et al propose a sample block-based image restoration algorithm, which calculates the priority of a block to be restored by using boundary information of the region to be restored, and then searches for a sample block with the maximum similarity to the block to be restored in the undamaged region of the image to perform filling and restoration. The algorithm has a good repairing effect on a large-area damaged area, but the efficiency of the algorithm is reduced due to the fact that the repairing time is too long.
With the recent development of deep neural network architectures, deep learning methods have great significance in computer vision tasks such as object detection, image classification and image noise reduction. However, the deep learning based method requires a large number of labeled samples, which are difficult to obtain and consume a large amount of computation, so that research and application of the conventional method are still necessary and there is a great room for improvement.
Disclosure of Invention
The present invention provides an image restoration method based on smooth tack decomposition and high-order tensor hank-based method to overcome the above problems in the prior art. ,
in order to solve the visual processing problem of image data distortion, the invention expands the Hank structuring technology into the high-order tensor visual data, fully considers the essential attribute of the image and introduces the Discrete Total Variation (TV)d) The regularization term factor integrates it into a uniform objective function.
The technical scheme adopted by the invention for solving the technical problems comprises the following steps:
the image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation comprises the following steps:
step 1) inputting an image to be restoredDetermining an image to-be-repaired area and performing blocking operation on the image to-be-repaired area, wherein pixels in the image are divided into known points and unknown points, the known points are points with pixels not being 0 in the image, the unknown points are points with pixels being 0 in the image, and all the unknown points in the image form a set omega;
step 2), constructing a high-order tensor Hankelization and discrete total variation model;
step 3) image restoration constructed by combining the step 2)The model is used for repairing the color image and finally reconstructing and outputting a high-quality visual data image
The invention has the following beneficial effects: the Hank structuring technology is expanded to be applied to the tensor field. Considering that the low rank and the smoothness of the matrix also exist in the tensor, firstly, data are embedded into a high-dimensional tensor, and tensor hank is structured through multi-dimensional linear replication and multi-dimensional folding linear operation; and secondly, considering data smoothness, introducing a discrete total variation factor to optimize a model, and finally, better finding the optimal rank through a low-rank incremental algorithm, wherein the algorithm has good convergence and more accurately restores a natural image.
The invention has the advantages that: the efficiency of image processing and the accuracy of image restoration are both considered.
Drawings
FIG. 1 is a schematic view of an area to be repaired;
FIG. 2 is a natural image with a pixel loss rate of 90%;
FIG. 3 is a natural image restored using the present invention;
fig. 4 is a flow chart of a method of the present invention.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings.
The image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation comprises the following steps:
step 1) inputting an image to be restoredDetermining an image to-be-repaired area and performing blocking operation on the image to-be-repaired area, wherein pixels in the image are divided into known points and unknown points, the known points are points with pixels not being 0 in the image, the unknown points are points with pixels being 0 in the image, and all the unknown points in the image form a set omega;
step 2), constructing a high-order tensor Hankelization and discrete total variation model;
step 3) repairing the color image by combining the image repairing model constructed in the step 2), and finally reconstructing and outputting a high-quality visual data image
The treatment process of the step 2) is as follows:
(2-1) the high-order hank structured image inpainting model is defined as follows:
in the formula (I), the compound is shown in the specification,which represents the input of the image to be repaired,a post-repair image is represented as,represents the Frobenius norm;where 1 represents an observable pixel and 0 represents a missing pixel;is defined as
Wherein fold(I,τ):Through the fold(I,τ)The input N-order tensor can be constructed into a 2N-order tensor, and the operation can be regarded as multi-dimensional linear replication and multi-dimensional foldingOperating; whereinThe matrix is a copy matrix, namely a matrix comprising a plurality of identity matrixes, and the specific form is as follows:
(2-2) the discrete total variation model is defined as follows:
x represents a two-dimensional image and v represents a gradient;l. representing bilinear interpolation operations, particularly in a grid(n1,n2) The interpolation of (1); a*Representing an accompanying operation; the definition of the discrete operator D is (Dx)1[n1,n2]=x[n1+1,n2]-x[n1,n2],(Dx)2[n1,n2]=x[n1,n2+1]-x[n1,n2];
Let l of v be1,1,2Norm represents three vector componentsL of v. a1,1,2Norm sum, i.e.The discrete total variation model can therefore be redefined as:
and fully utilizing low-rank complementary information and potential smoothing characteristics.
The treatment process of the step 3) is as follows:
(3-1) constructing an image restoration model based on smooth Tak decomposition and high-order tensor Hank transformation, which is defined as follows
In the formula, λ represents a balance parameter,represents a decomposition factor U(n)The (J, R) th entry of (A), and
(3-2) solving equation (6) is dependent on variablesLeast Squares (ALS) optimization can be used, algorithm 1 describes the main process of ALS,
algorithm 1:
3.2.2 when N is 1, …,2N
3.2.4U(n)And algorithm 2.
(3-3) Algorithm 1 describes a traditional ALS-based Take factorization whose computational and storage bottleneck is the update factor matrix U(n)To solve the sub-problem U(n)We update it as follows
Due to the complexity of equation (7), an Alternating Proximal Gradient Method (APG) algorithm is used to solve the above equation,
let G (v) | | v | | non-phosphor1,1,2,C=-L*Thus can obtain
Thus, it can translate into a dual problem:
final subproblem U(n)As described in algorithm 2;
and 2, algorithm:
3.3.1τ,μ>0;θ∈[0,1];k=0
3.3.6k=k+1
Wherein proxσAnd proxτIs mapped to
(3-4) in addition, the tach-based method can obtain a satisfactory effect by minimizing the rank of the tach, but it is difficult to set an appropriate rank (R)1,…,R2N). In our method, we minimize the following objective function, and this process is to obtain an approximate minimum of a sufficiently low rank
E(1)≥E(2)≥…≥E(Rn′-1)≥ε≥E(Rn′) (13)
finally, an image restoration model based on smooth Tak decomposition and high-order tensor Hank transformation is optimized by using low-rank increment, and the method is described in algorithm 3
Algorithm 3:
3.4.3n′←n,Rn′←Rn
The embodiments described in this specification are merely illustrative of implementations of the inventive concept and the scope of the present invention should not be considered limited to the specific forms set forth in the examples but rather by equivalents thereof as may occur to those skilled in the art upon consideration of the inventive concept.
Claims (3)
1. An image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation comprises the following steps:
step 1), inputting an image to be repairedDetermining an image to-be-repaired area and carrying out blocking operation on the image to-be-repaired area, wherein pixels in the image are divided into known points and unknown points, the known points are points with pixels not being 0 in the image, and the unknown points are points with pixels being 0 in the image; all unknown points in the image form a set omega;
step 2), constructing a high-order tensor Hankelization and discrete total variation model;
2. The image restoration method according to claim 1, wherein the image restoration method based on smooth Tak decomposition and high-order tensor Hank decomposition comprises: the step 2) specifically comprises the following steps:
(2-1) the high-order hank structured image inpainting model is defined as follows:
in the formula (I), the compound is shown in the specification,which represents the input of the image to be repaired,a post-repair image is represented as,represents the Frobenius norm;where 1 represents an observable pixel and 0 represents a missing pixel;is defined as
Wherein fold(I,τ):Through the fold(I,τ)The input N-order tensor can be constructed into a 2N-order tensor, and the operation can be regarded as a multi-dimensional linear copying operation and a multi-dimensional folding operation; whereinIs a copyA matrix, that is, a matrix including a plurality of identity matrices, is specifically formed by:
(2-2) the discrete total variation model is defined as follows:
x represents a two-dimensional image and v represents a gradient;representing bilinear interpolation operations, particularly in grids(n1,n2) The interpolation of (1); a*Representing an accompanying operation; the definition of the discrete operator D is (Dx)1[n1,n2]=x[n1+1,n2]-x[n1,n2],(Dx)2[n1,n2]=x[n1,n2+1]-x[n1,n2];
Let l of v be1,1,2Norm represents three vector componentsL of1,1,2Norm sum, i.e.The discrete total variation model can therefore be redefined as:
and fully utilizing low-rank complementary information and potential smoothing characteristics.
3. The image restoration method according to claim 1, wherein the image restoration method based on smooth Tak decomposition and high-order tensor Hank decomposition comprises: the step 3) specifically comprises the following steps:
(3-1) constructing an image restoration model based on smooth Tak decomposition and high-order tensor Hank transformation, which is defined as follows
In the formula, λ represents a balance parameter,represents a decomposition factor U(n)The (J, R) th entry of (A), and
(3-2) solving equation (6) is dependent on variablesLeast Squares (ALS) optimization can be used, algorithm 1 describes the main process of ALS,
algorithm 1:
3.2.2 when N is 1, …,2N
3.2.4 U(n)And algorithm 2.
(3-3) Algorithm 1 describes a traditional ALS-based Take factorization whose computational and storage bottleneck is the update factor matrix U(n)To solve the sub-problem U(n)We update it as follows
Due to the complexity of equation (7), an Alternating Proximal Gradient Method (APG) algorithm is used to solve the above equation,
let G (v) | | v | | non-phosphor1,1,2,C=-L*Thus can obtain
Thus, it can translate into a dual problem:
final subproblem U(n)As described in algorithm 2;
and 2, algorithm:
3.3.1τ,μ>0;θ∈[0,1];k=0
3.3.6 k=k+1
Wherein proxσAnd proxτIs mapped to
(3-4) in addition, the tach-based method can obtain a satisfactory effect by minimizing the rank of the tach, but it is difficult to set an appropriate rank (R)1,…,R2N). In our method, we minimize the following objective function, and this process is to obtain an approximate minimum of a sufficiently low rank
Wherein ε represents an error threshold; order toRn′The optimal rank of the constraint is represented,
E(1)≥E(2)≥…≥E(Rn′-1)≥ε≥E(Rn′) (13)
finally, an image restoration model based on smooth Tak decomposition and high-order tensor Hank transformation is optimized by using low-rank increment, and the method is described in algorithm 3
Algorithm 3:
3.4.3 n′←n,Rn′←Rn
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010846940.8A CN112241938B (en) | 2020-08-21 | 2020-08-21 | Image restoration method based on smooth Tak decomposition and high-order tensor Hakking |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010846940.8A CN112241938B (en) | 2020-08-21 | 2020-08-21 | Image restoration method based on smooth Tak decomposition and high-order tensor Hakking |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112241938A true CN112241938A (en) | 2021-01-19 |
CN112241938B CN112241938B (en) | 2024-02-13 |
Family
ID=74170478
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010846940.8A Active CN112241938B (en) | 2020-08-21 | 2020-08-21 | Image restoration method based on smooth Tak decomposition and high-order tensor Hakking |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112241938B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114119426A (en) * | 2022-01-26 | 2022-03-01 | 之江实验室 | Image reconstruction method and device by non-local low-rank conversion domain and full-connection tensor decomposition |
CN114202107A (en) * | 2021-11-18 | 2022-03-18 | 华南理工大学 | Ultra-short-term wind speed prediction method and device |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109886884A (en) * | 2019-01-21 | 2019-06-14 | 长沙理工大学 | A kind of vision data complementing method based on the low-rank tensor estimation for limiting nuclear norm |
CN110298798A (en) * | 2019-06-20 | 2019-10-01 | 浙江工业大学 | A kind of image repair method based on the completion of low-rank tensor Yu discrete full variation |
CN111260571A (en) * | 2020-01-11 | 2020-06-09 | 浙江工业大学 | Depth image restoration method based on non-convex low-rank low gradient |
CN111325697A (en) * | 2020-03-04 | 2020-06-23 | 西安交通大学 | Color image restoration method based on tensor eigen transformation |
-
2020
- 2020-08-21 CN CN202010846940.8A patent/CN112241938B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109886884A (en) * | 2019-01-21 | 2019-06-14 | 长沙理工大学 | A kind of vision data complementing method based on the low-rank tensor estimation for limiting nuclear norm |
CN110298798A (en) * | 2019-06-20 | 2019-10-01 | 浙江工业大学 | A kind of image repair method based on the completion of low-rank tensor Yu discrete full variation |
CN111260571A (en) * | 2020-01-11 | 2020-06-09 | 浙江工业大学 | Depth image restoration method based on non-convex low-rank low gradient |
CN111325697A (en) * | 2020-03-04 | 2020-06-23 | 西安交通大学 | Color image restoration method based on tensor eigen transformation |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114202107A (en) * | 2021-11-18 | 2022-03-18 | 华南理工大学 | Ultra-short-term wind speed prediction method and device |
CN114202107B (en) * | 2021-11-18 | 2024-04-19 | 华南理工大学 | Ultra-short-term wind speed prediction method and device |
CN114119426A (en) * | 2022-01-26 | 2022-03-01 | 之江实验室 | Image reconstruction method and device by non-local low-rank conversion domain and full-connection tensor decomposition |
Also Published As
Publication number | Publication date |
---|---|
CN112241938B (en) | 2024-02-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Ding et al. | Low-rank tensor completion using matrix factorization based on tensor train rank and total variation | |
WO2018120329A1 (en) | Single-frame super-resolution reconstruction method and device based on sparse domain reconstruction | |
Mrázek et al. | On robust estimation and smoothing with spatial and tonal kernels | |
Balmelli et al. | Space‐optimized texture maps | |
Lefkimmiatis et al. | Nonlocal structure tensor functionals for image regularization | |
Wu et al. | The application of nonlocal total variation in image denoising for mobile transmission | |
CN111340697B (en) | Image super-resolution method based on clustered regression | |
CN105631807A (en) | Single-frame image super resolution reconstruction method based on sparse domain selection | |
Liu et al. | Block matching local SVD operator based sparsity and TV regularization for image denoising | |
CN111242999B (en) | Parallax estimation optimization method based on up-sampling and accurate re-matching | |
Patel et al. | Separated component-based restoration of speckled SAR images | |
CN112241938A (en) | Image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation | |
Mikaeli et al. | Single-image super-resolution via patch-based and group-based local smoothness modeling | |
Li et al. | A novel weighted anisotropic total variational model for image applications | |
Jiang et al. | A new nonlocal means based framework for mixed noise removal | |
CN109741258B (en) | Image super-resolution method based on reconstruction | |
CN111260585A (en) | Image recovery method based on similar convex set projection algorithm | |
CN112734763B (en) | Image decomposition method based on convolution and K-SVD dictionary joint sparse coding | |
Liu et al. | Image inpainting algorithm based on tensor decomposition and weighted nuclear norm | |
CN112270650B (en) | Image processing method, system, medium, and apparatus based on sparse autoencoder | |
Bigdeli et al. | Image restoration using plug-and-play cnn map denoisers | |
Li et al. | A mixed noise removal algorithm based on multi-fidelity modeling with nonsmooth and nonconvex regularization | |
Keren et al. | Denoising color images using regularization and “correlation terms” | |
Zhang et al. | Optimization-inspired Cumulative Transmission Network for image compressive sensing | |
CN111951185B (en) | Low-rank tensor loop decomposition-based high-order structuring and image restoration method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |