CN111325697B - Color image restoration method based on tensor eigen transformation - Google Patents
Color image restoration method based on tensor eigen transformation Download PDFInfo
- Publication number
- CN111325697B CN111325697B CN202010144793.XA CN202010144793A CN111325697B CN 111325697 B CN111325697 B CN 111325697B CN 202010144793 A CN202010144793 A CN 202010144793A CN 111325697 B CN111325697 B CN 111325697B
- Authority
- CN
- China
- Prior art keywords
- tensor
- matrix
- column
- pixel
- eigen
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 230000009466 transformation Effects 0.000 title claims abstract description 28
- 239000011159 matrix material Substances 0.000 claims description 127
- 238000000354 decomposition reaction Methods 0.000 claims description 39
- 238000007634 remodeling Methods 0.000 claims description 10
- 125000004432 carbon atom Chemical group C* 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 150000001875 compounds Chemical class 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a color image restoration method based on tensor eigen transformation, which utilizes designed tensor eigen transformation to directly obtain the structural characteristics of an image to be restored and better reserve the internal association between image pixels. The image restoration method is better in the aspect of keeping the internal relation of the pixels, so that the method can restore the detailed characteristics of the damaged pixels and can be used for restoring the damaged natural color images.
Description
[ technical field ] A
The invention belongs to the technical field of image processing, and relates to a color image restoration method based on tensor eigen transformation.
[ background ] A method for producing a semiconductor device
In the modern society, in the era of data explosion, the development of computer technology is promoted by data collection and processing, however, unlike the past, in the era of big data, the collected data has a more complex structure, and the data can be well characterized by using a high-dimensional multi-linear structure. In the receiving and transmitting process of data, the obtained data is likely to be partially damaged due to environmental interference, and even if the data is received and transmitted under ideal conditions, it is desirable to compress the data to improve the utilization rate of resources. Based on the low rank matrix completion model, we can adopt fewer samples to mine possible values of other unknown samples, and this technique has been applied to various recommendation systems. In real life, if an attempt is made to mine an unknown sample, the influence factors are likely to be more, and therefore, the efficiency of solving the unknown sample by using the matrix is not high. The tensor is a high-dimensional multi-linear structure which can well reflect the internal connection between high-order data. On the basis of a low-rank matrix completion model, the low-rank tensor completion model is widely applied to the fields of computer vision, machine learning, data mining, neuroscience and the like.
Image processing is an important direction of research in computer vision, where image inpainting techniques aim to recover with a high probability damaged pixel values in an image. Under a low-rank tensor completion model, a color image can be represented as a third-order tensor, and how to mine the low-rank property of the tensor becomes an important problem of color image completion. The low rank property of the tensor is generally defined by various tensor decomposition methods, and currently, the common tensor decomposition methods include: 1) TUCKER Decomposition method 2) CANDECOMP-PARAFAC (CP) Decomposition method 3) Tensor column Decomposition method (sensor Train Decomposition) 4) Tensor Singular Value Decomposition method (sensor Singular Value Decomposition) and the like. LIU et al (the article is named as tension Completion for Estimating Missing Values in Visual Data) utilizes a TUCKER decomposition method to respectively expand tensors into mode matrixes according to different modes, and utilizes the rank of the mode matrixes to define the low rank of original tensors, but the method has great relevance damage to the Data in the original tensors. Under the definition of the CP decomposition method, the problem of NP difficulty in Tensor resolution is solved, and the CP rank of the Bayesian factor estimation Tensor is introduced by ZHAO et al (the article is named Bayesian Robust test factor Factorization for incorporated Multi way Data), so that the low rank of the original Tensor is defined, but the complexity of the calculation process of the method is high. Bengua et al (thesis entitled Efficient Tensor composition for Color Image and Video Recovery) defines a new Tensor generating matrix method by using traditional Tensor column decomposition, and further excavates the low-rank property of the original Tensor according to the rank of the generating matrix under the new method. ZHANG et al (article titled Novel Methods for Multilinear Data Completion and De-noised base on sensor-SVD) defines TUBAL rank of Tensor Based on Tensor singular value decomposition method, and the method does not rely on Tensor development, well preserves Data relevance of Tensor, but TUBAL rank has the possibility of large description error in describing Tensor low rank. LI (paper title is Low Rank document Completion with Total Variation for Visual Data Inpainting) and other people propose embedding a Total Variation method into a Tensor Completion process without the help of Tensor development to define Low Rank, but the principle process of the method is complex, and the applicability to the image with larger damage is not strong.
[ summary of the invention ]
The invention aims to solve the problems in the prior art and provides a color image restoration method based on tensor eigen transformation, which can better save the relevance among color pixels in color image restoration, thereby obtaining a clearer restoration result and achieving the effect that human eyes cannot easily perceive restoration traces.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
a color image restoration method based on tensor eigen transformation comprises the following steps:
step 1, obtaining damaged image, namely to-be-compensated tensor
Determining a set of all missing pixels corresponding to the damaged region
Setting preset parameters epsilon, beta, r e ,r c Setting remodeling parameters rho 1 ,ρ 2 Determining a set error requirement and a maximum iteration requirement;
step 2, obtaining the full tensor to be compensated by utilizing the tensor eigen transformation tau
The corresponding eigen-matrix E, noted:
step 3, using a threshold operator D ε,β Updating the intrinsic matrix E in the step 2, and recording the updated intrinsic matrix as E n ：
E n ＝D ε,β (E)
Step 4, introducing auxiliary tensor
Using inverse tensor eigentransformations τ -1 Determining an updated eigen matrix E n Corresponding tensor
Recording as follows:
step 5, determining the update tensor
If the update tensor pixel exists in the pixel damaged area
Order to
If the update tensor pixel is not in the pixel damage area
Order to
Step 6, judging the relative error
Whether the set error requirement is met or not, or whether the iteration frequency reaches the maximum iteration requirement or not; if the error requirement or the maximum iteration requirement is met, outputting the latest tensor
Namely the repaired image; otherwise make
And returning to the step 2 to enter an iterative loop.
The invention further improves the following steps:
in step 1, a set of all missing pixels corresponding to the damaged area is determined
The specific method comprises the following steps:
step 1.1, reading all pixel points of an image to be restored, enabling all the points with pixel values not being 0 to be known pixel points, and recording the positions of the known pixel points as a set omega;
step 1.2, all the points with the pixel values of 0 are made to be unknown pixel points, and the position set of the unknown pixel points is recorded to be
In the step 2, the tensor to be compensated is obtained by utilizing the tensor eigen transformation tau
The specific method of the corresponding eigen matrix E is as follows:
step 2.1, using improved tensor column decomposition to complete tensor to be compensated
Decomposed into tensor column kernels
The concrete expression is as follows:
in the above-mentioned formula, the compound has the following structure,
representing the coordinate in the tensor to be complemented as (i) c ,j c ,k c ) The value of the pixel of (a) is,
indicating the amount of tension to be compensated
The tensor of the reshaped intermediate medium,
a b-th front tangent matrix representing an a-th tensor column core;
step 2.2, carrying out tensor singular value decomposition on one tensor column core in the step:
in the above equation, a tensor column kernel is decomposed into the form of the product of three tensors, where
Called f-diagonal tensor, taking initial value a =1;
step 2.3, in the Fourier domain, the tensor column in the previous step is centered
Corresponding f-diagonal tensor
The values in (1) are mapped into a matrix E according to coordinates, and the corresponding coordinate relationship is expressed as:
in the above formula, the first and second carbon atoms are,
representing tensor column kernels
F-diagonal tensor in Fourier domain, ζ denotes tensor column core pointer, ξ denotes eigenmatrix pointer, I a Representing the tensor of the intermediate medium to be complemented
The a-th dimension; the specific value of the tensor column core pointer zeta is zeta =1 when a =1 or a =3, and zeta = i in other cases; the specific values of the intrinsic matrix pointers are:
in the above formula, r e And r c Respectively representing the edge rank and the center rank in the tensor eigen transformation, wherein the edge rank is always set as r e ＝1；
Step 2.4, if a<3, taking the intermediate tensor to be compensated
A +1 th tensor column core of
I.e. let a = a +1, return to step 2.3 and enter iteration, otherwise determine tensor
The eigenmatrix of (a) is E.
The specific method of the improved tensor column decomposition is as follows:
step 2.1.1, first, the tensor corresponding to the color image is calculated
Remodel into
Wherein the setting parameter p is utilized 1 ,ρ 2 The remodeling relationship of (A) is that,
step 2.1.2, introducing an auxiliary temporary tensor
Order to
Tensor is expressed
Reshaped into a corresponding matrix
Step 2.1.3, remodeling the matrix obtained in the step 2.1.2
Decomposition according to matrix singular values:
C＝U·S·V T (5)
wherein, the first and the second end of the pipe are connected with each other,
respectively called left and right singular matrices of the matrix C,
is a diagonal matrix;
step 2.1.4, get matrix
Front r of c The columns form a new matrix
Get matrix
Front r of c Column forming a new matrix
Get matrix
Front r of c Row and r c Columns forming a new matrix
Step 2.1.5, matrix
Remodelling into first volumetric core
Determining updated matrices
Step 2.1.6, matrix
Remolding into a new matrix
And is obtained according to the singular value decomposition of the matrix, i.e. the decomposition in equation (5)
Get matrix
Front r of c Column forming a new matrix
Get matrix
Front r of c The columns form a new matrix
Get matrix
Front r of (2) c Row and r c Columns forming a new matrix
Step 2.1.7, matrix
Reshaping to a second tensor column core
Determining updated matrices
C is to be d Remodeled to a third tensor column core
In step 3, the specific method of tensor singular value decomposition is as follows:
step 2.2.1, get a quantitative core
Performing three-dimensional Fourier transform to obtain tensor column core in corresponding Fourier domain
Taking an initial value s =1;
step 2.2.2, take tensor column core
And recording the matrix as the s-th front section matrix
Namely have
To pair
Performing singular value decomposition of the matrix to obtain left and right singular matrices thereof
And diagonal matrix
Namely that
Step 2.2.3, create tensor
Setting the initial value of the newly-built tensor to be 0, and setting the left and right singular matrixes in the previous step
And diagonal matrix
The s-th front tangent plane matrix given to the newly created tensor, i.e.
Step 2.2.4, if s<I 3 And (3) returning to the step 2.2.2 to enter a loop by making s = s +1, otherwise, determining three tensors
Step 2.2.5, for
Respectively carrying out three-dimensional inverse Fourier transform to obtain tensors of real number domain
To sum up, a tensor column core is obtained
Singular value decomposition, i.e. having
The threshold operator D in step 3 ε,β The concrete expression is as follows:
where sign (x) is a sign function, parameter c 0 ,c 1 ,c 2 Comprises the following steps:
wherein epsilon and beta are preset parameters input in advance, and a symbol D is defined ε,β (X) represents the thresholding operation on all elements in matrix X.
In said step 4, the inverse tensor eigen-transform τ -1 The specific method comprises the following steps:
step 4.1, in the Fourier domain, a tensor eigen transformation method is adopted, and a plurality of corresponding tensor column cores are obtained through calculation of the eigen matrix E
F-diagonal tensor of
Taking an initial value a =1;
step 4.2, mixing
Is obtained by inverse Fourier transform
From the f-diagonal tensor in the real domain
And corresponding left and right singular tensors
Calculating to obtain tensor column core
In the above formula, the symbol "+" represents the tensor product;
step 4.3, if a<3, the a +1 f-diagonal tensor is taken
Making a = a +1, returning to the step 4.2 to enter iteration, and otherwise, entering the next step;
step 4.4, according to the obtained tensor column cores
Calculating to obtain the updated tensor corresponding to the intrinsic matrix E
In the above formula, subscripts i, j, k represent pixel points in tensor
The position in (1); will update the tensor
Reshaped into an original image
Tensor of equal size
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a novel tensor eigen-transformation, which is not subjected to any tensor development matrix to define the low rank of the tensor, but directly excavates the low rank of the tensor by decomposing the structural characteristics of the tensor. Compared with the prior art, the method has the advantages that based on tensor eigentransformation, the low-rank property of the tensor to be complemented can be solved in an iterative manner, and meanwhile, the relevance among tensor elements is greatly saved; in addition, under a low-rank tensor completion model, the proposed tensor eigen transformation recovers damaged pixels with higher probability by utilizing the relevance between known pixels and the damaged pixels, and the method is better at exploring the tensor internal relation, so that a more accurate repaired image can be obtained, and the effect that the repaired trace is not easily distinguished in the aspect of image detail repair can be achieved.
[ description of the drawings ]
Fig. 1 is a flowchart of a color image restoration method based on tensor eigen transformation according to the present invention;
FIG. 2 is a diagram of a color image defect area according to the present invention
And a schematic diagram of a known region Ω, wherein the black region is used to illustrate the damaged missing region;
FIG. 3 is a schematic diagram of the tensor eigen-transform of the present invention;
fig. 4 is a comparison graph of the application effect of the present invention and the prior art, which shows the original image of a color image, a damaged image and an image repaired by each method, and the repaired image has a relative error of recovery attached.
[ detailed description ] embodiments
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments, and are not intended to limit the scope of the present disclosure. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Various structural schematics according to the disclosed embodiments of the invention are shown in the drawings. The figures are not drawn to scale, wherein certain details are exaggerated and some details may be omitted for clarity of presentation. The shapes of the various regions, layers and their relative sizes, positional relationships are shown in the drawings as examples only, and in practice deviations due to manufacturing tolerances or technical limitations are possible, and a person skilled in the art may additionally design regions/layers with different shapes, sizes, relative positions, according to the actual needs.
In the context of the present disclosure, when a layer/element is referred to as being "on" another layer/element, it can be directly on the other layer/element or intervening layers/elements may be present. In addition, if a layer/element is "on" another layer/element in one orientation, then that layer/element may be "under" the other layer/element when the orientation is reversed.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The invention is described in further detail below with reference to the accompanying drawings:
in the prior art, the relation between known and unknown pixels cannot be utilized to the maximum extent, various generating modes are generally adopted when low rank is defined, the relevance among the pixels is greatly damaged, and the recovered image has certain distortion in visual effect. Therefore, we propose tensor eigen transformation to overcome the defect, as shown in fig. 1, in an embodiment of the present invention, a tensor eigen transformation-based color image restoration method does not need to define the low rank property of the original tensor by means of a tensor matrix-generating approach, directly obtains the low rank property of the original tensor by using the inherent structural features of the tensor eigen transformation, greatly utilizes and saves the inherent correlation between pixels of the color image, overcomes the defect of insufficient storage of pixel correlation in the prior art, and makes the restored color image more accurate and have greater generalization.
Referring to fig. 1, the color image restoration method based on tensor eigen transformation of the present invention includes the following steps:
step 1, obtaining damaged image (i.e. to-be-compensated tensor)
) With a pixel of 256, i.e. corresponding to the dimension of the tensor to be compensated
Determining a set of all missing pixels corresponding to the damaged region
Setting proper preset parameters epsilon, beta and r c Wherein the edge rank of the tensor eigen-transform is always set to r e =1; setting remodeling parameters rho 1 ,ρ 2 Determining a set error requirement and a maximum iteration requirement; in this embodiment, the parameters are set to e =0.3, β =100 c =20; remodeling parameter ρ 1 ＝4,ρ 2 =4, the maximum number of iterations is 500, and the error requirement is less than 1 × 10 -3 The missing pixel ratio is set to 70%, i.e. 70% of the color image pixels are missing or damaged;
as shown in FIG. 2, to identify the pixel positions to be repaired, a set of all missing pixels corresponding to the damaged area is first determined
The concrete setting is as follows:
step 1.1, reading all pixel points of an image to be restored, enabling all the points with pixel values not being 0 to be known pixel points, and recording the positions of the known pixel points as a set omega;
step 1.2, all the points with the pixel values of 0 are made to be unknown pixel points, and the position set of the unknown pixel points is recorded to be
Step 2, obtaining the tensor to be compensated by utilizing the tensor eigen transformation tau
The corresponding eigen-matrix E, denoted
As shown in FIG. 3, the method uses tensor eigen transformation tau to obtain the tensor to be compensated
The corresponding intrinsic matrix E specifically comprises the following steps:
step 2.1, the tensor to be compensated is decomposed by using the improved tensor column
Decomposed into tensor column kernels
The concrete expression is as follows:
in the above-mentioned formula, the compound has the following structure,
representing the coordinate in the tensor to be complemented as (i) c ,j c ,k c ) The value of the pixel of (a) is,
indicating the amount of tension to be compensated
The tensor of the reshaped intermediate medium,
a b-th Frontal Slice matrix (front Slice) representing an a-th Tensor column Core (sensor Train Core);
specifically, the improved tensor column decomposition described in the tensor eigentransform has the following steps:
step 2.1.1, in order to ensure the reconstruction precision, firstly, the tensor corresponding to the color image
Remodel into
Wherein use is made of a set parameter p 1 ,ρ 2 The re-shaping relationship of (1) is that,
step 2.1.2, introduce the auxiliary temporary tensor
Order to
Will tensor
Reshaped into a corresponding matrix
Step 2.1.3, remodeling the matrix obtained in the step 2.1.2
The matrix singular value decomposition can be specifically expressed as:
C＝U·S·V T (2)
wherein
Left and right singular matrices referred to as matrix C respectively,
is a diagonal matrix;
step 2.1.4, get matrix
Front r of (2) c The columns form a new matrix
Get matrix
Front r of c The columns form a new matrix
Get matrix
Front r of (2) c Row and r c Columns forming a new matrix
Step 2.1.5, matrix
Remodelling into first volumetric core
Determining updated matrices
Step 2.1.6, matrix
Remolding into a new matrix
And is obtained according to the singular value decomposition of the matrix, i.e. the decomposition in equation (2)
Get matrix
Front r of (2) c Column forming a new matrix
Get matrix
Front r of c Column forming a new matrix
Get matrix
Front r of c Row and r c Columns forming a new matrix
Step 2.1.7, matrix
Reshaping to a second tensor column core
Determining updated matrices
C is to be d Remoulds intoCore of three tensor columns
Step 2.2, carrying out tensor singular value decomposition on one tensor column core in the step, wherein the tensor singular value decomposition is specifically represented as follows:
in the above equation, a tensor column kernel is decomposed into the form of the product of three tensors, where
Called f-diagonal Tensor (f-diagonal Tensor), taking the initial value a =1;
the tensor singular value decomposition in the tensor eigentransformation specifically comprises the following steps:
step 2.2.1, get a quantitative core
Performing three-dimensional Fourier transform to obtain tensor column core in corresponding Fourier domain
Taking an initial value s =1;
step 2.2.2, get tensor column core
And recording the matrix as the s-th front section matrix
Namely have
For is to
Performing singular value decomposition of the matrix to obtain left and right singular matrices thereof
And diagonal matrix
Namely that
Step 2.2.3 creating tensors
Setting the initial value of the newly-built tensor to be 0, and setting the left and right singular matrixes in the previous step
And diagonal matrix
The s-th front tangent plane matrix given to the newly created tensor, i.e.
Step 2.2.4, if s<I 3 Let s = s +1, return to step 2.2.2 to enter a loop, otherwise determine the three tensors
Step 2.2.5, for
Respectively carrying out three-dimensional inverse Fourier transform to obtain tensors of real number domain
To sum up, a tensor column core is obtained
Singular value decomposition, i.e. having
Step 2.3, in the Fourier domain, the tensor column in the previous step is centered
Corresponding f-diagonal tensor
The values in (1) are mapped into a matrix E according to coordinates, and the corresponding coordinate relationship can be expressed as:
in the above formula, the first and second carbon atoms are,
representing tensor column core
F-diagonal tensor in the Fourier domain, ζ denotes the tensor column core pointer, ξ denotes the eigenmatrix pointer, I a Representing the tensor of the intermediate medium to be complemented
The a-th dimension; the specific value of the tensor column core pointer zeta is zeta =1 when a =1 or a =3, and zeta = i in other cases; the specific value of the intrinsic matrix pointer is
In the above formula, r e And r c Respectively representing the edge rank and the center rank in the tensor eigen transformation, wherein the edge rank of the method is always set as r e ＝1；
Step 2.4, if a<3, taking the intermediate tensor to be compensated
A +1 st tensor column core of
I.e. let a = a +1, return to step 2.3 and enter iteration, otherwise determine tensor
The eigenmatrix of (a) is E.
Step 3, using a threshold operator D ε,β Updating the intrinsic matrix E in the step 2, and recording the updated intrinsic matrix as E n I.e. E n ＝D ε,β (E)；
The threshold operator D described in this step ε,β The method specifically comprises the following steps:
where sign (x) is a sign function, parameter c 0 ,c 1 ,c 2 Is required to be determined as
Epsilon and beta are preset parameters input in advance, and a symbol D is defined ε,β (X) represents the thresholding operation performed on all elements in matrix X.
Step 4, introducing auxiliary tensor
Using inverse tensor eigentransformations tau -1 Determining an updated eigen matrix E n Corresponding tensor
Record as
Inverse tensor eigentransform τ in this step -1 Is the inverse process of the tensor eigen-transform tau, it is emphasized that the inverse tensor eigen-transform tau -1 Subject to τ, otherwise τ -1 Will become meaningless; inverse tensor eigentransform τ -1 The method comprises the following specific steps:
step 4.1, in the Fourier domain, referring to the tensor eigen transformation method in the formulas (4) and (5), calculating the corresponding tensor column cores by the eigen matrix E
F-diagonal tensor of
Taking an initial value a =1;
step 4.2, mixing
Is obtained by inverse Fourier transform
From the f-diagonal tensor in the real domain
And corresponding left and right singular tensors
Calculating to obtain tensor column core
The expression is as follows:
the symbol "+" in the above formula represents the tensor product;
step 4.3, if a<3, the a +1 f-diagonal tensor is taken
I.e. let a = a +1, return to step 4.2 to enter iteration, otherwise enter the next step.
Step 4.4, according to the obtained plurality of tensor column cores
Calculating to obtain the updated tensor corresponding to the intrinsic matrix E
The concrete expression is as follows:
subscript i, j, k in the above formula represents the tensor of the pixel point
The position of (1); in addition, the tensor will be updated
Reshaped into an original image
Tensor of equal size
Step 5, determining the update tensor
If the update tensor pixel exists in the pixel damaged area
Order to
If the update tensor pixel is not in the pixel damage area
Order to
Step 6, judging relative error
Whether the set error requirement is met or not, or whether the iteration frequency reaches the maximum iteration requirement or not; if the error requirement or the maximum iteration requirement is met, outputting the latest tensor
(i.e., the repaired image), otherwise let
And returning to the step 2 to enter an iterative loop.
As shown in fig. 4, the present embodiment recovers the color image by a tensor eigen transformation-based color image recovery method, and the recovery effect of the recovered image is still better even under an extremely high damage rate, i.e., 70% of the pixels are damaged, and compared with the original image, an accurate recovery result is obtained. For comparison, a low-rank tensor total variation method and a tensor nuclear norm method are respectively set according to default algorithms; the relative error of the recovery of the method of the invention is 10.2%, which is superior to 12.6% and 12.8% of the prior art, and the recovery result of the prior art is visually seen to have errors in brightness and detail. The experimental results show that the method provided by the invention has better performance no matter in objective index evaluation or subjective visual effect.
In the embodiment, the low rank property is defined without any method for expanding the tensor into the matrix, but the proposed tensor eigen transformation is directly utilized to form the framework, so that the low rank property of the tensor can be iteratively obtained under the framework, and meanwhile, the relevance among pixels is ensured and an effective recovery image is obtained.
With reference to this embodiment, an algorithm of a color image restoration method based on tensor eigen transformation is given as follows:
inputting: spoiled colour images
Suitable preset parameters epsilon, beta, r c Wherein r is e =1; setting remodeling parameters rho 1 ,ρ 2 Determining a set error requirement tol and a maximum iteration requirement K;
step 1, initializing a damaged color image
Initializing iteration parameter i =1, initializing relative error
Step 2, judging whether the maximum iteration requirement or the specified error requirement is met, if i is less than or equal to K or error i >tol proceeds to the next step, otherwise let
And terminate the algorithm;
step 3, obtaining tensor based on the eigentransformation of tensor
The eigen matrix E of (a);
step 4, updating the intrinsic matrix E through formulas (6) and (7) to obtain E n ；
Step 5, obtaining an updated eigen matrix E based on the inverse eigen transformation of the tensor n Corresponding tensor of
And step 6, updating the damaged color image,
obtaining the color image restored in this step
Calculating the relative error of the iteration
Step 7, enabling the iteration parameter i = i +1; update the damaged color image
And returning to the step 2;
and (3) outputting: repairing color images
The parameters and related mathematical symbols involved in the algorithm are the same as the partial parameters and symbols in the invention, so that repeated definition of the parameters and the mathematical symbols is avoided.
The color image restoration method based on tensor eigen transformation comprises the steps of firstly reading a damaged color image, identifying and defining a set of known pixels and unknown pixels, and creating a premise for restoring the image by utilizing internal relevance between the known pixels and the unknown pixels; in addition, based on the proposed tensor eigentransformation, corresponding tensor low-rank performance is mined directly according to the structure of the color image, the method greatly preserves pixel relevance, and damaged pixels can be repaired iteratively through simple transformation under the framework of tensor eigentransformation, so that missing pixels can be recovered more accurately.
In this embodiment, a computer device is provided, which may be a terminal. The computing device generally includes a processor, a memory, a network interface, and an input-output device. The processor of the computer device provides computing support for the method of the invention; the memorizer provides a built-in operating system, a running environment of the program and a stored computer program for the method; the network interface provides network connection and communication exchange functions for the method; the input device is used for inputting an image to be restored, and may be a keyboard, a mouse or the like; the output device is used for presenting the repaired image, and may be specifically a display screen or the like. More specifically, the software platform of the present embodiment uses MATLAB R2015a; the hardware platform uses INTEL CORE CPU and memory 4GB; the experimental method is the method, the low-rank tensor total variation method and the tensor nuclear norm method.
The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (3)
1. A color image restoration method based on tensor eigen transformation is characterized by comprising the following steps:
step 1, obtaining damaged image, namely to-be-compensated tensor
Determining a set omega of all missing pixels corresponding to the damaged region ⊥ Setting preset parameters epsilon, beta and r e ,r c Setting remodeling parameters rho 1 ,ρ 2 Determining a set error requirement and a maximum iteration requirement;
step 2, obtaining the full tensor to be compensated by utilizing the tensor eigen transformation tau
The corresponding eigen-matrix E, noted:
the specific method comprises the following steps:
step 2.1, using improved tensor column decomposition to complete tensor to be compensated
Decomposed into tensor column kernels
The concrete expression is as follows:
in the above formula, the first and second carbon atoms are,
representing the coordinate in the tensor to be complemented as (i) c ,j c ,k c ) The value of the pixel of (a) is,
indicating the amount of tension to be compensated
The tensor of the reshaped intermediate medium,
a b-th front tangent matrix representing an a-th tensor column core;
the specific method of the improved tensor column decomposition is as follows:
step 2.1.1, first, the tensor corresponding to the color image
Is reshaped into
Wherein the setting parameter p is utilized 1 ,ρ 2 The remodeling relationship of (A) is that,
step 2.1.2, introduce the auxiliary temporary tensor
Order to
Will tensor
Reshaped into a corresponding matrix
Step 2.1.3, remodeling the matrix obtained in the step 2.1.2
Decomposition according to matrix singular values:
C＝U·S·V T (4)
wherein, the first and the second end of the pipe are connected with each other,
left and right singular matrices referred to as matrix C respectively,
is a diagonal matrix;
step 2.1.4, get matrix
Front r of c The columns form a new matrix
Get matrix
Front r of (2) c The columns form a new matrix
Get matrix
Front r of c Row and r c Columns forming a new matrix
Step 2.1.5, matrix
Remodelling into first volumetric core
Determining updated matrices
Step 2.1.6, matrix
Remolding into a new matrix
And is obtained according to the singular value decomposition of the matrix, i.e. the decomposition in equation (5)
Get matrix
Front r of c Column forming a new matrix
Get matrix
Front r of (2) c The columns form a new matrix
Get matrix
Front r of (2) c Row and r c Columns forming a new matrix
Step 2.1.7, matrix
Remodeled to a second tensor column core
Determining updated matrices
Will C d Reshaping to a third tensor column core
Step 2.2, carrying out tensor singular value decomposition on one tensor column core in the step:
in the above equation, a tensor column core is decomposed into the form of the product of three tensors, where
Called f-diagonal tensor, taking initial value a =1;
the specific method of tensor singular value decomposition is as follows:
step 2.2.1, get a quantitative core
Performing three-dimensional Fourier transform to obtain tensor column core in corresponding Fourier domain
Taking an initial value s =1;
step 2.2.2, get tensor column core
Of (2) and recording the matrix as the s-th front section matrix
Namely have
To pair
Performing singular value decomposition of the matrix to obtain left and right singular matrices thereof
And diagonal matrix
Namely, it is
Step 2.2.3, create tensor
Setting the initial value of the newly-built tensor to be 0, and setting the left and right singular matrixes in the previous step
And diagonal matrix
The s-th front tangent matrix to the newly created tensor, i.e.
Step 2.2.4, if s is less than I 3 Let s = s +1, return to step 2.2.2 to enter a loop, otherwise determine the three tensors
Step 2.2.5, for
Respectively carrying out three-dimensional inverse Fourier transform to obtain tensors of real number domain
To sum up, a tensor column core is obtained
Singular value decomposition of, i.e. having
Step 2.3, in the Fourier domain, the tensor column in the previous step is centered
Corresponding f-diagonal tensor
The values in (b) are mapped into a matrix E according to coordinates, and the corresponding coordinate relationship is expressed as:
in the above formula, the first and second carbon atoms are,
representing tensor column kernelsHeart with heart-shaped
F-diagonal tensor in the Fourier domain, ζ denotes the tensor column core pointer, ξ denotes the eigenmatrix pointer, I a Representing the tensor of the intermediate to be complemented
The a-th dimension; the specific value of the tensor column core pointer zeta is zeta =1 when a =1 or a =3, and zeta = i in other cases; the specific values of the eigen matrix pointers are:
in the above formula, r e And r c Respectively representing the edge rank and the center rank in the tensor eigen transformation, wherein the edge rank needs to be always set as r e ＝1；
Step 2.4, if a is less than 3, the intermediate tensor to be compensated is taken
A +1 th tensor column core of
I.e. let a = a +1, return to step 2.3 and enter iteration, otherwise determine tensor
The intrinsic matrix of (a) is E;
step 3, using a threshold operator D ε，β Updating the intrinsic matrix E in the step 2, and recording the updated intrinsic matrix as E n ：
E n ＝D ε，β (E)
Step 4, introducing auxiliary tensor
Using inverse tensor eigentransformations tau -1 Determining an updated eigen matrix E n Corresponding tensor
Recording as follows:
inverse tensor eigentransform τ -1 The specific method comprises the following steps:
step 4.1, in the Fourier domain, a tensor eigen transformation method is adopted, and a plurality of corresponding tensor column cores are obtained through calculation of the eigen matrix E
F-diagonal tensor of
Taking an initial value a =1;
step 4.2, mixing
Is subjected to inverse Fourier transform to obtain
From the f-diagonal tensor in the real domain
And corresponding left and right singular tensors
Calculating to obtain tensor column core
In the above formula, the symbol "+" represents the tensor product;
step 4.3, if a is less than 3, the a +1 f-diagonal tensor is taken
If a = a +1, returning to the step 4.2 to enter iteration, otherwise, entering the next step;
step 4.4, according to the obtained tensor column cores
Calculating to obtain the updated tensor corresponding to the intrinsic matrix E
In the above formula, subscript i, j, k represents the tensor of the pixel point
The position in (1); will update the tensor
Reshaped into an original image
Tensor of equal size
Step 5, determining the update tensor
If the update tensor pixel exists in the pixel damaged area omega ⊥ Let us order
If the update tensor pixel is not in the pixel damaged area omega ⊥ Let us order
Step 6, judging relative error
Whether the set error requirement is met or not, or whether the iteration frequency reaches the maximum iteration requirement or not; if the error requirement or the maximum iteration requirement is met, outputting the latest tensor
Namely the repaired image; otherwise make the instruction
And returning to the step 2 to enter an iterative loop.
2. The tensor eigentransform-based color image restoration method according to claim 1, wherein in step 1, a set Ω of all missing pixels corresponding to a damaged area is determined ⊥ The specific method comprises the following steps:
step 1.1, reading all pixel points of an image to be restored, enabling all the points with pixel values not being 0 to be known pixel points, and recording the positions of the known pixel points as a set omega;
step 1.2, all the points with the pixel value of 0 are set as unknown pixel points, and the position set of the unknown pixel points is recorded to be omega ⊥ 。
3. The tensor eigentransform-based color image restoration method according to claim 1, characterized in that the threshold operator D in step 3 ε，β The concrete expression is as follows:
where sign (x) is a sign function, parameter c 0 ,c 1 ,c 2 Comprises the following steps:
wherein epsilon and beta are preset parameters input in advance, and a symbol D is defined ε，β (X) represents the thresholding operation performed on all elements in matrix X.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010144793.XA CN111325697B (en) | 2020-03-04 | 2020-03-04 | Color image restoration method based on tensor eigen transformation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010144793.XA CN111325697B (en) | 2020-03-04 | 2020-03-04 | Color image restoration method based on tensor eigen transformation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111325697A CN111325697A (en) | 2020-06-23 |
| CN111325697B true CN111325697B (en) | 2022-10-25 |
Family
ID=71173085
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010144793.XA Active CN111325697B (en) | 2020-03-04 | 2020-03-04 | Color image restoration method based on tensor eigen transformation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111325697B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112241938B (en) * | 2020-08-21 | 2024-02-13 | 浙江工业大学 | Image restoration method based on smooth Tak decomposition and high-order tensor Hakking |
| CN112381725B (en) * | 2020-10-16 | 2024-02-02 | 广东工业大学 | Image restoration method and device based on depth convolution countermeasure generation network |
| CN114119426B (en) * | 2022-01-26 | 2022-07-01 | 之江实验室 | Image reconstruction method and device by non-local low-rank conversion domain and full-connection tensor decomposition |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107392107A (en) * | 2017-06-24 | 2017-11-24 | 天津大学 | A kind of face feature extraction method based on isomery tensor resolution |
| CN109886884A (en) * | 2019-01-21 | 2019-06-14 | 长沙理工大学 | A kind of vision data complementing method based on the low-rank tensor estimation for limiting nuclear norm |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7369699B1 (en) * | 2003-08-29 | 2008-05-06 | Apple Inc. | Methods and apparatuses for restoring color and enhancing electronic images |
| JP6327922B2 (en) * | 2014-04-25 | 2018-05-23 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
| CN107016649A (en) * | 2017-02-24 | 2017-08-04 | 同济大学 | A kind of vision data complementing method estimated based on local low-rank tensor |
| CN109241491A (en) * | 2018-07-28 | 2019-01-18 | 天津大学 | The structural missing fill method of tensor based on joint low-rank and rarefaction representation |
| CN109191404B (en) * | 2018-09-07 | 2020-06-26 | 西安交通大学 | Hyperspectral image restoration method based on E-3DTV regularization |
| CN109978783A (en) * | 2019-03-19 | 2019-07-05 | 上海交通大学 | A kind of color image restorative procedure |
| CN110298798B (en) * | 2019-06-20 | 2021-02-19 | 浙江工业大学 | Image restoration method based on low-rank tensor completion and discrete total variation |
-
2020
- 2020-03-04 CN CN202010144793.XA patent/CN111325697B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107392107A (en) * | 2017-06-24 | 2017-11-24 | 天津大学 | A kind of face feature extraction method based on isomery tensor resolution |
| CN109886884A (en) * | 2019-01-21 | 2019-06-14 | 长沙理工大学 | A kind of vision data complementing method based on the low-rank tensor estimation for limiting nuclear norm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111325697A (en) | 2020-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Nonlocal low-rank tensor completion for visual data | |
| Webster et al. | Detecting overfitting of deep generative networks via latent recovery | |
| CN110298798B (en) | Image restoration method based on low-rank tensor completion and discrete total variation | |
| CN111325697B (en) | Color image restoration method based on tensor eigen transformation | |
| CN112115783A (en) | Human face characteristic point detection method, device and equipment based on deep knowledge migration | |
| CN113240596A (en) | Color video recovery method and system based on high-order tensor singular value decomposition | |
| Cao et al. | New architecture of deep recursive convolution networks for super-resolution | |
| CN112184547B (en) | Super resolution method of infrared image and computer readable storage medium | |
| Muhammad et al. | Multi-scale Xception based depthwise separable convolution for single image super-resolution | |
| CN115147426A (en) | Model training and image segmentation method and system based on semi-supervised learning | |
| Wang et al. | End-to-end trainable network for superpixel and image segmentation | |
| Shao et al. | Generative image inpainting with salient prior and relative total variation | |
| Wang et al. | Joint depth map super-resolution method via deep hybrid-cross guidance filter | |
| Xu et al. | Image compressive sensing recovery via group residual based nonlocal low-rank regularization | |
| CN109902720B (en) | Image classification and identification method for depth feature estimation based on subspace decomposition | |
| CN116778164A (en) | Semantic segmentation method for improving deep V < 3+ > network based on multi-scale structure | |
| Wei et al. | 3D face image inpainting with generative adversarial nets | |
| Yang et al. | Face inpainting via learnable structure knowledge of fusion network | |
| Meng et al. | Siamese CNN-based rank learning for quality assessment of inpainted images | |
| Ye et al. | A sparsity-promoting image decomposition model for depth recovery | |
| Li et al. | An optimization-driven network with knowledge prior injection for HSI denoising | |
| CN113763313A (en) | Text image quality detection method, device, medium and electronic equipment | |
| CN113052798A (en) | Screen aging detection model training method and screen aging detection method | |
| CN112346126A (en) | Low-order fault identification method, device, equipment and readable storage medium | |
| Navasardyan et al. | The family of onion convolutions for image inpainting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |