CN111325697A - Color image restoration method based on tensor eigen transformation - Google Patents
Color image restoration method based on tensor eigen transformation Download PDFInfo
- Publication number
- CN111325697A CN111325697A CN202010144793.XA CN202010144793A CN111325697A CN 111325697 A CN111325697 A CN 111325697A CN 202010144793 A CN202010144793 A CN 202010144793A CN 111325697 A CN111325697 A CN 111325697A
- Authority
- CN
- China
- Prior art keywords
- tensor
- matrix
- column
- eigen
- color image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 230000009466 transformation Effects 0.000 title claims abstract description 36
- 239000011159 matrix material Substances 0.000 claims description 126
- 238000000354 decomposition reaction Methods 0.000 claims description 39
- 238000007634 remodeling Methods 0.000 claims description 11
- 125000004432 carbon atom Chemical group C* 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 2
- 238000011084 recovery Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a color image restoration method based on tensor eigen transformation, which utilizes designed tensor eigen transformation to directly obtain the structural characteristics of an image to be restored and better reserve the internal association between image pixels. The image restoration method is better in the aspect of keeping the internal relation of the pixels, so that the method can restore the detail characteristics of the damaged pixels and can be used for restoring the damaged natural color image.
Description
[ technical field ] A method for producing a semiconductor device
The invention belongs to the technical field of image processing, and relates to a color image restoration method based on tensor eigen transformation.
[ background of the invention ]
The modern society is in the era of data explosion, and the collection and processing of data promote the development of computer technology, however, different from the past, in the era of big data, the collected data has a more complex structure, and the data can be well represented by using a high-dimensional multi-linear structure. In the receiving and transmitting process of data, the obtained data is likely to be partially damaged due to environmental interference, and even if the data is received and transmitted under ideal conditions, it is desirable to compress the data to improve the utilization rate of resources. Based on the low rank matrix completion model, we can adopt fewer samples to mine the possible values of other unknown samples, and the technology is applied to various recommendation systems. In real life, if an attempt is made to mine an unknown sample, the influence factors are likely to be more, and therefore, the efficiency of solving the unknown sample by using the matrix is not high. The tensor is a high-dimensional multi-linear structure which can well reflect the internal connection between high-order data. On the basis of a low-rank matrix completion model, the low-rank tensor completion model is widely applied to the fields of computer vision, machine learning, data mining, neuroscience and the like.
Image processing is an important research direction of computer vision, wherein image restoration techniques are aimed at restoring damaged pixel values in an image with a high probability. Under a low-rank tensor completion model, a color image can be represented as a third-order tensor, and how to mine the low-rank property of the tensor becomes an important problem of color image completion. The low rank property of the tensor is generally defined by decomposition methods of various tensors, and currently, common tensor decomposition methods include: 1) tucker Decomposition 2) CANDECOMP-PARAFAC (CP) Decomposition 3) Tensor column Decomposition 4) Tensor Singular Value Decomposition 4 (Tensor Singular Value Decomposition). LIU et al (thesis entitled Tensor completing and Estimating Missing Values in Visual Data) respectively expand tensors into mode matrixes according to different modes by using a TUCKER decomposition method, and the low rank of the original Tensor is defined by using the rank of the mode matrixes, but the relevance damage of the method to the Data in the original Tensor is large. Under the definition of a CP decomposition method, the order of Tensor solving is an NP difficult problem, and the CP order of the Bayesian factor estimation Tensor is introduced by ZHAO et al (a paper titled Bayesian Robust test factor for incorporated Multi Data), so that the low order of the original Tensor is defined, but the complexity of the calculation process of the method is higher. Bengua et al (thesis entitled Efficient Tensor composition for Color Image and video recovery) defines a new Tensor generating matrix method by using traditional Tensor column decomposition, and further excavates the low-rank property of the original Tensor according to the rank of the generating matrix under the new method. ZHANG et al (article titled Novel Methods for multilinear Data Completion and De-noised base on sensor-SVD) defines TUBAL rank of Tensor Based on Tensor singular value decomposition method, and the method does not rely on Tensor development, well preserves Data relevance of Tensor, but TUBAL rank has the possibility of large description error in describing Tensor low rank. LI (thesis entitled Low rank document Assembly with Total Variation for Visual Data Inpainting) et al propose embedding the Total Variation method into the Tensor Completion process without the help of Tensor development to define low rank, but the principle process of the method is more complicated, and the applicability to the image with larger damage is not strong.
[ summary of the invention ]
The invention aims to solve the problems in the prior art and provides a color image restoration method based on tensor eigen transformation, which can better save the relevance among color pixels in color image restoration, thereby obtaining a clearer restoration result and achieving the effect that human eyes cannot easily perceive restoration traces.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
a color image restoration method based on tensor eigen transformation comprises the following steps:
step 1, obtaining damaged image, namely to-be-compensated tensorDetermining a set of all missing pixels corresponding to the damaged regionSetting preset parameters epsilon, β, re,rcSetting the remodeling parameter ρ1,ρ2Determining a set error requirement and a maximum iteration requirement;
step 2, obtaining the full tensor to be compensated by utilizing the tensor eigen transformation tauThe corresponding eigen-matrix E, noted:
step 3, using a threshold operator Dε,βUpdating the intrinsic matrix E in the step 2, and recording the updated intrinsic matrix as En:
En=Dε,β(E)
Step 4, introducing auxiliary tensorUsing inverse tensor eigentransformations tau-1Determining an updated eigen matrix EnCorresponding tensorRecording as follows:
step 5, determining the update tensorIf the update tensor pixel exists in the pixel damaged areaOrder toIf the update tensor pixel is not in the pixel damage areaOrder to
Step 6, judging relative errorWhether the set error requirement is met or not, or whether the iteration frequency reaches the maximum iteration requirement or not; if the error requirement or the maximum iteration requirement is met, outputting the latest tensorNamely the repaired image; otherwise makeAnd returning to the step 2 to enter an iterative loop.
The invention further improves the following steps:
in step 1, a set of all missing pixels corresponding to the damaged area is determinedThe specific method comprises the following steps:
step 1.1, reading all pixel points of an image to be restored, enabling all the points with pixel values not being 0 to be known pixel points, and recording the positions of the known pixel points as a set omega;
step 1.2, all the points with the pixel values of 0 are made to be unknown pixel points, and the position set of the unknown pixel points is recorded to be
In the step 2, the tensor to be compensated is obtained by utilizing the tensor eigen transformation tauThe specific method of the corresponding eigen matrix E is as follows:
step 2.1, using improved tensor column decomposition to complete tensor to be compensatedDecomposed into tensor column kernelsThe concrete expression is as follows:
in the above formula, the first and second carbon atoms are,representing the coordinate in the tensor to be complemented as (i)c,jc,kc) The value of the pixel of (a) is,indicating the amount of tension to be compensatedThe tensor of the reshaped intermediate medium,a b-th front tangent matrix representing an a-th tensor column kernel;
step 2.2, carrying out tensor singular value decomposition on one tensor column core in the step:
in the above equation, a tensor column kernel is decomposed into the form of the product of three tensors, whereCalled f-diagonal tensor, taking the initial value a as 1;
step 2.3, in the Fourier domain, the tensor column in the previous step is centeredCorresponding f-diagonal tensorThe values in (1) are mapped into a matrix E according to coordinates, and the corresponding coordinate relationship is expressed as:
in the above formula, the first and second carbon atoms are,representing tensor column coreF-diagonal tensor in Fourier domain, ζ denotes tensor column core pointer, ξ denotes eigenmatrix pointer, IaRepresenting the tensor of the intermediate medium to be complementedThe a-th dimension; the tensor column core pointer ζ has a specific value of ζ being 1 when a is 1 or 3, and ζ being i in other cases; the specific values of the eigen matrix pointers are:
in the above formula, reAnd rcRespectively representing the edge rank and the center rank in the tensor eigen transformation, wherein the edge rank is always set as re=1;
Step 2.4, if a<3, taking the intermediate tensor to be compensatedA +1 th tensor column core ofI.e. let a be a +1, go back to step 2.3 to enter iteration, otherwise determine tensorThe eigenmatrix of (a) is E.
The specific method of the improved tensor column decomposition is as follows:
step 2.1.1, first, the tensor corresponding to the color imageRemodel intoWherein the setting parameter p is utilized1,ρ2The remodeling relationship of (A) is that,
step 2.1.2, introduce the auxiliary temporary tensorOrder toWill tensorReshaped into a corresponding matrix
Step 2.1.3, remodeling the matrix obtained in the step 2.1.2Decomposition according to matrix singular values:
C=U·S·VT(5)
wherein the content of the first and second substances,respectively called left and right singular matrices of the matrix C,is a diagonal matrix;
step 2.1.4, get matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcRow and rcColumns forming a new matrix
Step 2.1.6, matrixRemolding into a new matrixAnd is obtained according to the singular value decomposition of the matrix, i.e. the decomposition in equation (5)Get matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcRow and rcColumns forming a new matrix
Step 2.1.7, matrixRemodeled to a second tensor column coreDetermining updated matricesC is to bedRemodeled to a third tensor column core
In step 3, the specific method of tensor singular value decomposition is as follows:
step 2.2.1, get a quantitative corePerforming three-dimensional Fourier transform to obtain tensor column core in corresponding Fourier domainTaking an initial value s as 1;
step 2.2.2, get tensor column coreAnd recording the matrix as the s-th front section matrixNamely haveTo pairPerforming singular value decomposition of the matrix to obtain left and right singular matrices thereofAnd diagonal matrixNamely, it is
Step 2.2.3 creating tensorsSetting the initial value of the newly-built tensor to be 0, and setting the left and right singular matrixes in the previous stepAnd diagonal matrixThe s-th front tangent plane matrix given to the newly created tensor, i.e.
Step 2.2.4, if s<I3If s is equal to s +1, the procedure returns to step 2.2.2 to enter a loop, otherwise, the three tensors are determined
Step 2.2.5, forRespectively carrying out three-dimensional inverse Fourier transform to obtain tensors of real number domainTo sum up, a tensor column core is obtainedSingular value decomposition of, i.e. having
The threshold operator D in the step 3ε,βThe concrete expression is as follows:
wherein sign (x) is a sign function, parameter c0,c1,c2Comprises the following steps:
wherein ε, β is the preset parameter input in advance, defining the symbol Dε,β(X) represents the thresholding operation on all elements in matrix X.
In said step 4, the inverse tensor eigen-transform τ-1The specific method comprises the following steps:
step 4.1, in the Fourier domain, a tensor eigen transformation method is adopted, and a plurality of corresponding tensor column cores are obtained through calculation of the eigen matrix EF-diagonal tensor ofTaking an initial value a as 1;
step 4.2, mixingIs obtained by inverse Fourier transformFrom the f-diagonal tensor in the real domainAnd corresponding left and right singular tensorsCalculating to obtain tensor column core
In the above formula, the symbol "+" represents the tensor product;
step 4.3, if a<3, the a +1 f-diagonal tensor is takenIf a is equal to a +1, returning to the step 4.2 to enter iteration, otherwise, entering the next step;
step 4.4, according to the obtained plurality of tensor column coresCalculating to obtain the updating tensor corresponding to the eigen matrix E
In the above formula, subscript i, j, k represents the tensor of the pixel pointThe position of (1); will update the tensorReshaped into an original imageTensor of equal sizeCompared with the prior art, the invention has the following beneficial effects:
the invention provides a novel tensor eigen-transformation, which is not subjected to any tensor development matrix to define the low rank of the tensor, but directly excavates the low rank of the tensor by decomposing the structural characteristics of the tensor. Compared with the prior art, the method has the advantages that based on tensor eigen transformation, the low-rank property of the tensor to be complemented can be solved in an iterative manner, and meanwhile, the relevance among tensor elements is greatly saved; in addition, under a low-rank tensor completion model, the proposed tensor eigen transformation recovers damaged pixels with higher probability by utilizing the relevance between known pixels and the damaged pixels, and the method is better at exploring the tensor internal relation, so that a more accurate repaired image can be obtained, and the effect that the repaired trace is not easily distinguished in the aspect of image detail repair can be achieved.
[ description of the drawings ]
Fig. 1 is a flowchart of a color image restoration method based on tensor eigen transformation according to the present invention;
FIG. 2 is a diagram of a color image defect area according to the present inventionAnd a diagram of a known region Ω, wherein the black area is used to indicate the damaged missing region;
FIG. 3 is a schematic diagram of the tensor eigen-transforms of the present invention;
fig. 4 is a comparison graph of the application effect of the present invention and the prior art, which shows the original image of a color image, a damaged image and an image repaired by each method, and the repaired image has a relative error of recovery attached.
[ detailed description ] embodiments
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments, and are not intended to limit the scope of the present disclosure. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Various structural schematics according to the disclosed embodiments of the invention are shown in the drawings. The figures are not drawn to scale, wherein certain details are exaggerated and possibly omitted for clarity of presentation. The shapes of various regions, layers and their relative sizes and positional relationships shown in the drawings are merely exemplary, and deviations may occur in practice due to manufacturing tolerances or technical limitations, and a person skilled in the art may additionally design regions/layers having different shapes, sizes, relative positions, according to actual needs.
In the context of the present disclosure, when a layer/element is referred to as being "on" another layer/element, it can be directly on the other layer/element or intervening layers/elements may be present. In addition, if a layer/element is "on" another layer/element in one orientation, then that layer/element may be "under" the other layer/element when the orientation is reversed.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The invention is described in further detail below with reference to the accompanying drawings:
the prior art cannot utilize the relation between known and unknown pixels to the maximum extent, various generating modes are generally used when defining low rank, the relevance among the pixels is greatly damaged, and the recovered image has certain distortion in visual effect. Therefore, we propose tensor eigen transformation to overcome the defect, as shown in fig. 1, in an embodiment of the present invention, a color image restoration method based on tensor eigen transformation does not need to define the low rank property of the original tensor by means of a tensor matrix-generating approach, directly obtains the low rank property of the original tensor by using the inherent structural features of the tensor eigen transformation, greatly utilizes and stores the inherent relevance between the color image pixels, overcomes the defect of insufficient storage of the pixel relevance in the prior art, and makes the restored color image more accurate and has greater generalization.
Referring to fig. 1, the color image restoration method based on tensor eigen transformation of the present invention includes the following steps:
step 1, obtaining damaged image (i.e. to-be-compensated tensor)) With a pixel of 256, i.e. corresponding to the dimension of the tensor to be compensatedDetermining a set of all missing pixels corresponding to the damaged regionSet suitablyPresetting parameters epsilon, β, rcWherein, the edge rank of the tensor eigen-transform is always set to re1 is ═ 1; setting a remodeling parameter ρ1,ρ2In the embodiment, the parameters are set to be epsilon 0.3, β is 100, and r isc20; remodeling parameter ρ1=4,ρ24, the maximum iteration number is 500, and the error requirement is less than 1 × 10-3The missing pixel ratio is set to 70%, i.e. 70% of the color image pixels are missing or damaged;
as shown in FIG. 2, to identify the pixel positions to be repaired, a set of all missing pixels corresponding to the damaged area is first determinedThe concrete setting is as follows:
step 1.1, reading all pixel points of an image to be restored, enabling all the points with pixel values not being 0 to be known pixel points, and recording the positions of the known pixel points as a set omega;
step 1.2, all the points with the pixel values of 0 are made to be unknown pixel points, and the position set of the unknown pixel points is recorded to be
Step 2, obtaining the full tensor to be compensated by utilizing the tensor eigen transformation tauThe corresponding eigen-matrix E, denoted
As shown in FIG. 3, the method uses tensor eigen transformation tau to obtain the tensor to be compensatedThe corresponding intrinsic matrix E specifically comprises the following steps:
step 2.1, using improved tensor column decomposition to complete tensor to be compensatedDecomposed into tensor column kernelsThe concrete expression is as follows:
in the above formula, the first and second carbon atoms are,representing the coordinate in the tensor to be complemented as (i)c,jc,kc) The value of the pixel of (a) is,indicating the amount of tension to be compensatedThe tensor of the reshaped intermediate medium,a b-th Frontal Slice matrix (Frontal Slice) representing an a-th Tensor column Core (Tensor Train Core);
specifically, the improved tensor column decomposition described in the tensor eigentransform has the following steps:
step 2.1.1, in order to ensure the reconstruction precision, firstly, the tensor corresponding to the color imageRemodel intoWherein the setting parameter p is utilized1,ρ2The remodeling relationship of (A) is that,
step 2.1.2, introduce the auxiliary temporary tensorOrder toWill tensorReshaped into a corresponding matrix
Step 2.1.3, remodeling the matrix obtained in the step 2.1.2The matrix singular value decomposition can be specifically expressed as:
C=U·S·VT(2)
step 2.1.4, get matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcRow and rcColumns forming a new matrix
Step 2.1.6, matrixRemolding into a new matrixAnd is obtained according to the singular value decomposition of the matrix, i.e. the decomposition in equation (2)Get matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcRow and rcColumns forming a new matrix
Step 2.1.7, matrixRemodeled to a second tensor column coreDetermining updated matricesC is to bedRemodeled to a third tensor column core
Step 2.2, carrying out tensor singular value decomposition on one tensor column core in the step, wherein the tensor singular value decomposition is specifically represented as follows:
in the above equation, a tensor column kernel is decomposed into the form of the product of three tensors, whereCalled f-diagonal Tensor (f-diagonal Tensor), takes initial value a as 1;
the tensor singular value decomposition in the tensor eigen transformation specifically comprises the following steps:
step 2.2.1, get a quantitative corePerforming three-dimensional Fourier transform to obtain tensor column core in corresponding Fourier domainTaking an initial value s as 1;
step 2.2.2, sheet takingCore of the quantum arrayAnd recording the matrix as the s-th front section matrixNamely haveTo pairPerforming singular value decomposition of the matrix to obtain left and right singular matrices thereofAnd diagonal matrixNamely, it is
Step 2.2.3 creating tensorsSetting the initial value of the newly-built tensor to be 0, and setting the left and right singular matrixes in the previous stepAnd diagonal matrixThe s-th front tangent plane matrix given to the newly created tensor, i.e.
Step 2.2.4, if s<I3If s is equal to s +1, the procedure returns to step 2.2.2 to enter a loop, otherwise, the three tensors are determined
Step 2.2.5, forRespectively carrying out three-dimensional inverse Fourier transform to obtain tensors of real number domainTo sum up, a tensor column core is obtainedSingular value decomposition of, i.e. having
Step 2.3, in the Fourier domain, the tensor column in the previous step is centeredCorresponding f-diagonal tensorThe values in (1) are mapped into a matrix E according to coordinates, and the corresponding coordinate relationship can be expressed as:
in the above formula, the first and second carbon atoms are,representing tensor column coreF-diagonal tensor in Fourier domain, ζ denotes tensor column core pointer, ξ denotes eigenmatrix pointer, IaRepresenting the tensor of the intermediate medium to be complementedThe a-th dimension; the specific value of the tensor column core pointer zeta is that when a is equal toζ ═ 1 when 1 or a ═ 3, and ζ ═ i in other cases; the specific value of the eigen matrix pointer is
In the above formula, reAnd rcRespectively representing the edge rank and the center rank in the tensor eigen transformation, wherein the edge rank of the method is always set as re=1;
Step 2.4, if a<3, taking the intermediate tensor to be compensatedA +1 th tensor column core ofI.e. let a be a +1, go back to step 2.3 to enter iteration, otherwise determine tensorThe eigenmatrix of (a) is E.
Step 3, using a threshold operator Dε,βUpdating the intrinsic matrix E in the step 2, and recording the updated intrinsic matrix as EnI.e. En=Dε,β(E);
The threshold operator D described in this stepε,βThe method specifically comprises the following steps:
wherein sign (x) is a sign function, parameter c0,c1,c2Needs to be determined as
ε, β is the preset parameter input in advance, defining the symbol Dε,β(X) represents the thresholding operation on all elements in matrix X.
In the step 4, the step of,introducing an auxiliary tensorUsing inverse tensor eigentransformations tau-1Determining an updated eigen matrix EnCorresponding tensorRecord as
Inverse tensor eigentransform τ in this step-1Is the inverse process of the tensor eigen-transform tau, it is emphasized that the inverse tensor eigen-transform tau-1Subject to τ, otherwise τ-1Will become meaningless; inverse tensor eigentransform τ-1The method comprises the following specific steps:
step 4.1, in the Fourier domain, referring to the tensor eigen transformation method in the formulas (4) and (5), calculating the corresponding tensor column cores by the eigen matrix EF-diagonal tensor ofTaking an initial value a as 1;
step 4.2, mixingIs obtained by inverse Fourier transformFrom the f-diagonal tensor in the real domainAnd corresponding left and right singular tensorsCalculating to obtain tensor column coreThe expression is as follows:
the symbol "+" in the above formula represents the tensor product;
step 4.3, if a<3, the a +1 f-diagonal tensor is takenI.e. let a be a +1, go back to step 4.2 to enter iteration, otherwise go to the next step.
Step 4.4, according to the obtained plurality of tensor column coresCalculating to obtain the updating tensor corresponding to the eigen matrix EThe concrete expression is as follows:
subscript i, j, k in the above formula represents the tensor of the pixel pointThe position of (1); in addition, the tensor will be updatedReshaped into an original imageTensor of equal size
Step 5, determining the update tensorIf the update tensor pixel exists in the pixel damaged areaOrder toIf the update tensor pixel is not in the pixel damage areaOrder to
Step 6, judging relative errorWhether the set error requirement is met or not, or whether the iteration frequency reaches the maximum iteration requirement or not; if the error requirement or the maximum iteration requirement is met, outputting the latest tensor(i.e., the repaired image), otherwise letAnd returning to the step 2 to enter an iterative loop.
As shown in fig. 4, the present embodiment recovers the color image by a tensor eigen transformation-based color image recovery method, and the recovery effect of the recovered image is still better even under an extremely high damage rate, i.e., 70% of the pixels are damaged, and compared with the original image, an accurate recovery result is obtained. For comparison, the low-rank tensor total variation method and the tensor nuclear norm method are respectively set according to default algorithms; the relative error of the recovery of the method of the invention is 10.2%, which is better than 12.6% and 12.8% of the prior art, and the recovery result of the prior art is visually seen to have errors in brightness and detail. The experimental results show that the method provided by the invention has better performance no matter in objective index evaluation or subjective visual effect.
In the embodiment, the low rank property is defined without any method for expanding the tensor into the matrix, but the proposed tensor eigen transformation is directly utilized to form the framework, so that the low rank property of the tensor can be iteratively obtained under the framework, and meanwhile, the relevance among pixels is ensured and an effective recovery image is obtained.
With reference to this embodiment, an algorithm of a color image restoration method based on tensor eigen transformation is given as follows:
inputting: spoiled colour imagesSuitable preset parameters ε, β, rcWherein r ise1 is ═ 1; setting a remodeling parameter ρ1,ρ2Determining a set error requirement tol and a maximum iteration requirement K;
step 1, initializing a damaged color imageInitializing iteration parameter i as 1, initializing relative error
Step 2, judging whether the maximum iteration requirement or the specified error requirement is met, if i is less than or equal to K or errori>tol proceeds to the next step, otherwise letAnd terminate the algorithm;
step 4, updating the intrinsic matrix E through formulas (6) and (7) to obtain En;
Step 5, obtaining an updated eigen matrix E based on the inverse eigen transformation of the tensornCorresponding tensor of
And step 6, updating the damaged color image,obtaining the color image restored in this stepCalculating the relative error of the iteration
Step 7, enabling the iteration parameter i to be i + 1; update the damaged color imageAnd returning to the step 2;
The parameters and related mathematical symbols involved in the algorithm are the same as those in the present invention, so that the repeated definition of the parameters and the mathematical symbols is avoided.
The color image restoration method based on tensor eigen transformation described in the embodiment is characterized in that firstly, a damaged color image is read, a set of known pixels and an unknown pixel is identified and defined, and a premise is created for restoring the image by utilizing internal relevance between the known pixels and the unknown pixels; in addition, based on the proposed tensor eigen transformation, the corresponding tensor low-rank property is mined directly according to the structure of the color image, the method greatly preserves the pixel relevance, and under the framework of the tensor eigen transformation, the damaged pixels can be iteratively repaired through simple transformation, so that the missing pixels can be more accurately recovered.
In this embodiment, a computer device is provided, which may be a terminal. The computing device generally includes a processor, a memory, a network interface, and an input-output device. The processor of the computer device provides computational support for the method of the invention; the memorizer provides a built-in operating system, a running environment of the program and a stored computer program for the method; the network interface provides network connection and communication exchange functions for the method; the input device is used for inputting an image to be restored, and can be a keyboard, a mouse or the like specifically; the output device is used for presenting the repaired image, and may specifically be a display screen or the like. More specifically, the software platform of the present embodiment uses MATLAB R2015 a; the hardware platform uses INTEL CORE CPU and memory 4 GB; the experimental method is the method, the low-rank tensor total variation method and the tensor nuclear norm method.
The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (7)
1. A color image restoration method based on tensor eigen transformation is characterized by comprising the following steps:
step 1, obtaining damaged image, namely to-be-compensated tensorDetermining a set omega of all missing pixels corresponding to the damaged region⊥Setting preset parameters epsilon, β, re,rcSetting the remodeling parameter ρ1,ρ2Determining a set error requirement and a maximum iteration requirement;
step 2, obtaining the full tensor to be compensated by utilizing the tensor eigen transformation tauThe corresponding eigen-matrix E, noted:
step 3, using a threshold operator Dε,βUpdating the intrinsic matrix E in the step 2, and recording the updated intrinsic matrix as En:
En=Dε,β(E)
Step 4, introducing auxiliary tensorUsing inverse tensor eigentransformations tau-1Determining an updated eigen matrix EnCorresponding tensorRecording as follows:
step 5, determining the update tensorIf the update tensor pixel exists in the pixel damaged area omega⊥Let us orderIf the update tensor pixel is not in the pixel damaged area omega⊥Let us order
Step 6, judging relative errorWhether the set error requirement is met or not, or whether the iteration frequency reaches the maximum iteration requirement or not; if the error requirement or the maximum iteration requirement is met, outputting the latest tensorNamely the repaired image; otherwise makeAnd returning to the step 2 to enter an iterative loop.
2. The tensor eigen transformation-based color image restoration method as recited in claim 1, wherein in the step 1, a set Ω of all missing pixels corresponding to a damaged area is determined⊥The specific method comprises the following steps:
step 1.1, reading all pixel points of an image to be restored, enabling all the points with pixel values not being 0 to be known pixel points, and recording the positions of the known pixel points as a set omega;
step 1.2, all the points with the pixel values of 0 are made to be unknown pixel points, and the position set of the unknown pixel points is recorded to be omega⊥。
3. The method for restoring a color image based on tensor eigen transformation as claimed in claim 1, wherein in the step 2, the tensor to be compensated is obtained by using tensor eigentransformation τThe specific method of the corresponding eigen matrix E is as follows:
step 2.1, using improved tensor column decomposition to complete tensor to be compensatedDecomposed into tensor column kernelsThe concrete expression is as follows:
in the above formula, the first and second carbon atoms are,representing the full sheet to be supplementedThe coordinate in the quantity is (i)c,jc,kc) The value of the pixel of (a) is,indicating the amount of tension to be compensatedThe tensor of the reshaped intermediate medium,a b-th front tangent matrix representing an a-th tensor column kernel;
step 2.2, carrying out tensor singular value decomposition on one tensor column core in the step:
in the above equation, a tensor column kernel is decomposed into the form of the product of three tensors, whereCalled f-diagonal tensor, taking the initial value a as 1;
step 2.3, in the Fourier domain, the tensor column in the previous step is centeredCorresponding f-diagonal tensorThe values in (1) are mapped into a matrix E according to coordinates, and the corresponding coordinate relationship is expressed as:
in the above formula, the first and second carbon atoms are,representing tensor column coreF-diagonal tensor in Fourier domain, ζ denotes tensor column core pointer, ξ denotes eigenmatrix pointer, IaRepresenting the tensor of the intermediate medium to be complementedThe a-th dimension; the tensor column core pointer ζ has a specific value of ζ being 1 when a is 1 or 3, and ζ being i in other cases; the specific values of the eigen matrix pointers are:
in the above formula, reAnd rcRespectively representing the edge rank and the center rank in the tensor eigen transformation, wherein the edge rank is always set as re=1;
4. The tensor eigenconversion-based color image restoration method as recited in claim 3, wherein the specific method of the improved tensor column decomposition is as follows:
step 2.1.1, first, the tensor corresponding to the color imageRemodel intoWherein the setting parameter p is utilized1,ρ2The remodeling relationship of (A) is that,
step 2.1.2, introduce the auxiliary temporary tensorOrder toWill tensorReshaped into a corresponding matrix
Step 2.1.3, remodeling the matrix obtained in the step 2.1.2Decomposition according to matrix singular values:
C=U·S·VT(5)
wherein the content of the first and second substances,respectively called left and right singular matrices of the matrix C,is a diagonal matrix;
step 2.1.4, get matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcRow and rcColumns forming a new matrix
Step 2.1.6, matrixRemolding into a new matrixAnd is obtained according to the singular value decomposition of the matrix, i.e. the decomposition in equation (5)Get matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcColumn forming a new matrixGet matrixFront r ofcRow and rcColumns forming a new matrix
5. The tensor eigen transformation-based color image restoration method according to claim 3 or 4, wherein in the step 3, the specific method of tensor singular value decomposition is as follows:
step 2.2.1, get a quantitative corePerforming three-dimensional Fourier transform to obtain tensor column core in corresponding Fourier domainTaking an initial value s as 1;
step 2.2.2, get tensor column coreAnd recording the matrix as the s-th front section matrixNamely haveTo pairPerforming singular value decomposition of the matrix to obtain left and right singular matrices thereofAnd diagonal matrixNamely, it is
Step 2.2.3 creating tensorsSetting the initial value of the newly-built tensor to be 0, and setting the left and right singular matrixes in the previous stepAnd diagonal matrixThe s-th front tangent plane matrix given to the newly created tensor, i.e.
Step 2.2.4, if s<I3If s is equal to s +1, the procedure returns to step 2.2.2 to enter a loop, otherwise, the three tensors are determined
6. The tensor eigenconversion-based color image restoration method as recited in claim 1, wherein the threshold operator D in the step 3 is set toε,βThe concrete expression is as follows:
wherein sign (x) is a sign function, parameter c0,c1,c2Comprises the following steps:
wherein ε, β is the preset parameter input in advance, defining the symbol Dε,β(X) represents the thresholding operation on all elements in matrix X.
7. The tensor eigenconversion-based color image restoration method as recited in claim 1, wherein in the step 4, the inverse tensor eigenconversion τ-1The specific method comprises the following steps:
step 4.1, in the Fourier domain, a tensor eigen transformation method is adopted, and a plurality of corresponding tensor column cores are obtained through calculation of the eigen matrix EF-diagonal tensor ofTaking an initial value a as 1;
step 4.2, mixingIs obtained by inverse Fourier transformFrom the f-diagonal tensor in the real domainAnd corresponding left and right singular tensorsCalculating to obtain tensor column core
In the above formula, the symbol "+" represents the tensor product;
step 4.3, if a<3, the a +1 f-diagonal tensor is takenIf a is equal to a +1, returning to the step 4.2 to enter iteration, otherwise, entering the next step;
step 4.4, according to the obtained plurality of tensor column coresCalculating to obtain the updating tensor corresponding to the eigen matrix E
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010144793.XA CN111325697B (en) | 2020-03-04 | 2020-03-04 | Color image restoration method based on tensor eigen transformation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010144793.XA CN111325697B (en) | 2020-03-04 | 2020-03-04 | Color image restoration method based on tensor eigen transformation |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111325697A true CN111325697A (en) | 2020-06-23 |
CN111325697B CN111325697B (en) | 2022-10-25 |
Family
ID=71173085
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010144793.XA Active CN111325697B (en) | 2020-03-04 | 2020-03-04 | Color image restoration method based on tensor eigen transformation |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111325697B (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112241938A (en) * | 2020-08-21 | 2021-01-19 | 浙江工业大学 | Image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation |
CN112381725A (en) * | 2020-10-16 | 2021-02-19 | 广东工业大学 | Image restoration method and device based on deep convolution countermeasure generation network |
CN114119426A (en) * | 2022-01-26 | 2022-03-01 | 之江实验室 | Image reconstruction method and device by non-local low-rank conversion domain and full-connection tensor decomposition |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120269431A1 (en) * | 2003-08-29 | 2012-10-25 | Greg Christie | Methods and appartuses for restoring color and enhancing electronic images |
US20150310592A1 (en) * | 2014-04-25 | 2015-10-29 | Canon Kabushiki Kaisha | Image processing apparatus that performs image restoration processing and image processing method |
CN107016649A (en) * | 2017-02-24 | 2017-08-04 | 同济大学 | A kind of vision data complementing method estimated based on local low-rank tensor |
CN107392107A (en) * | 2017-06-24 | 2017-11-24 | 天津大学 | A kind of face feature extraction method based on isomery tensor resolution |
CN109191404A (en) * | 2018-09-07 | 2019-01-11 | 西安交通大学 | A kind of high spectrum image restorative procedure based on E-3DTV canonical |
CN109241491A (en) * | 2018-07-28 | 2019-01-18 | 天津大学 | The structural missing fill method of tensor based on joint low-rank and rarefaction representation |
CN109886884A (en) * | 2019-01-21 | 2019-06-14 | 长沙理工大学 | A kind of vision data complementing method based on the low-rank tensor estimation for limiting nuclear norm |
CN109978783A (en) * | 2019-03-19 | 2019-07-05 | 上海交通大学 | A kind of color image restorative procedure |
CN110298798A (en) * | 2019-06-20 | 2019-10-01 | 浙江工业大学 | A kind of image repair method based on the completion of low-rank tensor Yu discrete full variation |
-
2020
- 2020-03-04 CN CN202010144793.XA patent/CN111325697B/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120269431A1 (en) * | 2003-08-29 | 2012-10-25 | Greg Christie | Methods and appartuses for restoring color and enhancing electronic images |
US20150310592A1 (en) * | 2014-04-25 | 2015-10-29 | Canon Kabushiki Kaisha | Image processing apparatus that performs image restoration processing and image processing method |
CN107016649A (en) * | 2017-02-24 | 2017-08-04 | 同济大学 | A kind of vision data complementing method estimated based on local low-rank tensor |
CN107392107A (en) * | 2017-06-24 | 2017-11-24 | 天津大学 | A kind of face feature extraction method based on isomery tensor resolution |
CN109241491A (en) * | 2018-07-28 | 2019-01-18 | 天津大学 | The structural missing fill method of tensor based on joint low-rank and rarefaction representation |
CN109191404A (en) * | 2018-09-07 | 2019-01-11 | 西安交通大学 | A kind of high spectrum image restorative procedure based on E-3DTV canonical |
CN109886884A (en) * | 2019-01-21 | 2019-06-14 | 长沙理工大学 | A kind of vision data complementing method based on the low-rank tensor estimation for limiting nuclear norm |
CN109978783A (en) * | 2019-03-19 | 2019-07-05 | 上海交通大学 | A kind of color image restorative procedure |
CN110298798A (en) * | 2019-06-20 | 2019-10-01 | 浙江工业大学 | A kind of image repair method based on the completion of low-rank tensor Yu discrete full variation |
Non-Patent Citations (4)
Title |
---|
HAN LIU ET AL: "Adaptive Rank Estimation Based Tensor Factorization Algorithm for Low-Rank Tensor Completion", 《 2019 CHINESE CONTROL CONFERENCE (CCC)》 * |
YONG CHEN ET AL: "Weighted Group Sparsity Regularized Low-Rank Tensor Decomposition for Hyperspectral Image Restoration", 《IGARSS 2019 - 2019 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM》 * |
刘奎等: "基于结构张量的图像修复方法", 《计算机应用》 * |
刘静等: "基于自动秩估计的黎曼优化矩阵补全算法及其在图像补全中的应用", 《电子与信息学报》 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112241938A (en) * | 2020-08-21 | 2021-01-19 | 浙江工业大学 | Image restoration method based on smooth Tak decomposition and high-order tensor Hank transformation |
CN112241938B (en) * | 2020-08-21 | 2024-02-13 | 浙江工业大学 | Image restoration method based on smooth Tak decomposition and high-order tensor Hakking |
CN112381725A (en) * | 2020-10-16 | 2021-02-19 | 广东工业大学 | Image restoration method and device based on deep convolution countermeasure generation network |
CN112381725B (en) * | 2020-10-16 | 2024-02-02 | 广东工业大学 | Image restoration method and device based on depth convolution countermeasure generation network |
CN114119426A (en) * | 2022-01-26 | 2022-03-01 | 之江实验室 | Image reconstruction method and device by non-local low-rank conversion domain and full-connection tensor decomposition |
Also Published As
Publication number | Publication date |
---|---|
CN111325697B (en) | 2022-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Zhang et al. | Nonlocal low-rank tensor completion for visual data | |
Webster et al. | Detecting overfitting of deep generative networks via latent recovery | |
CN110298798B (en) | Image restoration method based on low-rank tensor completion and discrete total variation | |
CN111325697B (en) | Color image restoration method based on tensor eigen transformation | |
CN112528969B (en) | Face image authenticity detection method and system, computer equipment and storage medium | |
Cao et al. | New architecture of deep recursive convolution networks for super-resolution | |
CN113240596A (en) | Color video recovery method and system based on high-order tensor singular value decomposition | |
CN112184547B (en) | Super resolution method of infrared image and computer readable storage medium | |
Muhammad et al. | Multi-scale Xception based depthwise separable convolution for single image super-resolution | |
CN110751599B (en) | Visual tensor data completion method based on truncated nuclear norm | |
CN110163095B (en) | Loop detection method, loop detection device and terminal equipment | |
Shao et al. | Generative image inpainting with salient prior and relative total variation | |
Xu et al. | Image compressive sensing recovery via group residual based nonlocal low-rank regularization | |
Li et al. | A mixed noise removal algorithm based on multi-fidelity modeling with nonsmooth and nonconvex regularization | |
Wei et al. | 3D face image inpainting with generative adversarial nets | |
Yang et al. | Face inpainting via learnable structure knowledge of fusion network | |
Meng et al. | Siamese CNN-based rank learning for quality assessment of inpainted images | |
Jing et al. | Single image super-resolution via low-rank tensor representation and hierarchical dictionary learning | |
Ye et al. | A sparsity-promoting image decomposition model for depth recovery | |
CN113763313A (en) | Text image quality detection method, device, medium and electronic equipment | |
CN112346126A (en) | Low-order fault identification method, device, equipment and readable storage medium | |
Navasardyan et al. | The family of onion convolutions for image inpainting | |
Pan et al. | “Sparse+ Low-Rank” tensor completion approach for recovering images and videos | |
Ge et al. | A novel image reconstruction algorithm based on texture aware multiscale GAN for veneer defects | |
Yang et al. | Blind image quality assessment via probabilistic latent semantic analysis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |