CN107133930A - Ranks missing image fill method with rarefaction representation is rebuild based on low-rank matrix - Google Patents

Abstract

The invention belongs to the field of computer vision and aims to realize accurate filling of missing images in pixel rows and columns. The technical scheme adopted by the present invention is a method for filling missing images in rows and columns based on low-rank matrix reconstruction and sparse representation. Each column of can be sparsely represented by a column dictionary, and each row of a column-missing image can be sparsely represented by a row dictionary, so a separable two-dimensional sparse prior is introduced based on the sparse representation theory; thus based on the above joint low-rank and separable binary Dimensional sparse prior, the problem of image filling with missing rows and columns is specifically expressed as solving constrained optimization equations, so as to realize image filling with missing rows and columns. The invention is mainly applied to computer vision processing occasions.

Description

Image filling method with missing rows and columns based on low-rank matrix reconstruction and sparse representation

technical field

The invention belongs to the field of computer vision. In particular, it concerns row and column missing image filling methods based on low-rank matrix reconstruction and sparse representation.

Background technique

The problem of recovering an unknown complete matrix from a part of the known pixels of the matrix has attracted great attention in recent years. Such problems are frequently encountered in many application domains of computer vision and machine learning, such as image inpainting, recommender systems, and background modeling.

There have been many research results on methods to solve the image filling problem. Due to the ill-conditioned nature of the matrix filling problem, current matrix filling methods generally consider the latent matrix to be low-rank or approximately low-rank, and then fill in the missing pixel values through low-rank matrix reconstruction. Such as singular value threshold method (SVT), augmented Lagrangian multiplier method (ALM), accelerated neighbor gradient method (APG) and so on. However, these existing filling algorithms use the low-rank characteristics of the image to fill the missing pixel values, which is effective for the case where pixels are randomly missing and each row and column of the image has observation values, but when there are entire rows in the image Existing algorithms cannot solve this kind of image filling problem when the pixel and the entire column are missing. Because the matrix filling problem with a large number of missing pixels in rows and columns cannot be solved under the condition of only using the low-rank property as a constraint. However, in practical applications, such as image transmission, seismic data acquisition, etc., the image matrix is likely to be degraded by missing rows and columns. Therefore, it is very necessary to design a filling algorithm that can effectively fill the missing rows and columns of the matrix.

At this stage, in view of the shortcomings of the above-mentioned matrix filling method that only uses low-rank characteristics, the academic community has introduced sparse constraints on column vectors on this basis to realize the recovery of missing image rows. However, due to the lack of prior conditions, the problem of missing both rows and columns of the matrix has not yet been solved. For this reason, the present invention introduces a low-rank and separable two-dimensional sparse prior into the model so as to realize accurate filling of the matrix with missing rows and columns.

Contents of the invention

The present invention aims to make up for the deficiency of the prior art, that is, to realize accurate filling of images with missing rows and columns of pixels. The technical scheme adopted by the present invention is a method for filling missing images in rows and columns based on low-rank matrix reconstruction and sparse representation. Each column of can be sparsely represented by a column dictionary, and each row of a column-missing image can be sparsely represented by a row dictionary, so a separable two-dimensional sparse prior is introduced based on the sparse representation theory; thus based on the above joint low-rank and separable binary Dimensional sparse prior, the problem of image filling with missing rows and columns is specifically expressed as solving constrained optimization equations, so as to realize image filling with missing rows and columns.

The image filling problem with missing rows and columns is specifically expressed as solving the constrained optimization equation, and the specific steps are refined as follows:

1) The image filling problem with missing rows and columns is specifically expressed as solving the following constrained optimization equation:

where tr( ) is the trace of the matrix, representing the low-rank prior term; Indicates the dot product operation of two matrices, ||·|| ₁ represents the one-norm of the matrix, and the two one-norm items represent separable two-dimensional sparse prior items respectively; Ω is the observation space, indicating that there are rows and columns missing Observe the known pixels in the matrix D, P _Ω (·) is the projection operator, which represents the value of the variable projected into the space domain Ω, A is the filled matrix, Σ=diag([σ ₁ ,σ ₂ ,.. .,σ _n ]) represents the diagonal matrix composed of the singular values of A in non-increasing order, W _a , W _b and W _c represent the weight matrix of weighted low-rank items and separable two-dimensional sparse items, respectively, γ _B , γ _C represent the regularization coefficients of separable two-dimensional sparse items, Φ _c and Φ _r represent the trained column dictionary and row dictionary respectively, the corresponding coefficient matrices are represented by B and C respectively, and E represents the missing of pixels;

The augmented Lagrangian multiplier method (ALM) is used to transform the constrained optimization problem (1) into an unconstrained optimization problem to solve, and the augmented Lagrangian equation is as follows:

Among them, Y ₁ , Y ₂ and Y ₃ represent Lagrange multiplier matrices, μ ₁ , μ ₂ and μ ₃ are penalty factors, <·,·> represent the inner product of two matrices, and ||·|| _F represents the Frobenius norm of the matrix;

The solution process is to train column and row dictionaries Φ _c and Φ _r , initialize weight matrices W _a , W _b and W _c , update coefficient matrices B and C alternately, restore matrix A, missing pixel matrix E, Lagrangian Multiplier matrices Y ₁ , Y ₂ and Y ₃ , penalty factors μ ₁ , μ ₂ and μ ₃ and weight matrices W _a , W _b and W _c , until the algorithm converges, then the iteration result A ^(l) is the original problem The final solution A of .

Specifically, training dictionaries Φ _c and Φ _r : use an online learning algorithm to train out-of-column dictionaries and row dictionaries Φ _c and Φ _r on high-quality image datasets.

Specifically, initialize the weight matrices W _a , W _b and W _c : set the number of reweighting to l, when l=0, the weight matrix with The initial values are all assigned a value of 1, indicating that there is no reweighting for the first iteration.

Specifically, the alternating direction method ADM is used to transform equation (2) into the following sequence for iterative solution:

in the above formula with respectively represent the values of variables B, C, A and E when the objective function takes the minimum value, ρ ₁ , ρ ₂ and ρ ₃ are multiple factors, and k is the number of iterations; then iteratively solve according to the following steps:

1) Solve B ^k+1 : use accelerated neighbor gradient algorithm to get B ^k+1 ;

Remove the terms irrelevant to B in the objective function for solving B in formula (3), and get the following equation:

Using the method of Taylor expansion, construct a second-order function to approximate the above formula, and then solve the original equation for this second-order function, let Introducing the variable Z again, we can finally solve:

Among them, soft(·,·) is the contraction operator, The gradient of , L _f is a constant whose value is The update rule of the variable Z _j is as follows:

Among them, t _j is a set of constant sequences, and j is the number of variable iterations;

2) Solve C ^k+1 : use accelerated neighbor gradient algorithm to find C ^k+1 ;

Remove the terms irrelevant to C in the objective function for solving C in formula (3), and get the following equation:

Using the method of Taylor expansion, construct a second-order function to approximate the above formula, and then solve the original equation for this second-order function, let reintroduce variable Finally it can be solved:

Among them, soft(·,·) is the contraction operator, for The gradient of , L _f is a constant whose value is variable The update rules for are as follows:

Among them, t _j is a set of constant sequences, and j is the number of variable iterations;

3) Solve A ^k+1 : use Singular Value Thresholding (SVT) to solve A ^k+1 ;

Remove the items irrelevant to A in the objective function for solving A in formula (3), and obtain through the formula:

in, Using the singular value threshold method for Q ^k+1 to solve:

Among them, H ^k+1 and V ^k+1 are the left singular matrix and right singular matrix of Q ^k+ 1 respectively;

4) Solve E ^k+1 : the solution of E ^k+1 consists of two parts;

In the observation space Ω, the value of E is 0; outside the observation space Ω, that is, the complementary space Inside, use the first-order derivation to solve, and combine the two parts to get the final solution of E:

5) Repeat the above steps 1), 2), 3) and 4) until the algorithm converges, then the iteration results A ^k+1 , Σ ^k+1 , B ^k+1 , C ^k+1 and E ^k+1 are The original problem has no reweighted results A ^(l) , Σ ^(l) , B ^(l) , C ^(l) and E ^(l) , where l is the number of reweighted times;

6) Updating weight matrices W _a , W _b and W _c ;

In order to offset the influence of the signal amplitude on the nuclear norm item and the one norm item, a reweighting scheme is introduced, and the inverse proportional principle is adopted according to the amplitudes of the currently estimated singular value matrix Σ ^(l) and coefficient matrices B ^l and C ^l Iteratively update the weight matrices W _a , W _b and W _c :

in is the position coordinate of the pixel in the image, and ε is an arbitrarily small positive number.

7) Repeat the above steps 1)-7) until the algorithm converges, then the iteration result A ^(l) is the final solution A of the original problem.

Technical characteristics and effects of the present invention:

The method of the invention aims at the problem of image filling with missing row and column, and realizes the solution to the problem of image filling with missing row and column by introducing a separable two-dimensional sparse prior. The present invention has the following characteristics:

1. Algorithms such as Augmented Lagrangian Multiplier Method (ALM), Alternating Direction Method (ADM), Accelerated Neighbor Gradient Algorithm, and Singular Value Threshold Method are used to solve sub-problems, and the advantages of existing algorithms are integrated.

2. Use the column dictionary and row dictionary to sparsely represent the columns and rows of the image, which is more efficient than the traditional block dictionary.

3. Combining low-rank matrix reconstruction theory with sparse representation theory, dictionary learning is introduced into the traditional low-rank matrix reconstruction model, and a joint low-rank information and separable two-dimensional sparsity prior is proposed, so that the row and column can be simultaneously Missing images are accurately filled.

4. Through the low-rank and sparse joint constraints on the missing damaged graph, the filling performance is improved, which can fill the missing row and column, and can also fill the random missing more accurately.

Description of drawings

The above-mentioned advantages of the present invention will become obvious and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

Fig. 1 is a flowchart of the present invention;

Figure 2 is the original ground-truth image without missing;

Figure 3 is a damaged image with rows and columns and random deletions, black represents missing pixels, and the total missing rates from left to right are: (1) 10% missing; (2) 20% missing; (3) 30% missing; ( 4) 50% missing;

Fig. 4 is the filling result figure of the missing figure under four kinds of missing rates with the method of the present invention: (1) 10% missing filling result, PSNR=40.79; (2) 20% missing filling result, PSNR=37.32; (3) 30% missing filling result, PSNR=35.69; (4) 50% missing filling result, PSNR=32.23.

detailed description

The method for filling missing rows and columns of images based on low-rank matrix reconstruction and sparse representation of the present invention will be described in detail below in conjunction with the embodiments and drawings.

The present invention combines low-rank matrix reconstruction with sparse representation, introduces a dictionary learning model on the basis of traditional low-rank matrix reconstruction models, and adopts joint low-rank and separable two-dimensional sparse priori constraints on missing images, Therefore, the problem that existing algorithms cannot realize image filling with missing rows and columns is solved. The specific method includes the following steps:

1) Considering the low-rank characteristics of the natural image itself, a low-rank prior is introduced based on the low-rank matrix reconstruction theory to constrain the latent image; at the same time, considering that each column of a row-missing image can be sparsely represented by a column dictionary, and a column-missing image Each row of can be sparsely represented by a row dictionary, so a separable two-dimensional sparse prior is introduced based on the sparse representation theory; thus, based on the above joint low-rank and separable two-dimensional sparse prior, the image filling problem with missing rows and columns Specifically, it is expressed as solving the following constrained optimization equation:

where tr( ) is the trace of the matrix, representing the low-rank prior term; Indicates the dot product operation of two matrices, ||·|| ₁ represents the one-norm of the matrix, and the two one-norm items represent separable two-dimensional sparse prior items respectively; Ω is the observation space, indicating that there are rows and columns missing Observe the known pixels in the matrix D, P _Ω (·) is the projection operator, which represents the value of the variable projected into the space domain Ω, A is the filled matrix, Σ=diag([σ ₁ ,σ ₂ ,.. .,σ _n ]) represents the diagonal matrix composed of the singular values of A in non-increasing order, W _a , W _b and W _c represent the weight matrix of weighted low-rank items and separable two-dimensional sparse items, respectively, γ _B , γ _C represent the regularization coefficients of separable two-dimensional sparse items, Φ _c and Φ _r represent the trained column dictionary and row dictionary respectively, the corresponding coefficient matrices are represented by B and C respectively, and E represents the missing of pixels;

11) The present invention adopts the augmented Lagrangian multiplier method (ALM) to convert the constrained optimization problem (1) into an unconstrained optimization problem to solve, and the augmented Lagrangian equation is as follows:

Among them, Y ₁ , Y ₂ and Y ₃ represent Lagrange multiplier matrices, μ ₁ , μ ₂ and μ ₃ are penalty factors, <·,·> represent the inner product of two matrices, and ||·|| _F represents the Frobenius norm of the matrix;

12) The solution process is as follows: train the column dictionary and row dictionary Φ _c and Φ _r , initialize the weight matrix W _a , W _b and W _c , update the coefficient matrix B and C alternately, restore the matrix A, the missing pixel matrix E, Lag Langerian multiplier matrices Y ₁ , Y ₂ and Y ₃ , penalty factors μ ₁ , μ ₂ and μ ₃ and weight matrices W _a , W _b and W _c ;

2) Training dictionaries Φ _c and Φ _r : use online learning algorithm to train column and row dictionaries Φ _c and Φ _r on high-quality image data sets;

21) Construct a column dictionary Φ _c so that the matrix A can be sparsely represented by the column dictionary, that is, satisfy A=Φ _c B, where B is a coefficient matrix and is sparse; construct a row dictionary Φ _r so that the transposition of the matrix A can be represented by the row dictionary Sparse representation, which satisfies A ^Τ = Φ _r C, where C is a coefficient matrix and is sparse. The present invention uses an Online Learning algorithm to train a column dictionary and a row dictionary Φ _c and Φ _r on a Kodak image set.

22) The relevant parameters of the training dictionary are set as follows: the number of rows of the matrix A to be reconstructed is equal to the dimension m of elements in the dictionary Φ _c , that is, the number of rows of A and the number of rows of Φ _c are both m; the number of columns of A is equal to The dimension n of the elements in the dictionary Φ _r is equal, that is, the number of columns of A and the number of rows of Φ _r are both n. The trained dictionaries Φ _c and Φ _r are both over-complete dictionaries, that is, the number of columns of the dictionary must be greater than the number of rows.

3) Initialize weight matrices W _a , W _b and W _c ;

Set the number of times of reweighting to l, when l=0, the weight matrix with The initial values are all assigned a value of 1, indicating that there is no reweighting for the first iteration.

4) Using Alternating Direction Method (ADM) to convert Equation (2) into the following sequence for iterative solution:

in the above formula with Represent the values of variables B, C, A and E when the objective function takes the minimum value, ρ ₁ , ρ ₂ and ρ ₃ are multiplication factors, k is the number of iterations; set the initial value of each parameter, and then follow step 5 ), 6), 7), and 8) to iteratively solve the results without reweighting.

5) Solve B ^k+1 : use the accelerated nearest neighbor gradient algorithm to find B ^k+1 .

51) Remove the item irrelevant to B in the objective function for solving B in the formula (3), and obtain the following equation:

Through Taylor expansion, a second-order function is constructed to approximate the above formula, and then the original equation is solved for this second-order function. make Then introduce the variable Z and define the following function:

in, is the gradient of f(Z), L _f is a constant, the value is It is used to ensure that F(Z)≤Q(B,Z) for all Z.

52) After the transformation in the previous step, the equation (4) is transformed into the problem of solving the minimum value of Q(B, Z _j ), and the formula is obtained as follows:

in, The update rule of the variable Z _j is as follows:

Among them, _tj is a set of constant sequences, and j is the number of variable iterations. Using the contraction operator to solve:

Among them, soft(·,·) is the contraction operator.

6) Solve C ^k+1 : use accelerated neighbor gradient algorithm to find C ^k+1 .

61) Remove the terms irrelevant to C in the objective function for solving C in formula (3), and obtain the following equation:

Using Taylor expansion, a second-order function is constructed to approximate the above formula, and then the original equation is solved for this second-order function. make reintroduce variable Define the following functions:

in, for The gradient of , L _f is a constant whose value is used to ensure that all have

62) After the transformation in the previous step, the equation (9) is transformed into the solution The minimum value problem of is obtained by formula as follows:

in, variable The update rules for are as follows:

Among them, _tj is a set of constant sequences, and j is the number of variable iterations. Using the contraction operator to solve:

Among them, soft(·,·) is the contraction operator.

7) Solve for A ^k+1 : use SVT to solve for A ^k+1 .

Remove the items irrelevant to A in the objective function for solving A in formula (3):

Using the formula method, the above formula can be rewritten as:

in, Using the singular value threshold method for Q ^k+1 to solve:

Among them, H ^k+1 and V ^k+1 are the left singular matrix and right singular matrix of Q ^k+ 1 respectively;

8) Solving E ^k+1 : The solution of E ^k+1 consists of two parts.

81) In the observation space Ω, the value of E is 0; that is, P _Ω (E)=0.

82) Outside the observation space Ω, that is, the complementary space Inside, the equation about E ^k+1 is as follows:

Using the first-order derivative to solve, we get

83) Combining the solutions inside and outside the space domain Ω is the final solution of E:

9) Repeat the above steps 5), 6), 7) and 8) until the algorithm converges, then the iteration results A ^k+1 , Σ ^k+1 , B ^k+1 , C ^k+1 and E ^k+1 are The original problem has no reweighted outcomes A ^(l) , Σ ^(l) , B ^(l) , C ^(l) and E ^(l) . Here, l is the number of reweighting times.

10) Update the weight matrices W _a , W _b and W _c .

In order to offset the influence of the signal amplitude on the nuclear norm item and the one norm item, a reweighting scheme is introduced, and the inverse proportional principle is adopted according to the amplitudes of the currently estimated singular value matrix Σ ^(l) and coefficient matrices B ^l and C ^l Iteratively update the weight matrices W _a , W _b and W _c :

in is the position coordinate of the pixel in the image, and ε is an arbitrarily small positive number.

11) Repeat the above steps 4), 5), 6), 7), 8), 9), and 10) until the algorithm converges, and the iterative result A ^(l) is the final solution A of the original problem.

The method of the present invention combines low-rank matrix reconstruction with sparse representation theory, introduces a dictionary learning model on the basis of traditional low-rank matrix reconstruction models, and adopts joint low-rank and separable two-dimensional sparse prior conditions for missing images Constraints, so as to solve the problem that the existing technology cannot handle, that is, to realize the filling of the image with missing rows and columns (the experimental flow chart is shown in Figure 1). The detailed description in conjunction with accompanying drawing and embodiment is as follows:

1) In the experiment, a 321×481-pixel image randomly selected from the BSDS500 dataset (as shown in Figure 2) was used as the original image, and four missing rates were constructed on it with 10%, 20%, and 30% and 50% of the damaged images (as shown in Figure 3), which contain row and column deletions and random deletions. The present invention uses a dictionary whose atomic size is fixed at 100, so the image to be filled is divided into several image blocks of 100×100 in the manner of sliding windows from top to bottom and from left to right. The sliding window has a step size of 90 pixels. These several 100×100 image blocks are filled in sequence, and finally combined to obtain a filled image with an original size of 321×481. When filling the first image block, it is represented by a matrix D, and the problem of filling the current image block with missing rows and columns is specifically expressed as solving the following constrained optimization equation:

where tr( ) is the trace of the matrix, representing the low-rank prior term; Indicates the dot product operation of two matrices, ||·|| ₁ represents the one-norm of the matrix, and the two one-norm items represent separable two-dimensional sparse prior items respectively; Ω is the observation space, indicating that there are rows and columns missing Observe the known pixels in the matrix D, P _Ω (·) is the projection operator, which represents the value of the variable projected into the space domain Ω, A is the filled matrix, Σ=diag([σ ₁ ,σ ₂ ,.. .,σ _n ]) represents the diagonal matrix composed of the singular values of A in non-increasing order, W _a , W _b and W _c represent the weight matrix of weighted low-rank items and separable two-dimensional sparse items, respectively, γ _B , γ _C represent the regularization coefficients of separable two-dimensional sparse items, Φ _c and Φ _r represent the trained column dictionary and row dictionary respectively, the corresponding coefficient matrices are represented by B and C respectively, and E represents the missing of pixels;

11) The present invention adopts the augmented Lagrangian multiplier method (ALM) to convert the constrained optimization problem (1) into an unconstrained optimization problem to solve, and the augmented Lagrangian equation is as follows:

Among them, Y ₁ , Y ₂ and Y ₃ represent Lagrange multiplier matrices, μ ₁ , μ ₂ and μ ₃ are penalty factors, <·,·> represent the inner product of two matrices, and ||·|| _F represents the Frobenius norm of the matrix;

12) The solution process is as follows: train the column dictionary and row dictionary Φ _c and Φ _r , initialize the weight matrix W _a , W _b and W _c , update the coefficient matrix B and C alternately, restore the matrix A, the missing pixel matrix E, Lag Langerian multiplier matrices Y ₁ , Y ₂ and Y ₃ , penalty factors μ ₁ , μ ₂ and μ ₃ and weight matrices W _a , W _b and W _c ;

2) Training dictionaries Φ _c and Φ _r : use online learning algorithm to train column and row dictionaries Φ _c and Φ _r on high-quality image data sets;

21) Construct a column dictionary Φ _c so that the matrix A can be sparsely represented by the column dictionary, that is, satisfy A=Φ _c B, where B is a coefficient matrix and is sparse; construct a row dictionary Φ _r so that the transposition of the matrix A can be represented by the row dictionary Sparse representation, which satisfies A ^Τ = Φ _r C, where C is a coefficient matrix and is sparse. The present invention uses an Online Learning algorithm to randomly select 230,000 pixel columns with a size of 100×1 on all images in the Kodak image set as training data to train a column dictionary and a row dictionary Φ _c and Φ _r .

22) The relevant parameters of the training dictionary are set as follows: the number of rows of the reconstruction matrix A is equal to the dimension m of elements in the dictionary Φ _c , that is, the number of rows of A and the number of rows of Φ _c are both m, and m=100 is taken in the experiment . The number of columns of A is equal to the dimension n of elements in the dictionary Φ _r , that is, the number of columns of A and the number of rows of Φ _r are both n, and n=100 is taken in the experiment. The trained dictionaries Φ _c and Φ _r are both over-complete dictionaries, that is, the number of columns of the dictionary must be greater than the number of rows. In the experiment, the number of rows and columns in the dictionary is 400, and the specifications of dictionaries Φ _c and Φ _r are both 100×400.

3) Initialize weight matrices W _a , W _b and W _c ;

Set the number of times of reweighting to l, when l=0, the weight matrix with The initial values are all assigned a value of 1, indicating that there is no reweighting for the first iteration.

4) Using Alternating Direction Method (ADM) to convert Equation (2) into the following sequence for iterative solution:

in the above formula with Represent the values of variables B, C, A and E when the objective function takes the minimum value, ρ ₁ , ρ ₂ and ρ ₃ are multiplication factors, k is the number of iterations; set the initial value of each parameter, and then follow step 5 ), 6), 7), and 8) to iteratively solve the results without reweighting. The initial value is set in the experiment: l=0; k=1; ρ ₁ =ρ ₂ =ρ ₃ =1.1; A ¹ =B ¹ =C ¹ =E ¹ =0.

5) Solve B ^k+1 : use the accelerated nearest neighbor gradient algorithm to find B ^k+1 .

51) Remove the item irrelevant to B in the objective function for solving B in the formula (3), and obtain the following equation:

Through Taylor expansion, a second-order function is constructed to approximate the above formula, and then the original equation is solved for this second-order function. make Then introduce the variable Z and define the following function:

in, is the gradient of f(Z), L _f is a constant, the value is It is used to ensure that F(Z)≤Q(B,Z) for all Z.

52) After the transformation in the previous step, the equation (4) is transformed into the problem of solving the minimum value of Q(B, Z _j ), and the formula is obtained as follows:

in, The update rule of the variable Z _j is as follows:

Among them, _tj is a set of constant sequences, and j is the number of variable iterations. After the above conversion, the initial values of each parameter are set as follows: j=1; t ₁ =1; Z ₁ =0. When it converges, it can be solved:

Among them, soft(·,·) is the contraction operator.

6) Solve C ^k+1 : use accelerated neighbor gradient algorithm to find C ^k+1 .

61) Remove the terms irrelevant to C in the objective function for solving C in formula (3), and obtain the following equation:

Using Taylor expansion, a second-order function is constructed to approximate the above formula, and then the original equation is solved for this second-order function. make reintroduce variable Define the following functions:

in, for The gradient of , L _f is a constant whose value is used to ensure that all have

62) After the transformation in the previous step, the equation (9) is transformed into the solution The minimum value problem of is obtained by formula as follows:

in, variable The update rules for are as follows:

Among them, _tj is a set of constant sequences, and j is the number of variable iterations. After the above conversion, the initial values of each parameter are set as follows: j=1;

t ₁ =1; When it converges, it can be solved:

Among them, soft(·,·) is the contraction operator.

7) Solve for A ^k+1 : use SVT to solve for A ^k+1 .

Remove the items irrelevant to A in the objective function for solving A in formula (3):

Using the formula method, the above formula can be rewritten as:

in, Using the singular value threshold method for Q ^k+1 to solve:

Among them, H ^k+1 and V ^k+1 are the left singular matrix and right singular matrix of Q ^k+ 1 respectively;

8) Solving E ^k+1 : The solution of E ^k+1 consists of two parts.

81) In the observation space Ω, the value of E is 0; that is, P _Ω (E)=0.

82) Outside the observation space Ω, that is, the complementary space Inside, the equation about E ^k+1 is as follows:

Using the first-order derivative to solve, we get

83) Combining the solutions inside and outside the space domain Ω is the final solution of E:

9) Repeat the above steps 5), 6), 7) and 8) until the algorithm converges, then the iteration results A ^k+1 , Σ ^k+1 , B ^k+1 , C ^k+1 and E ^k+1 are The original problem has no reweighted outcomes A ^(l) , Σ ^(l) , B ^(l) , C ^(l) and E ^(l) . Here, l is the number of reweighting times.

10) Update the weight matrices W _a , W _b and W _c .

In order to offset the influence of the signal amplitude on the nuclear norm item and the one norm item, a reweighting scheme is introduced, and the inverse proportional principle is adopted according to the amplitudes of the currently estimated singular value matrix Σ ^(l) and coefficient matrices B ^l and C ^l Iteratively update the weight matrices W _a , W _b and W _c :

in is the position coordinate of the pixel in the image, and ε is an arbitrarily small positive number. In the experiment, ε=0.001 was taken.

11) Repeat the above steps 4), 5), 6), 7), 8), 9), and 10) until the algorithm converges, and the iterative result A ^(l) is the final solution A of the original problem.

12) Process the remaining several image blocks obtained in step 1) until they are all filled, and then combine these image blocks into a final filling map (as shown in FIG. 4 ). When combining, the pixel points that are divided and overlapped by multiple fillings take the average value of multiple fillings as the final value.

Experimental result: the present invention adopts PSNR (Peak Signal-to-Noise Ratio) as the measure measure of image filling result, and unit is dB:

Among them, I is the filled image, I ₀ is the real image without missing, w is the width of the image, h is the height of the image, (x, y) represents the pixel value of row x and column y of the image, and Σ represents the sum Operation, |·| is an absolute value. In this experiment, n=8, and the filling results of the 4 pictures with different degrees of missing rows and columns used in the experiment are shown in Figure 4.

Claims (5)

1. A row and column missing image filling method based on low-rank matrix reconstruction and sparse representation, characterized in that the steps are, based on low-rank matrix reconstruction theory, introducing low-rank priors to constrain latent images; meanwhile, considering row-missing images Each column of can be sparsely represented by a column dictionary, and each row of a column-missing image can be sparsely represented by a row dictionary, so a separable two-dimensional sparse prior is introduced based on the sparse representation theory; thus based on the above joint low-rank and separable binary Dimensional sparse prior, the problem of image filling with missing rows and columns is specifically expressed as solving constrained optimization equations, so as to realize image filling with missing rows and columns. 2. The image filling method based on low-rank matrix reconstruction and sparse representation as claimed in claim 1, wherein the image filling problem with missing rows and columns is specifically expressed as solving a constrained optimization equation and the specific steps are refined as : 1) The image filling problem with missing rows and columns is specifically expressed as solving the following constrained optimization equation: where tr( ) is the trace of the matrix, representing the low-rank prior term; Indicates the dot product operation of two matrices, ||·|| ₁ represents the one-norm of the matrix, and the two one-norm items represent separable two-dimensional sparse prior items respectively; Ω is the observation space, indicating that there are rows and columns missing Observe the known pixels in the matrix D, P _Ω (·) is the projection operator, which represents the value of the variable projected into the space domain Ω, A is the filled matrix, Σ=diag([σ ₁ ,σ ₂ ,.. .,σ _n ]) represents the diagonal matrix composed of the singular values of A in non-increasing order, W _a , W _b and W _c represent the weight matrix of weighted low-rank items and separable two-dimensional sparse items, respectively, γ _B , γ _C represent the regularization coefficients of separable two-dimensional sparse items, Φ _c and Φ _r represent the trained column dictionary and row dictionary respectively, the corresponding coefficient matrices are represented by B and C respectively, and E represents the missing of pixels; The augmented Lagrangian multiplier method (ALM) is used to transform the constrained optimization problem (1) into an unconstrained optimization problem to solve, and the augmented Lagrangian equation is as follows: Among them, Y ₁ , Y ₂ and Y ₃ represent Lagrange multiplier matrices, μ ₁ , μ ₂ and μ ₃ are penalty factors, <·,·> represent the inner product of two matrices, and ||·|| _F represents the Frobenius norm of the matrix; The solution process is to train column and row dictionaries Φ _c and Φ _r , initialize weight matrices W _a , W _b and W _c , update coefficient matrices B and C alternately, restore matrix A, missing pixel matrix E, Lagrangian Multiplier matrices Y ₁ , Y ₂ and Y ₃ , penalty factors μ ₁ , μ ₂ and μ ₃ and weight matrices W _a , W _b and W _c , until the algorithm converges, then the iteration result A ^(l) is the original problem The final solution A of . 3. The row and column missing image filling method based on low-rank matrix reconstruction and sparse representation as claimed in claim 2, characterized in that, specifically, training dictionaries Φ _c and Φ _r : use online learning on high-quality image data sets The algorithm trains the column and row dictionaries Φ _c and Φ _r . 4. The row and column missing image filling method based on low-rank matrix reconstruction and sparse representation as claimed in claim 2, characterized in that, specifically, initializing weight matrices W _a , W _b and W _c : the number of times of weighting is set to 1, When l=0, the weight matrix , with The initial values are all assigned a value of 1, indicating that there is no reweighting for the first iteration. 5. the row and column missing image filling method based on low-rank matrix reconstruction and sparse representation as claimed in claim 2, is characterized in that, specifically, adopts alternating direction method ADM to convert equation (2) into following sequence and iteratively solves: <mrow> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <msup> <mi>B</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <mi>arg</mi> <munder> <mi>min</mi> <mi>B</mi> </munder> <mo>{</mo> <msub> <mi>L</mi> <mrow> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> </mrow> </msub> <mrow> <mo>(</mo> <msup> <mi>A</mi> <mi>k</mi> </msup> <mo>,</mo> <mi>B</mi> <mo>,</mo> <msup> <mi>C</mi> <mi>k</mi> </msup> <mo>,</mo> <msup> <mi>E</mi> <mi>k</mi> </msup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>3</mn> <mi>k</mi> </msubsup> <mo>)</mo> </mrow> <mo>}</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msup> <mi>C</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <mi>arg</mi> <munder> <mi>min</mi> <mi>C</mi> </munder> <mo>{</mo> <msub> <mi>L</mi> <mrow> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> </mrow> </msub> <mrow> <mo>(</mo> <msup> <mi>A</mi> <mi>k</mi> </msup> <mo>,</mo> <msup> <mi>B</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>,</mo> <mi>C</mi> <mo>,</mo> <msup> <mi>E</mi> <mi>k</mi> </msup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>3</mn> <mi>k</mi> </msubsup> <mo>)</mo> </mrow> <mo>}</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msup> <mi>A</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <mi>arg</mi> <munder> <mi>min</mi> <mi>A</mi> </munder> <mo>{</mo> <msub> <mi>L</mi> <mrow> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> </mrow> </msub> <mrow> <mo>(</mo> <mi>A</mi> <mo>,</mo> <msup> <mi>B</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>,</mo> <msup> <mi>C</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>,</mo> <msup> <mi>E</mi> <mi>k</mi> </msup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>3</mn> <mi>k</mi> </msubsup> <mo>)</mo> </mrow> <mo>}</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msup> <mi>E</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <mi>arg</mi> <munder> <mi>min</mi> <mrow> <msub> <mi>P</mi> <mi>&amp;Omega;</mi> </msub> <mrow> <mo>(</mo> <mi>E</mi> <mo>)</mo> </mrow> <mo>=</mo> <mn>0</mn> </mrow> </munder> <mo>{</mo> <msub> <mi>L</mi> <mrow> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> </mrow> </msub> <mrow> <mo>(</mo> <msup> <mi>A</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>,</mo> <msup> <mi>B</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>,</mo> <msup> <mi>C</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>,</mo> <mi>E</mi> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>Y</mi> <mn>3</mn> <mi>k</mi> </msubsup> <mo>)</mo> </mrow> <mo>}</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msubsup> <mi>Y</mi> <mn>1</mn> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msubsup> <mo>=</mo> <msubsup> <mi>Y</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>+</mo> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mrow> <mo>(</mo> <msup> <mi>A</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>-</mo> <msub> <mi>&amp;Phi;</mi> <mi>r</mi> </msub> <msup> <mi>B</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>)</mo> </mrow> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msubsup> <mi>Y</mi> <mn>2</mn> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msubsup> <mo>=</mo> <msubsup> <mi>Y</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>+</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mrow> <mo>(</mo> <msup> <mi>A</mi> <mrow> <mi>k</mi> <mo>+</mo> <msup> <mn>1</mn> <mi>T</mi> </msup> </mrow> </msup> <mo>-</mo> <msub> <mi>&amp;Phi;</mi> <mi>r</mi> </msub> <msup> <mi>C</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>)</mo> </mrow> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msubsup> <mi>Y</mi> <mn>3</mn> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msubsup> <mo>=</mo> <msubsup> <mi>Y</mi> <mn>3</mn> <mi>k</mi> </msubsup> <mo>+</mo> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> <mrow> <mo>(</mo> <mi>D</mi> <mo>-</mo> <msup> <mi>A</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>-</mo> <msup> <mi>E</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>)</mo> </mrow> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msubsup> <mo>=</mo> <msub> <mi>&amp;rho;</mi> <mn>1</mn> </msub> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msubsup> <mo>=</mo> <msub> <mi>&amp;rho;</mi> <mn>2</mn> </msub> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>,</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msubsup> <mo>=</mo> <msub> <mi>&amp;rho;</mi> <mn>3</mn> </msub> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>3</mn> <mo>)</mo> </mrow> </mrow> in the above formula with respectively represent the values of variables B, C, A and E when the objective function takes the minimum value, ρ ₁ , ρ ₂ and ρ ₃ are multiple factors, and k is the number of iterations; then iteratively solve according to the following steps: 1) Solve B ^k+1 : use accelerated neighbor gradient algorithm to get B ^k+1 ; Remove the terms irrelevant to B in the objective function for solving B in formula (3), and get the following equation: Using the method of Taylor expansion, construct a second-order function to approximate the above formula, and then solve the original equation for this second-order function, let Introducing the variable Z again, we can finally solve: <mrow> <msup> <mi>B</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <msubsup> <mi>B</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> <mi>k</mi> </msubsup> <mo>=</mo> <mi>s</mi> <mi>o</mi> <mi>f</mi> <mi>t</mi> <mrow> <mo>(</mo> <msub> <mi>U</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>,</mo> <mfrac> <msub> <mi>&amp;gamma;</mi> <mi>B</mi> </msub> <msub> <mi>L</mi> <mi>f</mi> </msub> </mfrac> <msub> <mi>W</mi> <mi>b</mi> </msub> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>5</mn> <mo>)</mo> </mrow> </mrow> Among them, soft(·,·) is the contraction operator, is the gradient of f(Z), L _f is a constant, the value is The update rule of the variable Z _j is as follows: <mrow> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <msub> <mi>t</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>=</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>+</mo> <msqrt> <mrow> <mn>4</mn> <msubsup> <mi>t</mi> <mi>j</mi> <mn>2</mn> </msubsup> <mo>+</mo> <mn>1</mn> </mrow> </msqrt> <mo>)</mo> </mrow> <mo>/</mo> <mn>2</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mi>Z</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>=</mo> <msubsup> <mi>B</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> <mi>k</mi> </msubsup> <mo>+</mo> <mfrac> <mrow> <msub> <mi>t</mi> <mi>j</mi> </msub> <mo>-</mo> <mn>1</mn> </mrow> <msub> <mi>t</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> </mfrac> <mrow> <mo>(</mo> <msubsup> <mi>B</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> <mi>k</mi> </msubsup> <mo>-</mo> <msubsup> <mi>B</mi> <mi>j</mi> <mi>k</mi> </msubsup> <mo>)</mo> </mrow> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>6</mn> <mo>)</mo> </mrow> </mrow> Among them, t _j is a set of constant sequences, and j is the number of variable iterations; 2) Solve C ^k+1 : use accelerated neighbor gradient algorithm to find C ^k+1 ; Remove the terms irrelevant to C in the objective function for solving C in formula (3), and get the following equation: Using the method of Taylor expansion, construct a second-order function to approximate the above formula, and then solve the original equation for this second-order function, let reintroduce variable Finally it can be solved: <mrow> <msup> <mi>C</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <msubsup> <mi>C</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> <mi>k</mi> </msubsup> <mo>=</mo> <mi>s</mi> <mi>o</mi> <mi>f</mi> <mi>t</mi> <mrow> <mo>(</mo> <msub> <mover> <mi>U</mi> <mo>~</mo> </mover> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>,</mo> <mfrac> <msub> <mi>&amp;gamma;</mi> <mi>C</mi> </msub> <msub> <mi>L</mi> <mi>f</mi> </msub> </mfrac> <msub> <mi>W</mi> <mi>c</mi> </msub> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>8</mn> <mo>)</mo> </mrow> </mrow> Among them, soft(·,·) is the contraction operator, for The gradient of , L _f is a constant whose value is variable The update rules for are as follows: <mrow> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <msub> <mi>t</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>=</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>+</mo> <msqrt> <mrow> <mn>4</mn> <msubsup> <mi>t</mi> <mi>j</mi> <mn>2</mn> </msubsup> <mo>+</mo> <mn>1</mn> </mrow> </msqrt> <mo>)</mo> </mrow> <mo>/</mo> <mn>2</mn> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mover> <mi>Z</mi> <mo>~</mo> </mover> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> <mo>=</mo> <msubsup> <mi>C</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> <mi>k</mi> </msubsup> <mo>+</mo> <mfrac> <mrow> <msub> <mi>t</mi> <mi>j</mi> </msub> <mo>-</mo> <mn>1</mn> </mrow> <msub> <mi>t</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> </msub> </mfrac> <mrow> <mo>(</mo> <msubsup> <mi>C</mi> <mrow> <mi>j</mi> <mo>+</mo> <mn>1</mn> </mrow> <mi>k</mi> </msubsup> <mo>-</mo> <msubsup> <mi>C</mi> <mi>j</mi> <mi>k</mi> </msubsup> <mo>)</mo> </mrow> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>9</mn> <mo>)</mo> </mrow> </mrow> Among them, t _j is a set of constant sequences, and j is the number of variable iterations; 3) Solve A ^k+1 : use Singular Value Thresholding (SVT) to solve A ^k+1 ; Remove the items irrelevant to A in the objective function for solving A in formula (3), and obtain through the formula: in, Using the singular value threshold method for Q ^{k +1} to solve: <mrow> <msup> <mi>A</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <msup> <mi>H</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mi>s</mi> <mi>o</mi> <mi>f</mi> <mi>t</mi> <mrow> <mo>(</mo> <msup> <mi>&amp;Sigma;</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>,</mo> <mfrac> <mn>1</mn> <mrow> <msubsup> <mi>&amp;mu;</mi> <mn>1</mn> <mi>k</mi> </msubsup> <mo>+</mo> <msubsup> <mi>&amp;mu;</mi> <mn>2</mn> <mi>k</mi> </msubsup> <mo>+</mo> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> </mrow> </mfrac> <msub> <mi>W</mi> <mi>a</mi> </msub> <mo>)</mo> </mrow> <msup> <mi>V</mi> <mrow> <mi>k</mi> <mo>+</mo> <msup> <mn>1</mn> <mi>T</mi> </msup> </mrow> </msup> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>11</mn> <mo>)</mo> </mrow> </mrow> Among them, H ^k+1 and V ^k+1 are the left singular matrix and right singular matrix of Q ^k+ 1 respectively; 4) Solve E ^k+1 : the solution of E ^k+1 consists of two parts; In the observation space Ω, the value of E is 0; outside the observation space Ω, that is, the complementary space Inside, use the first-order derivation to solve, and combine the two parts to get the final solution of E: <mrow> <msup> <mi>E</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>=</mo> <msub> <mi>P</mi> <mi>&amp;Omega;</mi> </msub> <mrow> <mo>(</mo> <mn>0</mn> <mo>)</mo> </mrow> <mo>+</mo> <msub> <mi>P</mi> <mover> <mi>&amp;Omega;</mi> <mo>&amp;OverBar;</mo> </mover> </msub> <mrow> <mo>(</mo> <mi>D</mi> <mo>-</mo> <msup> <mi>A</mi> <mrow> <mi>k</mi> <mo>+</mo> <mn>1</mn> </mrow> </msup> <mo>+</mo> <mfrac> <msubsup> <mi>Y</mi> <mn>3</mn> <mi>k</mi> </msubsup> <msubsup> <mi>&amp;mu;</mi> <mn>3</mn> <mi>k</mi> </msubsup> </mfrac> <mo>)</mo> </mrow> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>12</mn> <mo>)</mo> </mrow> </mrow> 5) Repeat the above steps 1), 2), 3) and 4) until the algorithm converges, then the iteration results A ^k+1 , Σ ^k+1 , B ^k+1 , C ^k+1 and E ^k+1 are The original problem has no reweighted results A ^(l) , Σ ^(l) , B ^(l) , C ^(l) and E ^(l) , where l is the number of reweighted times; 6) Updating weight matrices W _a , W _b and W _c ; In order to offset the influence of the signal amplitude on the nuclear norm item and the one norm item, a reweighting scheme is introduced, and the inverse proportional principle is adopted according to the amplitudes of the currently estimated singular value matrix Σ ^(l) and coefficient matrices B ^l and C ^l Iteratively update the weight matrices W _a , W _b and W _c : <mrow> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <msubsup> <mi>W</mi> <mi>a</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mrow> <mo>(</mo> <mover> <mi>i</mi> <mo>^</mo> </mover> <mo>,</mo> <mover> <mi>i</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mn>1</mn> <mrow> <msubsup> <mi>&amp;sigma;</mi> <mi>i</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msubsup> <mo>+</mo> <mi>&amp;epsiv;</mi> </mrow> </mfrac> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msubsup> <mi>W</mi> <mi>b</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mrow> <mo>(</mo> <mover> <mi>i</mi> <mo>^</mo> </mover> <mo>,</mo> <mover> <mi>j</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mn>1</mn> <mrow> <mo>|</mo> <msup> <mi>B</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mrow> <mo>(</mo> <mover> <mi>i</mi> <mo>^</mo> </mover> <mo>,</mo> <mover> <mi>j</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>|</mo> <mo>+</mo> <mi>&amp;epsiv;</mi> </mrow> </mfrac> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msubsup> <mi>W</mi> <mi>c</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> </msubsup> <mrow> <mo>(</mo> <mover> <mi>i</mi> <mo>^</mo> </mover> <mo>,</mo> <mover> <mi>j</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mn>1</mn> <mrow> <mo>|</mo> <msup> <mi>C</mi> <mrow> <mo>(</mo> <mi>l</mi> <mo>)</mo> </mrow> </msup> <mrow> <mo>(</mo> <mover> <mi>i</mi> <mo>^</mo> </mover> <mo>,</mo> <mover> <mi>j</mi> <mo>^</mo> </mover> <mo>)</mo> </mrow> <mo>|</mo> <mo>+</mo> <mi>&amp;epsiv;</mi> </mrow> </mfrac> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>13</mn> <mo>)</mo> </mrow> </mrow> in is the position coordinate of the pixel in the image, and ε is an arbitrarily small positive number. 7) Repeat the above steps 1)-7) until the algorithm converges, then the iteration result A ^(l) is the final solution A of the original problem.