CN108416723B - Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting - Google Patents

Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting Download PDF

Info

Publication number
CN108416723B
CN108416723B CN201810122490.0A CN201810122490A CN108416723B CN 108416723 B CN108416723 B CN 108416723B CN 201810122490 A CN201810122490 A CN 201810122490A CN 108416723 B CN108416723 B CN 108416723B
Authority
CN
China
Prior art keywords
image
iteration
variable
solving
total variation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810122490.0A
Other languages
Chinese (zh)
Other versions
CN108416723A (en
Inventor
孙权森
钟万强
陈强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN201810122490.0A priority Critical patent/CN108416723B/en
Publication of CN108416723A publication Critical patent/CN108416723A/en
Application granted granted Critical
Publication of CN108416723B publication Critical patent/CN108416723B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0007—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20192—Edge enhancement; Edge preservation

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a lensless imaging fast reconstruction method based on total variation regularization and variable splitting. The method adopts the ideas of total variation regularization and variable splitting aiming at the image reconstruction problem of a lens-free imaging system, splits an objective function to be solved into two subproblems, and finally alternately solves the subproblems to obtain a final result. Firstly, introducing a total variation regularization image reconstruction model according to a linear imaging mechanism in lens-free imaging; introducing an auxiliary variable, and splitting an objective function to be solved into two subproblems by using a variable splitting method; then solving the two subproblems by using Gihonov regularization and Total Variation (TV) regularization of anisotropy respectively; and finally, alternately solving the two sub-problems to find the optimal solution. The invention not only can ensure that the reconstruction of the image without the lens is stably carried out in the presence of non-ideal factors, but also can effectively remove noise, and simultaneously can keep the detailed information of the edge of the reconstructed image and the like.

Description

Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting
Technical Field
The invention relates to the field of lens-free coding plate imaging systems, in particular to an image reconstruction technology based on total variation regularization and variable splitting.
Background
In recent years, the advent of new imaging applications has driven the development of lensless imaging systems. The main direction of research in lens-less imaging systems was coded aperture (also called code plate) imaging, which was originally used for X-ray and Gamma-ray imaging in astronomy, and it was often technically difficult to manufacture imaging lenses suitable for such rays. In recent years, researchers have proposed lens-less imaging systems for the visible and infrared spectrum, which have different application scenarios depending on the type of encoding plate and the imaging principle. Lens-less imaging systems have unique advantages over conventional lens-based cameras, such as the imaging devices can be made very thin, non-planar, relatively inexpensive, and without limitation to the imaging spectrum. These advantages have led to the widespread use of lensless imaging systems in certain specific fields, such as astronomy, nanomaterial exploration, body cell monitoring, etc.
In general, a lens-less imaging system uses an optical mask (referred to as an encoder plate) instead of a lens, and the encoder plate is disposed parallel to a sensor (see fig. 2 for a structural schematic diagram). In contrast to conventional lens-based cameras, the measured values recorded on the sensor are not direct images of the imaging target, but a superposition of the images of the different apertures on the code plate. Such an imaging model results in that a corresponding reconstruction algorithm has to be used to reconstruct an image of the target scene. In general, the measurement values on the sensor and the target scene are in a linear relationship, in other words, assuming that u represents an image of the target scene with a size of N × N and f represents a measurement value of the sensor with a size of M × M, u and f are stretched into one-dimensional vectors, i.e., one-dimensional vectors
Figure BDA0001572502890000011
Then u and f satisfy this relationship: f ═ Φ u + e, where Φ is an M2×N2The system of (2) is to convert the matrix,
Figure BDA0001572502890000012
is the noise of the imaging system. This linear relationship generally results in a large matrix Φ dimension, which is not conducive to image reconstruction. Researchers have then proposed using separable code plate patterns (i.e., a code plate pattern can be mathematically written as the outer product of two vectors) to transform the mathematical model of the imaging structure into this form:
Figure BDA0001572502890000013
wherein phiLAnd phiR(
Figure BDA0001572502890000014
Is phiRTranspose of) are both conversion matrices of the imaging system. In this case, u and f do not need to be stretched into a one-dimensional vector, and ΦLAnd phiRAre all MxN, thereby reducingComputational complexity at reconstruction.
The existing reconstruction method of the lens-free imaging system is mainly based on Gihonkov [1. Deweet M J, Farm B. Lensless coded imaging with isolated Toeplitz masks [ J ]].Optical Engineering,2015,54(2):023102.]，[2.Asif M S,Ayremlou A,Sankaranarayanan A,et al.FlatCam:Thin,Lensless Cameras Using Coded Aperture and Computation[J].IEEE Transactions on Computational Imaging,2017,3(3):384-397]Regularization techniques that allow image reconstruction to be performed with non-idealities (e.g., transformation matrix Φ)LAnd phiRIll-conditioned, system noise, etc.) and does not require iteration, the reconstruction speed is fast, but the noise cannot be effectively removed, and the reconstruction quality is not good. Although the noise can be removed by the existing denoising technology (such as BM3D) after reconstruction, the detail information such as image edges is easily lost. In addition, the traditional partial differential equation-based total variation regularization can also be used for image reconstruction of a lens-free imaging system, and can effectively keep edge information, but the iteration times are large, the reconstruction time is slow, and the noise of an image smooth area cannot be effectively removed.
Disclosure of Invention
The invention aims to provide a lens-free image reconstruction method which can effectively remove noise of a reconstructed image, simultaneously keeps detailed information such as image edges and the like and has high reconstruction speed.
The technical solution for realizing the purpose of the invention is as follows: a lens-free imaging fast reconstruction method based on total variation regularization and variable splitting comprises the following steps:
step 1: acquiring a simulation measured value of the sensor: inputting a NxN natural image and transformation matrix of lens-free imaging system
Figure BDA0001572502890000021
And
Figure BDA0001572502890000022
imaging mechanism according to a lens-less imaging system
Figure BDA0001572502890000023
(
Figure BDA0001572502890000024
Is phiRTranspose) of the noise e) is obtained by computer simulation to obtain sensor measurement values under different noises e
Figure BDA0001572502890000025
Step 2: constructing an image reconstruction fidelity item: according to the imaging mechanism of the lens-free imaging system, the fidelity term for image reconstruction is constructed as follows:
Figure BDA0001572502890000026
in the formula | | | | non-conducting phosphor2Is a norm of the matrix L2,
Figure BDA0001572502890000027
is the measured value of the sensor or sensors,
Figure BDA0001572502890000028
is the target scene image to be solved;
and step 3: constructing an image gradient sparse regular term: according to image gradient sparse prior, constructing a total variation regular term of each anisotropy as follows:
||u||TV＝||ux||1+||uy||1＝||Dxu||1+||Dyu||1
in the formula | | u | | non-conducting phosphorTVIs an image
Figure BDA0001572502890000031
The total variation of (a) is,
Figure BDA0001572502890000032
and
Figure BDA0001572502890000033
for image u in both horizontal and vertical directionsThe gradient of the direction of the flow is,
Figure BDA0001572502890000034
and
Figure BDA0001572502890000035
gradient operators in the horizontal and vertical directions of the image respectively, | | | | | non-woven phosphor1Is the L1 norm of the matrix.
And 4, step 4: constructing a reconstruction model and splitting the model:
(1) according to the fidelity term and the regular term, an image reconstruction model is constructed:
Figure BDA0001572502890000036
wherein lambda > 0 is a regularization parameter,
Figure BDA0001572502890000037
is the image of the target scene to be solved,
Figure BDA0001572502890000038
is the measured value of the sensor or sensors,
Figure BDA0001572502890000039
and
Figure BDA00015725028900000310
the gradients of the image u in the horizontal and vertical directions respectively;
(2) introducing an auxiliary variable
Figure BDA00015725028900000311
Let d → u, change the reconstructed model to the following constrained model:
Figure BDA00015725028900000312
s.t.d＝u
→ represents the approach, applying the augmented Lagrange multiplier method yields the following unconstrained minimization model:
Figure BDA00015725028900000313
in the formula, lambda is more than 0, mu is more than 0 and is a regularization parameter;
(3) the solution process of the objective function is written into the following iteration format by using a Bregman iteration method:
Figure BDA00015725028900000314
wherein the parameter lambda is more than 0, mu is more than 0,
Figure BDA00015725028900000315
is an iterative error variable, u(k+1)Representing the target image of the (k + 1) th iteration, d(k+1)Auxiliary variable for the (k + 1) th iteration, b(k)An error variable representing the kth iteration;
(4) the model to be solved is split into two sub-problems by applying a split Bregman method, and written in the following iterative format:
Figure BDA0001572502890000041
in the formula, the parameters lambda is more than 0, mu is more than 0, u(k+1)Representing the target image of the (k + 1) th iteration, d(k)As an auxiliary variable for the kth iteration, b(k)The error variable for the kth iteration is indicated. In this iterative equation, the target image is solved
Figure BDA0001572502890000042
Can be regarded as an inverse problem solving problem, while solving the auxiliary variables
Figure BDA0001572502890000043
Can be regarded as a fully variant denoising problem.
And 5: solving an inverse sub-problem: first of all for the conversion matrix phiLAnd phiRSingular value decomposition is performed, assuming
Figure BDA0001572502890000044
Are respectively phiLAnd phiRSingular value decomposition of wherein
Figure BDA0001572502890000045
Is a unitary matrix of the matrix,
Figure BDA0001572502890000046
is a diagonal matrix of singular values. The objective function for solving this subproblem is a convex function, which is directly derived and its derivative is made equal to 0:
Figure BDA0001572502890000047
will phiLAnd phiRSubstituting and sorting singular value decomposition items to obtain a target scene image u:
Figure BDA0001572502890000048
wherein the regularization parameter mu is greater than 0,
Figure BDA0001572502890000049
are respectively diagonal matrixes
Figure BDA00015725028900000410
A one-dimensional vector composed of diagonal position elements,/representing a matrix dividing by element (also called a dot division operation), 1 representing a column vector with all elements 1, 11TForming a matrix of all 1 elements of size M × N, d(k)And b(k)The auxiliary variable and the error variable for the kth iteration are indicated, respectively.
Step 6: solving a total variation de-noising subproblem:
(1) firstly, two auxiliary variables R are introduced1,R2Separately replacing gradient terms in the objective functiondxAnd dyThen, solving a corresponding augmented Lagrangian minimization model:
Figure BDA00015725028900000411
where the parameters μ > 0, λ > 0, γ > 0, auxiliary variables
Figure BDA0001572502890000051
(2) The following iterative solution format can be obtained by applying a Bregman iterative method to the model:
Figure BDA0001572502890000052
in the above equation, t represents the t-th iteration,
Figure BDA0001572502890000053
an iteration error variable is represented. Splitting the first sub-problem in the iterative equation into three sub-problems to be solved respectively:
Figure BDA0001572502890000054
in the above formula, the parameters mu is more than 0, lambda is more than 0, and gamma is more than 0.
(3) Solving for d(t+1): the following solution d can be obtained by using Gauss-Seidel iteration and a two-dimensional Laplace operator(t+1)The formula (II) is as follows:
Figure BDA0001572502890000055
in the above formula, the first and second carbon atoms are,
Figure BDA0001572502890000056
the element value of the variable d of the t +1 th iteration at the ith row and the jth column of the matrix is represented, and the meaning of other variables with subscripts is similar.
(4) Solving for R1And R2: the objective functions corresponding to both variables can be solved using a two-dimensional soft threshold operator:
Figure BDA0001572502890000057
in the above formula, shrink (·) is a two-dimensional soft threshold operator,
Figure BDA0001572502890000058
and
Figure BDA0001572502890000059
bounded differential in the horizontal and vertical directions of the matrix d, respectively, i.e. the element in the ith row and jth column satisfies:
Figure BDA0001572502890000061
and 7: solving the model: iterating step 5 and step 6, and terminating iteration when iteration error is less than a certain threshold or maximum iteration number is met (iteration error threshold is generally 10)-5～10-3And the maximum iteration number is generally 10-20), and finally, a reconstructed image is output.
Compared with the prior art, the invention has the following remarkable advantages: (1) the invention divides the problem to be solved into two sub-problems by using the thought of variable splitting, solves the sub-problems by respectively applying Gihonov regularization and total variation regularization, combines the two processes of image reconstruction and denoising, effectively removes the noise in the process of image reconstruction, keeps the detailed information of the image edge and the like, and improves the operation speed of the algorithm and the quality of the reconstructed image compared with the traditional total variation regularization method based on partial differential equation. (2) The invention can be widely applied to the non-lens imaging system in the fields of sensor network video monitoring, environment monitoring and the like.
Drawings
FIG. 1 is a flow chart of the present invention lens-free imaging image reconstruction method based on total variation regularization and variable splitting.
Fig. 2 is a schematic diagram of a lensless imaging system.
Fig. 3(a) is a selected portion of the test image, and fig. 3(b) is a corresponding simulated sensor measurement.
Fig. 4(a) is the result of reconstruction by the Tikhonov method on the image "Lena", fig. 4(b) is the result of reconstruction by the Tikhonov + BM3D method on the image "Lena", fig. 4(c) is the result of reconstruction by the TV method on the image "Lena", and fig. 4(d) is the result of reconstruction by the method of the present invention on the image "Lena".
Detailed Description
The invention will be further explained with reference to the drawings.
The following detailed description of the implementation of the present invention, with reference to fig. 1, includes the following steps:
step 1: acquiring a simulation measured value of the sensor: inputting an NxN natural image u and a transformation matrix of a lens-free imaging system
Figure BDA0001572502890000062
And
Figure BDA0001572502890000063
imaging mechanism according to a lens-less imaging system
Figure BDA0001572502890000064
(
Figure BDA0001572502890000065
Is phiRTranspose) of (a) to (b), different noise is obtained by computer simulation
Figure BDA0001572502890000066
Measured value of sensor
Figure BDA0001572502890000067
In our experiments we used three means of 0The effectiveness of the invention is verified by Gaussian white noise with different standard deviations, and the three standard deviations are respectively 5,10 and 15.
Step 2: constructing an image reconstruction fidelity item: according to the imaging mechanism of the lens-free imaging system, the fidelity term for image reconstruction is constructed as follows:
Figure BDA0001572502890000071
in the formula | | | | non-conducting phosphor2Is the L2 norm of the matrix,
Figure BDA0001572502890000072
is the measured value of the sensor or sensors,
Figure BDA0001572502890000073
is the target scene graph to be solved,
Figure BDA0001572502890000074
and
Figure BDA0001572502890000075
is a transformation matrix of the imaging system;
and step 3: constructing an image gradient sparse regular term: according to image gradient sparse prior, constructing a total variation regular term of each anisotropy as follows:
||u||TV＝||ux||1+||uy||1＝||Dxu||1||Dyu||1
in the formula | | u | | non-conducting phosphorTVIs an image
Figure BDA0001572502890000076
The total variation of (a) is,
Figure BDA0001572502890000077
and
Figure BDA0001572502890000078
the gradient of the image u in both the horizontal and vertical directions,
Figure BDA0001572502890000079
and
Figure BDA00015725028900000710
gradient operators in the horizontal and vertical directions of the image respectively, | | | | | non-woven phosphor1Is the L1 norm of the matrix.
And 4, step 4: constructing a reconstruction model and splitting the model:
(1) according to the fidelity term and the regular term, an image reconstruction model is constructed:
Figure BDA00015725028900000711
wherein lambda > 0 is a regularization parameter,
Figure BDA00015725028900000712
is the image of the target scene to be solved,
Figure BDA00015725028900000713
is the measured value of the sensor or sensors,
Figure BDA00015725028900000714
and
Figure BDA00015725028900000715
the gradients of the image u in the horizontal and vertical directions respectively;
(2) introducing an auxiliary variable
Figure BDA00015725028900000716
Let d → u, change the reconstructed model to the following constrained model:
Figure BDA00015725028900000717
s.t.d＝u
→ represents the approach, applying the augmented Lagrange multiplier method yields the following unconstrained minimization model:
Figure BDA00015725028900000718
in the formula, lambda is more than 0, mu is more than 0 and is a regularization parameter;
(3) the solution process of the objective function is written into the following iteration format by using a Bregman iteration method:
Figure BDA0001572502890000081
wherein the parameter lambda is more than 0, mu is more than 0,
Figure BDA0001572502890000082
is an iterative error variable, u(k+1)Representing the target image of the (k + 1) th iteration, d(k+1)Auxiliary variable for the (k + 1) th iteration, b(k)An error variable representing the kth iteration;
(4) the model to be solved is split into two sub-problems by applying a split Bregman method, and written in the following iterative format:
Figure BDA0001572502890000083
in the formula, the parameters lambda is more than 0, mu is more than 0, u(k+1)Representing the target image of the (k + 1) th iteration, d(k)As an auxiliary variable for the kth iteration, b(k)The error variable for the kth iteration is indicated. In this iterative equation, the target image is solved
Figure BDA0001572502890000084
Can be regarded as an inverse problem solving problem, while solving the auxiliary variables
Figure BDA0001572502890000085
Can be regarded as a fully variant denoising problem.
And 5: solving an inverse sub-problem: first of all for the conversion matrix phiLAnd phiRSingular value decomposition is performed, assuming
Figure BDA0001572502890000086
Are respectively phiLAnd phiRSingular value decomposition of wherein
Figure BDA0001572502890000087
Is a unitary matrix of the matrix,
Figure BDA0001572502890000088
is a diagonal matrix of singular values. The objective function of this subproblem is:
Figure BDA0001572502890000089
it is clearly a convex function, and we directly derive the objective function and make its derivative equal to 0:
Figure BDA00015725028900000810
will phiLAnd phiRSubstituting and sorting singular value decomposition items to obtain:
Figure BDA00015725028900000811
in the above equation, the two sides of the equation are left-multiplied by VLRight-handed VRThe following can be obtained:
Figure BDA0001572502890000091
here we use two one-dimensional vectors
Figure BDA0001572502890000092
And
Figure BDA0001572502890000093
respectively representing diagonal matrices
Figure BDA0001572502890000094
And
Figure BDA0001572502890000095
the element at the middle diagonal position, the above equation is written as follows:
Figure BDA0001572502890000096
and finally, obtaining a target scene image u by sorting:
Figure BDA0001572502890000097
the regularization parameter μ > 0,/denotes that the matrix divides by element (also called a dot division operation), 1 denotes a column vector with all 1 elements, 11TForming a matrix of all 1 elements of size M × N, d(k)And b(k)The auxiliary variable and the error variable for the kth iteration are indicated, respectively.
Step 6: solving a total variation de-noising subproblem:
(1) first, two auxiliary variables are introduced
Figure BDA00015725028900000913
Separately replacing the gradient term d in the objective functionxAnd dyI.e. writing the objective function with solution into the following format:
Figure BDA0001572502890000098
s.t.R1＝dx and R2＝dy
then, a Lagrange multiplier method is applied to convert the model into an unconstrained minimization model as follows:
Figure BDA0001572502890000099
in the above formula, the parameters μ > 0, λ > 0, γ > 0, gradient variables
Figure BDA00015725028900000910
Auxiliary variable
Figure BDA00015725028900000911
(2) The following iterative solution format can be obtained by applying a Bregman iterative method to the model:
Figure BDA00015725028900000912
in the above equation, t represents the t-th iteration,
Figure BDA0001572502890000101
an iteration error variable is represented. Splitting the first sub-problem in the iterative equation into three sub-problems to be solved respectively:
Figure BDA0001572502890000102
in the above formula, the parameters mu is more than 0, lambda is more than 0, and gamma is more than 0.
(3) Solving the first sub-problem d(t+1): firstly, derivation is carried out on an objective function, the derivative of the objective function is 0, and the following results are obtained through sorting:
Figure BDA0001572502890000103
in the above formula Δ represents the laplacian operator,
Figure BDA0001572502890000104
and
Figure BDA0001572502890000105
representing differential operators in both horizontal and vertical directions, respectively, where we use a common operator4 field Laplacian of, i.e.
Δdi,j＝-4di,j+di-1,j+di+1,j+di,j-1+di,j+1，di,jRepresenting the values of the elements of the matrix d at the ith row and jth column position, similarly we approximate using backward differentiation
Figure BDA0001572502890000106
And
Figure BDA0001572502890000107
namely:
Figure BDA0001572502890000108
wherein (R)1)i,jDenotes an auxiliary variable R1The values of the elements at the ith row and jth column position are similar to those of the other variables. The above equation is substituted into the objective function, and the following solution d can be obtained by using Gauss-Seidel iteration method(t+1)The formula (II) is as follows:
Figure BDA0001572502890000109
in the above formula, the first and second carbon atoms are,
Figure BDA0001572502890000111
the element value of the variable d of the t +1 th iteration at the ith row and the jth column of the matrix is represented, and the meaning of other variables with subscripts is similar.
(4) Solving for R1And R2: the objective functions corresponding to both variables can be solved using a two-dimensional soft threshold operator:
Figure BDA0001572502890000112
Figure BDA0001572502890000113
in the above formula, the parameter λ is greater than 0, γ is greater than 0, shrnk (·) is a two-dimensional soft threshold operator,
Figure BDA0001572502890000114
and
Figure BDA0001572502890000115
is the error variable for the t-th iteration,
Figure BDA0001572502890000116
and
Figure BDA0001572502890000117
bounded differential in the horizontal and vertical directions of the matrix d, respectively, i.e. the element in the ith row and jth column satisfies:
Figure BDA0001572502890000118
and 7: solving the model: iterating step 5 and step 6, and terminating iteration when iteration error is less than a certain threshold or maximum iteration number is met (iteration error threshold is generally 10)-5～10-3And the maximum iteration number is generally 10-20), and finally, a reconstructed image is output.
The present invention will be described in detail with reference to specific examples.
The simulation experiment used 8 classical black and white images in digital image processing, Lena (512 × 512), Peppers (512 × 512), Barbara (512 × 512), Goldhill (512 × 512), Cameraman (256 × 256), House (256 × 256), Couple (256 × 256), and Pirate (256 × 256). In experiments, to verify the effectiveness of the present invention, we set the transformation matrix Φ of the lensless imaging systemLAnd phiREqual to the gaussian random matrix and equal in size to the test image. Specifically, if a given test image is Lena (512 × 512), then Φ isLAnd phiRAre two different gaussian random matrices of 512 x 512 size. In the simulation of lensless imaging, we introduceThree kinds of white Gaussian noise with different degrees are added, and the standard deviation sigma of the noise is respectively 5,10 and 15. Fig. 3(a) is a partial test image, and fig. 3(b) is a sensor measurement value obtained by simulation when the noise criterion σ is 5 (corresponding to fig. 3 (a)). The simulation experiments of the invention are all completed by using MATLAB R2014rb under one dual-core notebook (Intel i52.3Ghz, 8GB memory, Windows 7 system).
The invention uses the measured value of the sensor in the lens-free imaging system obtained by simulation to verify the image reconstruction effect. In order to test the performance of the algorithm, the total variation regularization and variable splitting based reconstruction method is compared with the existing reconstruction algorithm. The comparison method comprises the following steps: ginkhonv (Tikhonov) regularized reconstruction [ Deweet M J, Farm B P.Lensless Coded adaptation Imaging with segmented double Toeplitz masks [ J ]. Optical Engineering,2015,54(2):023102 ], Ginkhonv (Tikhonov) reconstruction + BM3D [ asset M S, Azeremulou A, Sankaraarayanan A, et al.Flatcam: Thin, lens reactors Using Coded and Computation [ J ] IEEE Transactions on computing, 2017, 3): 384-a ], conventional partial equation based total variation/TV regularization method [ Rudin, Osscience S, surface electronics ] analysis, publication No. 268, analysis.
Fig. 4(a) to 4(d) are graphs of the results of reconstruction by different algorithms on the image Lena (512 × 512) when the noise criterion σ is 5, respectively. In order to objectively evaluate the reconstruction effect of the algorithm and other comparison algorithms, a peak signal-to-noise ratio (PSNR) and a Structural Similarity (SSIM) are used as evaluation criteria of reconstruction quality, and the higher the values of the peak signal-to-noise ratio (PSNR) and the Structural Similarity (SSIM), the better the reconstruction quality is. Table 1 shows the PSNR values and SSIM values reconstructed by each algorithm under different noise standard deviations, where we bold the item with the highest value, and the last row of the table is the average PSNR value and average SSIM value of 8 reconstructed images. As can be seen from fig. 4(a) to 4(d), the present invention effectively removes noise on the reconstructed image, and retains detail information such as image edges, which is superior to other algorithms in image quality. As can be seen from Table 1, no matter under which standard deviation noise, the image reconstructed by the method is higher than other algorithms in PSNR value and SSIM value, and the average PSNR is about 3.37dB, 3.75dB and 3.61dB higher than Gihonov + BM3D under three kinds of noise respectively; the average SSIM is about 0.061, 0371, 0.0572.
TABLE 1 PSNR and SSIM values reconstructed by each algorithm under different noise levels
The upper value in the same row in the table is PSNR (in dB) and the lower value is SSIM
Figure BDA0001572502890000121
Figure BDA0001572502890000131

Claims (7)

1. A lens-free imaging fast reconstruction method based on total variation regularization and variable splitting is characterized by comprising the following steps:
step 1: acquiring a simulation measured value of the sensor: inputting a conversion matrix of a natural image and a lens-free imaging system, and obtaining sensor simulation measurement values under different noise degrees through computer simulation according to an imaging mechanism of the lens-free imaging system;
step 2: constructing an image reconstruction fidelity item; constructing an image reconstruction fidelity term by using a matrix L2 norm;
and step 3: constructing an image gradient sparse regular term; constructing a total variation regular term of each anisotropy according to image gradient sparse prior;
and 4, step 4: constructing a reconstruction model and splitting the model; constructing a reconstruction model according to a fidelity term and a regular term, introducing an auxiliary variable, and splitting a target function of the model into an inverse subproblem and a total variation denoising subproblem by using a split Bregman method; the step 4 of constructing the reconstruction model and the model splitting specifically comprise the following steps:
step 4.1, constructing an image reconstruction model according to the fidelity term and the regular term:
Figure FDA0003408569430000011
wherein lambda > 0 is a regularization parameter,
Figure FDA0003408569430000012
is the image of the target scene to be solved,
Figure FDA0003408569430000013
is a measurement value of the simulation of the sensor,
Figure FDA0003408569430000014
and
Figure FDA0003408569430000015
the gradients of the image u in the horizontal and vertical directions respectively;
step 4.2 introducing an auxiliary variable
Figure FDA0003408569430000016
Let d → u, change the reconstructed model to the following constrained model:
Figure FDA0003408569430000017
s.t.d＝u
→ represents the approach, applying the augmented Lagrange multiplier method yields the following unconstrained minimization model:
Figure FDA0003408569430000018
in the formula, lambda is more than 0, mu is more than 0 and is a regularization parameter;
and 4.3, writing the solving process of the objective function into the following iterative format by using a Bregman iterative method:
Figure FDA0003408569430000021
wherein the parameter lambda is more than 0, mu is more than 0,
Figure FDA0003408569430000022
is an iterative error variable, u(k+1)Representing the target image of the (k + 1) th iteration, d(k+1)Auxiliary variable for the (k + 1) th iteration, b(k)An error variable representing the kth iteration;
step 4.4, the model to be solved is split into two subproblems by using a split Bregman method, and the subproblems are written into the following iteration format:
Figure FDA0003408569430000023
in the formula, the parameters lambda is more than 0, mu is more than 0, u(k+1)Representing the target image of the (k + 1) th iteration, d(k)As an auxiliary variable for the kth iteration, b(k)An error variable representing the kth iteration; in this iterative equation, the target image is solved
Figure FDA0003408569430000024
Is regarded as an inverse problem solving problem, while solving the auxiliary variables
Figure FDA0003408569430000025
The sub-problem of (2) is regarded as a total variation denoising problem;
and 5: solving an inverse sub-problem; carrying out singular value decomposition on the conversion matrix, and then solving the subproblem by applying the thought of direct solution of Gihonov regularization;
step 6: solving a total variation de-noising subproblem; introducing an auxiliary variable to replace a gradient term in the objective function, and then solving the subproblem by using a split Bregman method;
and 7: solving the model; and step 5 and step 6 are iterated, and when the end condition is met, the iteration is ended, and a reconstructed image is output.
2. The lensless imaging fast reconstruction method based on total variation regularization and variable splitting according to claim 1, characterized in that: the specific implementation method of the step 1 comprises the following steps: inputting an NxN natural image u and a transformation matrix of a lens-free imaging system
Figure FDA0003408569430000026
And
Figure FDA0003408569430000027
imaging mechanism according to a lens-less imaging system
Figure FDA0003408569430000028
Obtaining simulated measurement values of the sensor under different noises e through computer simulation
Figure FDA0003408569430000031
3. The lensless imaging fast reconstruction method based on total variation regularization and variable splitting according to claim 1, characterized in that: in the step 2, according to the imaging mechanism of the lens-free imaging system, the fidelity term of the image reconstruction is constructed as follows:
Figure FDA0003408569430000032
in the formula | | | | non-conducting phosphor2Is a norm of the matrix L2,
Figure FDA0003408569430000033
is a measurement value of the simulation of the sensor,
Figure FDA0003408569430000034
is the target scene image to be solved.
4. The lensless imaging fast reconstruction method based on total variation regularization and variable splitting according to claim 1, characterized in that: in the step 3, according to the image gradient sparse prior, a total variation regular term of each anisotropy is constructed as follows:
||u||TV＝||ux||1+||uy||1＝||Dxu||1+||Dyu||1
in the formula, | u | non-conducting phosphorTVIs an image
Figure FDA0003408569430000035
The total variation of (a) is,
Figure FDA0003408569430000036
and
Figure FDA0003408569430000037
the gradient of the image u in both the horizontal and vertical directions,
Figure FDA0003408569430000038
and
Figure FDA0003408569430000039
gradient operators in the horizontal and vertical directions of the image respectively, | | | | | non-woven phosphor1Is the L1 norm of the matrix.
5. The lensless imaging fast reconstruction method based on total variation regularization and variable splitting according to claim 1, characterized in that: solving an inverse sub-problem in the step 5:
first of all for the conversion matrix phiLAnd phiRSingular value decomposition is performed, assuming
Figure FDA00034085694300000310
Figure FDA00034085694300000311
Are respectively phiLAnd phiRSingular value decomposition of wherein
Figure FDA00034085694300000312
Is a unitary matrix of the matrix,
Figure FDA00034085694300000313
is a singular value diagonal matrix; the objective function for solving this subproblem is a convex function, which is directly derived and its derivative is made equal to 0:
Figure FDA00034085694300000314
will phiLAnd phiRSubstituting and sorting singular value decomposition items to obtain a target scene image u:
Figure FDA00034085694300000315
wherein, the regularization parameter mu is more than 0,
Figure FDA00034085694300000316
are respectively diagonal matrixes;
Figure FDA0003408569430000041
a one-dimensional vector composed of middle diagonal position elements,/representing a matrix divided by elements, 1 representing a column vector with all elements 1, 11TForming a matrix of all 1 elements of size M × N, d(k)And b(k)The auxiliary variable and the error variable for the kth iteration are indicated, respectively.
6. The lensless imaging fast reconstruction method based on total variation regularization and variable splitting according to claim 1, characterized in that: the solving of the total variation de-noising subproblem in the step 6 specifically comprises the following steps:
step 6.1 introduction of two auxiliary variables R1,R2Separately replacing the gradient term d in the objective functionxAnd dyThen, solving a corresponding augmented Lagrangian minimization model:
Figure FDA0003408569430000042
where the parameters μ > 0, λ > 0, γ > 0, auxiliary variables
Figure FDA0003408569430000043
Step 6.2, applying a Bregman iteration method to the model to obtain the following iteration solving format:
Figure FDA0003408569430000044
in the above equation, t represents the t-th iteration,
Figure FDA0003408569430000045
representing an iteration error variable; splitting the first sub-problem in the iterative equation into three sub-problems to be solved respectively:
Figure FDA0003408569430000046
in the above formula, the parameter mu is more than 0, lambda is more than 0, and gamma is more than 0;
step 6.3 solving for d(t+1): using Gauss-Seidel iteration and two-dimensional Laplace operator to obtain the following solution d(t+1)The formula (II) is as follows:
Figure FDA0003408569430000051
in the above formula, the first and second carbon atoms are,
Figure FDA0003408569430000052
the element value of a t +1 th iteration variable d at the ith row and the jth column position of the matrix is represented, and the meanings of other variables with subscripts and subscripts are similar to the element value;
step 6.4 solving for R1And R2: the objective functions corresponding to these two variables are both solved using a two-dimensional soft threshold operator:
Figure FDA0003408569430000053
in the above formula, shrink (·) is a two-dimensional soft threshold operator,
Figure FDA0003408569430000054
and
Figure FDA0003408569430000055
bounded differential in the horizontal and vertical directions of the matrix d, respectively, i.e. the element in the ith row and jth column satisfies:
Figure FDA0003408569430000056
7. the lensless imaging fast reconstruction method based on total variation regularization and variable splitting according to claim 1, characterized in that: the model solving process in the step 7 comprises the following steps: and (5) iterating step (6), terminating iteration when the iteration error is less than a certain threshold or the maximum iteration number is reached, and finally outputting a reconstructed image.
CN201810122490.0A 2018-02-07 2018-02-07 Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting Active CN108416723B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810122490.0A CN108416723B (en) 2018-02-07 2018-02-07 Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810122490.0A CN108416723B (en) 2018-02-07 2018-02-07 Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting

Publications (2)

Publication Number Publication Date
CN108416723A CN108416723A (en) 2018-08-17
CN108416723B true CN108416723B (en) 2022-02-18

Family

ID=63127859

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810122490.0A Active CN108416723B (en) 2018-02-07 2018-02-07 Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting

Country Status (1)

Country Link
CN (1) CN108416723B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109471053B (en) * 2018-10-18 2020-01-31 电子科技大学 dielectric characteristic iterative imaging method based on double constraints
CN109636738B (en) * 2018-11-09 2019-10-01 温州医科大学 The single image rain noise minimizing technology and device of double fidelity term canonical models based on wavelet transformation
CN109767404B (en) * 2019-01-25 2023-03-31 重庆电子工程职业学院 Infrared image deblurring method under salt and pepper noise
CN110161459B (en) * 2019-05-20 2021-01-26 浙江大学 Rapid positioning method for amplitude modulation sound source
CN110426704B (en) * 2019-08-20 2023-03-24 中国科学院重庆绿色智能技术研究院 Total variation fast imaging algorithm for sparse array
CN110544215B (en) * 2019-08-23 2023-07-21 淮阴工学院 Traffic monitoring image rain removing method based on anisotropic sparse gradient
CN110780273B (en) * 2019-11-04 2022-03-04 电子科技大学 Hybrid regularization azimuth super-resolution imaging method
CN112802135B (en) * 2021-01-15 2022-12-02 安徽大学 Ultrathin lens-free separable compression imaging system and calibration and reconstruction method thereof
GB2623404A (en) * 2022-07-04 2024-04-17 Univ Sun Yat Sen Thermal data determination method, apparatus and device
CN114841023B (en) * 2022-07-04 2022-09-09 中山大学 Thermal data determination method, device and equipment

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102208100A (en) * 2011-05-31 2011-10-05 重庆大学 Total-variation (TV) regularized image blind restoration method based on Split Bregman iteration
CN104107044A (en) * 2014-06-27 2014-10-22 山东大学（威海） Compressed sensing magnetic resonance image reconstruction method based on TV norm and L1 norm
CN104134196A (en) * 2014-08-08 2014-11-05 重庆大学 Split Bregman weight iteration image blind restoration method based on non-convex higher-order total variation model
CN105184755A (en) * 2015-10-16 2015-12-23 西南石油大学 Parallel magnetic resonance imaging high quality reconstruction method based on self-consistency and containing combined total variation
CN105551005A (en) * 2015-12-30 2016-05-04 南京信息工程大学 Quick image restoration method of total variation model coupled with gradient fidelity term
CN105954994A (en) * 2016-06-30 2016-09-21 深圳先进技术研究院 Image enhancement method for lensless digital holography microscopy imaging
CN107146202A (en) * 2017-03-17 2017-09-08 中山大学 The method of the Image Blind deblurring post-processed based on L0 regularizations and fuzzy core

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9292905B2 (en) * 2011-06-24 2016-03-22 Thomson Licensing Method and device for processing of an image by regularization of total variation
US8885975B2 (en) * 2012-06-22 2014-11-11 General Electric Company Method and apparatus for iterative reconstruction
US10274652B2 (en) * 2016-02-05 2019-04-30 Rambus Inc. Systems and methods for improving resolution in lensless imaging

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102208100A (en) * 2011-05-31 2011-10-05 重庆大学 Total-variation (TV) regularized image blind restoration method based on Split Bregman iteration
CN104107044A (en) * 2014-06-27 2014-10-22 山东大学（威海） Compressed sensing magnetic resonance image reconstruction method based on TV norm and L1 norm
CN104134196A (en) * 2014-08-08 2014-11-05 重庆大学 Split Bregman weight iteration image blind restoration method based on non-convex higher-order total variation model
CN105184755A (en) * 2015-10-16 2015-12-23 西南石油大学 Parallel magnetic resonance imaging high quality reconstruction method based on self-consistency and containing combined total variation
CN105551005A (en) * 2015-12-30 2016-05-04 南京信息工程大学 Quick image restoration method of total variation model coupled with gradient fidelity term
CN105954994A (en) * 2016-06-30 2016-09-21 深圳先进技术研究院 Image enhancement method for lensless digital holography microscopy imaging
CN107146202A (en) * 2017-03-17 2017-09-08 中山大学 The method of the Image Blind deblurring post-processed based on L0 regularizations and fuzzy core

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
An efficient augmented lagrangian method with applications to total variation minimization;Li C et al.;《Computational Optimization and Applications》;20130330;第56卷(第3期);第507-530页 *
基于低秩和全变差正则化的图像压缩感知重构;杨桄 等;《江苏大学学报（自然科学版）》;20170530;第38卷(第5期);第571-575页 *

Also Published As

Publication number Publication date
CN108416723A (en) 2018-08-17

Similar Documents

Publication Publication Date Title
CN108416723B (en) Lens-free imaging fast reconstruction method based on total variation regularization and variable splitting
Yao et al. Dr2-net: Deep residual reconstruction network for image compressive sensing
Huang et al. WINNet: Wavelet-inspired invertible network for image denoising
Hawe et al. Analysis operator learning and its application to image reconstruction
Golbabaee et al. Compressive source separation: Theory and methods for hyperspectral imaging
Chen et al. Learning memory augmented cascading network for compressed sensing of images
Qi et al. Multi-dimensional sparse models
CN105931264B (en) A kind of sea infrared small target detection method
US20170272639A1 (en) Reconstruction of high-quality images from a binary sensor array
CN105761251A (en) Separation method of foreground and background of video based on low rank and structure sparseness
Yang et al. Ensemble learning priors driven deep unfolding for scalable video snapshot compressive imaging
Qu et al. TransFuse: A unified transformer-based image fusion framework using self-supervised learning
Meng et al. Deep unfolding for snapshot compressive imaging
Kim et al. Deeply aggregated alternating minimization for image restoration
Kato et al. Double sparsity for multi-frame super resolution
Yuan et al. SLOPE: Shrinkage of local overlapping patches estimator for lensless compressive imaging
Zhang et al. A separation–aggregation network for image denoising
Li et al. D 3 C 2-Net: Dual-Domain Deep Convolutional Coding Network for Compressive Sensing
CN112784747B (en) Multi-scale eigen decomposition method for hyperspectral remote sensing image
Yang et al. Revisit dictionary learning for video compressive sensing under the plug-and-play framework
Malézieux et al. Dictionary and prior learning with unrolled algorithms for unsupervised inverse problems
CN105427351B (en) Compression of hyperspectral images cognitive method based on manifold structure sparse prior
Yuan et al. Lensless compressive imaging
Amjad et al. Deep learning for inverse problems: Bounds and regularizers
Chang et al. TSRFormer: Transformer based two-stage refinement for single image shadow removal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant