CN107507135A - Image reconstructing method based on coding aperture and target - Google Patents

Abstract

The invention belongs to technical field of computer vision, to propose a kind of improved non-blind point spread function number estimation method, the picture quality after lifting reconstruct.To reach above-mentioned purpose, the present invention adopts the technical scheme that, the image reconstructing method based on coding aperture and drone design, comprises the following steps：Design coding aperture is placed in the nearly detector end of camera lens；Design target is placed at the light cylinder front end of infrared image light source；The point spread function of infrared image is obtained by fuzzy core derivation algorithm model；L is used by analyzing Infrared Image Features model₀Norm constrains as image regulation, and blurred picture is reconstructed recovery according to the point spread function above solved.Present invention is mainly applied to image procossing occasion.

Description

Image reconstruction method based on coded aperture and target

technical field

The invention belongs to the technical field of computer vision, specifically, mainly relates to the fields of digital image processing and computational photography, and is an image reconstruction method based on coded aperture and target design.

Background technique

Nowadays, society has higher and higher requirements for image quality. Compared with visible light imaging systems, imaging systems have been more and more widely used in the fields of national economy, military defense and other fields due to their strong ability to penetrate smoke and water vapor. In space optical imaging, due to factors such as diffraction and aberration of the optical system itself, atmospheric turbulence, relative compound motion between the space camera and the shooting scene, and camera defocus, the image obtained by the camera will be blurred, affecting the image of interest. Interpretation of the target. With the improvement of imaging quality requirements, image deblurring technology has gradually become a research hotspot in the field of optical imaging. Therefore, accurate estimation of point spread function in image deblurring is a key step in image reconstruction.

For the problem of image restoration, traditional methods such as Wiener filtering, inverse filtering, constrained least square filtering, etc., are more sensitive to noise and disturbance of point spread function. In order to overcome the ill-conditioned nature of inverse problem solving, restoration methods based on regularization and partial differential equations have emerged. These methods model the features of the image, apply prior knowledge to regularize the inverse problem, and then find a stable global minimum solution. _The earliest regularization method, such as the Tikhonov regularization method, uses the L2 norm to model the image. They believe that the pixel value distribution of the image satisfies a quadratic function. Although this type of method can remove noise well, it cannot Easy to lose image detail. Aiming at the problem of high complexity and slow operation speed of iterative algorithm. However, many and complex parameters are involved in the above method, and they are all set according to empirical values, and the parameters are not adaptively selected according to the structure and characteristics of the image. Different image restorations only use the same parameter setting, which will inevitably lead to some images not being restored well, or the optimal solution cannot be achieved. In order to solve the problem of inaccurate point spread function estimation in the image deblurring problem, and to better realize image restoration, the present invention uses the coding aperture to design the target to estimate the point spread function, and proposes an improved image reconstruction solution Model.

Contents of the invention

The invention aims at overcoming the deficiencies of the prior art, and provides an improved non-blind spot spread function estimation method to improve the quality of the reconstructed image. In order to achieve the above object, the technical solution adopted by the present invention is that the image reconstruction method based on the design of the coded aperture and the target comprises the following steps: designing the coded aperture and placing it at the end of the camera lens near the detector; designing the target and placing it at the end of the image light source At the front end of the light tube; the point spread function of the image is obtained through the fuzzy kernel solution algorithm model; the L ₂ norm is used as the image regularization constraint, and the blurred image is reconstructed and restored according to the previously solved point spread function.

Design the coding aperture and place it near the end of the detector of the camera lens; design the target and place it at the front end of the light tube of the image light source. The specific steps are:

1) The mathematical model of image blur is expressed as the following formula:

f=h*u+n (1)

Where f is the blurred image, u is the original image, h is the point spread function PSF, n is Gaussian white noise, and * represents convolution. If h is expressed in matrix form H, the formula is expressed as:

f=SHu+n(2)

Among them, S is the matrix form of the camera sampling function, which is the downsampling matrix of the camera itself. The defocus deblurring problem is to estimate the original image u by solving a maximum posterior probability problem, and use the minimum value of the energy function E to solve the maximum posterior probability problem. Experimental probability problem to achieve performance optimization for coded apertures:

in, Indicates the minimum value of the energy function, ||·|| ₂ indicates the L ₂ norm, Represents the noise-to-signal ratio of the image prior, expressed as where σ represents the standard deviation of Gaussian white noise, A _ξ represents the average power spectrum of multiple natural images, μ(u) is a measurement of the original image u in the image space, according to the 1/f rule of natural images, multiple natural images The average power spectrum expression of is:

u is the original image, ξ is the frequency, and u(ξ) is the representation of u on the frequency spectrum _. Performance evaluation criteria for constructing coded apertures:

where H _ξ is the matrix form of the point spread function corresponding to the frequency ξ, for each frequency ξ, It reflects the degree to which the noise is amplified. The optimal aperture has the smallest R(H), which also means that the image reconstruction and restoration results will be more accurate; the genetic algorithm is used to optimize the low-resolution coding aperture, and the gradient descent method is used to improve The resolution of the coded aperture, and then place the designed coded aperture at the near detector end of the camera lens;

2) Design targets with significant geometric features in all directions, respectively place the targets at the front of the light tube of the image light source for subsequent image shooting of different patterns, and use the coded aperture and the target to shoot images to make different patterns Preserve more spectral information of the image.

The point spread function of the image is obtained by solving the algorithm model of the blur kernel _; the L2 norm is used as the image regularization constraint, and the blurred image is reconstructed and restored according to the previously solved point spread function. The lens shoots the light source image of the front design target, the captured image i is planar, and then this mapping is simulated with a planar response w(i), considering the distortion phenomenon in the real camera, define d as geometric distortion function, constructing an image imaging model expressed in the following form:

b=v(d(w(i)))*h+n (6)

b is the captured image, h is the PSF of the lens deviation, v is the halo phenomenon caused by the camera lens, and the original image u to be solved is generated through the distortion of the radiation model, so that

u=v(d(w(i))) (7)

Formula (6) can be transformed into:

b=u*h+n (8) Using the light information of the blur kernel as a priori, then the above problem becomes the following optimization problem:

where _{|||| 2} is the L2 norm, Gradient operator, the first item is a data matching item, λ, μ, γ are parameters, the second and third items are kernel sparse items and kernel smooth items constrained by parameters λ and μ respectively, is the ideal blur kernel The magnitude of the spectral density function of is the PSF spectral density function constraint parameter, F ^* (b) is the conjugate of F(b), the estimated value of PSF is obtained by optimizing formula (10), and the image reconstruction is performed according to the point spread function.

Using the energy function minimization to solve the maximum a posteriori problem to estimate the original image, the image reconstruction solution model is as follows:

in To solve the data item of the model, ||·|| ₂ is the L ₂ norm, J(u) is a smooth item, η is a parameter, Represents finding the minimum value of the energy function. u*h is expressed in matrix form:

u*h=SHu=Au (11)

Where A is equal to SH, and the smoothing term of the solution model, that is, the TV regular term, that is, the total variation term J(u) is expressed as:

_{|||| 2} is the L2 norm, Represents the gradient of u(x), and the gradient Grad J(u) of J(u) obtained by solving the Euler-Lagrange equation is:

Among them, div(·) means to seek divergence, and replace J(u) with a smooth TV regular term, namely J _ε (u), the formula is:

in is replaced by ε represents the variable value, div(·) represents the divergence, then the gradient Grad J _ε (u) of the replaced TV regular term becomes:

Given a point spread function, the L2 norm is used to carry out regularization constraints for image reconstruction, and the solution is obtained according to the Richardon _- Lucy method to obtain the following multiplication iteration formula:

Among them, represents the dot product between elements, t represents the number of iterations, f represents the blurred image, ^AT represents the transposition matrix of A. η is a parameter, Grad J _ε (u) represents the gradient of J _ε (u), and the final original image u is obtained through iterative solution.

Technical characteristics and effects of the present invention:

The method of the present invention aims at the image reconstruction problem in the image restoration work, and proposes an improved reconstruction method. By designing the coding aperture and the imaging target, the detailed information of the image can be retained to the maximum extent, and the non-blind estimation method can be used for more accurate estimation. The point spread function of the image, and reconstruct and restore the blurred image according to the solved point spread function.

1. The program is simple and easy to implement. It can be integrated into the hardware so that the camera or other imaging equipment can directly solve the image point spread function and reconstruct the image, reducing the dependence on external equipment.

2. Self-designed coded aperture and a set of targets with significant geometric features in all directions in space, to preserve image detail information to the maximum extent for subsequent image reconstruction.

3. A point spread function solution model is proposed to obtain the point spread function of the image.

4. Design an image reconstruction optimization model, carry out the image reconstruction of the non-blind estimation of the scene image, and compare it with the blind estimation reconstruction method of the scene image.

Description of drawings

Fig. 1 is the flow chart of actual implementation;

Figure 2 is a schematic diagram of the fuzzy process of the design;

Figure 3 is the designed coded aperture (the rightmost picture is the sleeve used for assembly);

Figure 4 is the designed target pattern. In the figure: (a) commonly used four-column board, (b)-(h) 6 self-made targets;

Fig. 5 is the non-blind estimation flowchart of point spread function;

Fig. 6 is a comparison of image reconstruction results based on point spread function blind estimation and point spread function non-blind estimation. (a) clear target image, (b) blurred target image, (c) image reconstruction results of blind estimation, (d) image reconstruction results of non-blind estimation

detailed description

The technical scheme adopted by the invention is an image reconstruction method based on coded aperture and target design. The performance evaluation standard of the coded aperture imaging system is adopted, and the coded aperture model is optimized by genetic algorithm and coordinate descent method according to the evaluation standard, and the coded aperture is produced. At the same time, in order to retain the detailed information of the image to the greatest extent, design target patterns with significant geometric features in all directions, construct a function model for point spread function estimation, use the L2 _norm as the image regularization constraint, and according to the previously solved points The diffusion function is used to reconstruct and restore the blurred image.

1) The mathematical model of image blur can be expressed as the following formula:

f=h*u+n (1)

Where f is the blurred image, u is the original image, h is the point spread function, n is Gaussian white noise, and * stands for convolution. If h is expressed as a matrix form H, the formula can be expressed as:

f＝SHu+n (2)

Among them, S is the matrix form of the camera sampling function, which is the downsampling matrix of the camera itself. The defocus deblurring problem is to estimate the original image u by solving a maximum posterior probability problem, and use the minimum value of the energy function E to solve the maximum posterior probability problem. probabilistic problems to optimize the performance of coded apertures.

in, Indicates the minimum value of the energy function, ||·|| ₂ indicates the L ₂ norm, Represents the noise-to-signal ratio of the image prior, which can be expressed as σ ² /A _ξ , where σ represents the standard deviation of Gaussian white noise, and A _ξ represents the average power spectrum of multiple natural images. μ(u) is a measurement of the original image u in the image space. According to the 1/f rule of natural images, the average power spectrum of multiple natural images can be expressed as:

u is the original image, ξ is the frequency, u(ξ) is the representation of u on the frequency spectrum, and the performance evaluation standard of the coded aperture is constructed:

where H _ξ is the matrix form of the point spread function corresponding to the frequency ξ, for each frequency ξ, Reflects the degree to which the noise is amplified. The best aperture has the smallest R(H), designed and optimized using a genetic algorithm to obtain a low-resolution coded aperture. For an aperture with a resolution of L×L, there are as many as 2 ^L×L possibilities, and it is impossible to directly use high resolution to optimize the selection process for such a variety of possibilities. To do this, first use the search method of the genetic algorithm:

A. g=0, g represents the number of iterations of the genetic algorithm, and the initial number of iterations is 0; generate K=4000 initial random binary sequences with a length of L×L, that is, a set containing only 0 and 1, where L=11.

B. iterative process, carry out loop iterative process from g=1:G, wherein G=60

a. The optimal selection process in the genetic algorithm, for each binary sequence b, reintegrate it into an aperture matrix with a resolution of L×L. Evaluation is performed using formula (5), and the best Z=400 results are selected as the optimal solution set of one iteration result.

b. Repeat the following calculation process in this step until the number of sequence sets recovers from Z to K again. Crossover: Randomly select and copy two subsequences from the optimal set Z in step a, then align and sort the two subsequences, exchange bit values with probability p ₁ =0.2, and obtain two new subsequences . Mutation: For the two new sequences obtained in the previous step, flip the value of each bit of each new sequence with probability p ₂ =0.06.

C. Calculate the performance evaluation values of all the remaining sequences, and obtain the optimal result of the genetic algorithm step. For the low-resolution coded aperture obtained by the genetic algorithm, the resolution of the coded aperture is increased by the gradient descent method to obtain a high-resolution coded aperture:

A. Use the bicubic interpolation method to gradually increase the low-resolution coding aperture to high-resolution. Such as increasing from 11×11 to 14×14 for the first time.

B. Use the coordinate descent method to optimize the performance of the coded aperture, perform a binary flip on each element of the coded aperture matrix along the horizontal and vertical dimensions, and use the performance evaluation standard to evaluate and calculate the performance value of the newly generated coded aperture . Only one element value of the coding aperture matrix is changed at a time, until each element of the coding aperture matrix is traversed, and the result with the optimal aperture performance value is retained. The above iterative process is repeated until the performance evaluation standard index of the coded aperture converges and the aperture does not change any more.

C. Repeat steps A and B until the performance evaluation standard index of the coded aperture converges as the resolution increases, and the coded aperture no longer changes significantly, and the resolution of the coded aperture finally increases from a low resolution of 11×11 and converges to a high resolution 47 x 47. So far the final coded aperture pattern is obtained.

Make the coded aperture out of a stainless steel sheet according to the pattern, then place the coded aperture at the near detector end of the camera lens.

2) Due to the lack of geometric feature information in all directions, it is difficult to estimate a reliable point spread function from the target imaging of the collimator in the traditional four-panel target; for this purpose, CAD (computer-aided design) software is used to design a significant geometric feature in all directions. For the characteristic target pattern, targets of various styles are made of stainless steel sheets and placed in front of the image light source for subsequent image capture.

3) Estimating the point spread function through an improved non-blind method: For traditional blind estimation, construct a fuzzy kernel optimization model:

where |||| ₂ is the L2 norm, _f is the blurred image, s is the sampling function, u is the original image, h is the point spread function, * stands for convolution, and γ is the balance parameter. The result of blur kernel estimation based on the pixel intensity value directly solved by the above formula is inaccurate, so h is estimated in the gradient domain:

where q ₂ =s*u, Indicates the gradient operator, and || _· || ₂ is the L2 norm. The corresponding algorithm is as follows:

A. Input: blurred image f

B. Let i=1→5

a. to solve Get u, where P(u) is constrained, and λ is a parameter;

b. solving formula (6) to obtain fuzzy kernel h;

c.λ←max{λ/1.1,1e ^-4 };

C. Output: The blur kernel h and the underlying original image u produced by the intermediate process.

After the estimated kernel is obtained, the negative value in the kernel is reset to 0, and h is standardized so that the element sum is 1. In the non-blind estimation of the point spread function proposed by the present invention, the convergence of light rays is through the lens in front of the focal plane, assuming that the captured image i is planar, and then this mapping can be simulated with a planar response w(i), considering that in the real The distortion phenomenon existing in the camera, define d as the geometric distortion function, then this image imaging model can be expressed as the following form:

b=v(d(w(i)))*h+n (8)

b is the captured image, h is the PSF of the lens deviation, v is the halo phenomenon caused by the camera lens, n is the Gaussian noise, and the original image u to be solved is generated by the distortion of the radiation model, so that

u=v(d(w(i))) (9)

Equation (8) can be changed to:

b=u*h+n(10)

Using the light information of the blur kernel as a priori, then the above problem can be transformed into the following optimization problem:

where _{|||| 2} is the L2 norm, Gradient factor, the first item is the data matching item, λ, μ, γ are the parameters, the second and third items are the kernel sparse item and the kernel smooth item constrained by the parameters λ and μ respectively. is the ideal blur kernel The magnitude of the spectral density function of ( F ^* (b) is the conjugate of F(b)), which is the spectral density function constraint parameter of PSF. The estimated value of PSF is obtained by optimizing formula (12). For the blurred image and the original image of the same scale, the blurred image can be regarded as the result of the convolution of the original image and the blur kernel. The mathematical language under the ideal situation (without considering noise) is expressed as:

f=u*h (12)

Among them, u is the original image, h is the point spread function of the blur kernel, and f is the observed blurred image. In order to ensure the accuracy of non-blind convolution kernel estimation, the blurred image and GroundTruth (GT) image are first preprocessed, including sample-by-sample mean reduction and feature standardization, so as to ensure the stationarity of the data. Second, the blurred image and the GT image are calibrated through rotation, translation and scaling operations to ensure image consistency. Considering the solution feasibility and algorithm complexity, the fuzzy kernel solution algorithm model is converted to the frequency domain by using Fourier transform, and the transformation from the spatial deconvolution operation to the frequency domain point division operation is completed. The results of the fuzzy kernel solving algorithm model obtained in the domain domain are converted to the time domain to obtain the final point spread function, and the image reconstruction is performed according to the point spread function.

4) Use energy function minimization to solve the maximum a posteriori problem to estimate the original image. The image reconstruction solution model proposed by the present invention is as follows:

in To solve the data item of the model, ||·|| ₂ is the L ₂ norm, J(u) is a smooth item, η is a parameter, Represents finding the minimum value of the energy function. u*h is expressed in matrix form:

u*h=SHu=Au (14)

Where A is equal to SH, and the smoothing term of the solution model, that is, the TV regular term (total variation term) J(u) is expressed as:

_{|||| 2} is the L2 norm, Represents the gradient of u(x), and the gradient Grad J(u) of J(u) can be obtained by solving the Euler-Lagrange equation:

Among them, div( ) means seeking divergence. Since the gradient of the TV regular term is used directly at pixel x is not defined, it is difficult to directly minimize with the TV regular term, so the smooth TV regular term, namely J _ε (u) is used to replace J(u), the formula is:

in is replaced by ε represents the variable value, div(·) represents the divergence, then the gradient Grad J _ε (u) of the replaced TV regular term becomes:

Given a point spread function, the L ₂ norm is used to carry out regularization constraints for image reconstruction, and the solution is obtained according to the Richardon-Lucy (RL) method to obtain the following multiplication iteration formula:

Among them, represents the dot product between elements, t represents the number of iterations, f represents the blurred image, ^AT represents the transposition matrix of A. η is a parameter, and Grad J _ε (u) represents the gradient of J _ε (u). The final original image u is obtained by iterative solution.

The present invention proposes an image reconstruction method based on a coded aperture and a target (as shown in the flow chart of FIG. 1 ). Below in conjunction with accompanying drawing and embodiment describe in detail as follows:

1) The blurring process of the image is shown in Figure 2. The mathematical model of image blurring can be expressed as the following formula:

f=h*u+n (1)

Where f is the blurred image, u is the original image, h is the PSF (point spread function), n is Gaussian white noise, and * stands for convolution. If h is expressed as a matrix form H, the formula can be expressed as:

f＝SHu+n (2)

Among them, S is the matrix form of the camera sampling function, which is the downsampling matrix of the camera itself. The defocus deblurring problem is to estimate the original image u by solving a maximum posterior probability problem, and use the minimum value of the energy function E to solve the maximum A posterior probability problem for performance optimization of coded apertures.

in, Indicates to find the minimum value of the energy function, ||·|| ₂ is the L ₂ norm, Represents the noise-to-signal ratio of the image prior, which can be expressed as σ ² /A _ξ , where σ represents the standard deviation of Gaussian white noise, and A _ξ represents the average power spectrum of multiple natural images. μ(u) is a measurement of the original image u in the image space. According to the 1/f rule of natural images, the average power spectrum of multiple natural images can be expressed as:

u is the original image, ξ is the frequency, u(ξ) is the representation of u on the frequency spectrum, and the performance evaluation standard of the coded aperture is constructed:

where H _ξ is the matrix form of the point spread function corresponding to the frequency ξ, for each frequency ξ, Reflecting the degree to which the noise is amplified, the best aperture has the smallest R(H), and is designed and optimized using a genetic algorithm to obtain a low-resolution coded aperture. For an aperture with a resolution of L×L, there are as many as 2 ^L×L possibilities, and it is impossible to directly use high resolution to optimize the selection process for such a variety of possibilities. First use the search method of the genetic algorithm:

C.g=0, g represents the number of iterations of the genetic algorithm, the initial number of iterations is 0; generate K=4000 initial random binary sequences with a length of L×L, that is, a set containing only 0,1, where L=11.

D. iterative process, from g=1:G to carry out loop iterative process, wherein G=60

c. The optimization selection process in the genetic algorithm, for each binary sequence b, reintegrate it into an aperture matrix with a resolution of L×L. Evaluation is performed using formula (5), and the best Z=400 results are selected as the optimal solution set of one iteration result.

d. Repeat the following calculation process in this step until the number of sequence sets recovers from Z to K again. Crossover: Randomly select and copy two subsequences from the optimal set Z in step a, then align and sort the two subsequences, exchange bit values with probability p ₁ =0.2, and obtain two new subsequences . Mutation: For the two new sequences obtained in the previous step, flip the value of each bit of each new sequence with probability p ₂ =0.06.

C. Calculate the performance evaluation standard value of all the remaining sequences, and obtain the optimal result of the genetic algorithm step.

For the low-resolution coded aperture obtained by the genetic algorithm, the resolution of the coded aperture is increased by the gradient descent method to obtain a high-resolution coded aperture:

D. Use the bicubic interpolation method to gradually increase the low-resolution coding aperture to high-resolution. Such as increasing from 11×11 to 14×14 for the first time

E. Use the coordinate descent method to optimize the performance of the coded aperture, perform a binary flip on each element of the coded aperture matrix along the horizontal and vertical dimensions, and use the performance evaluation standard formula to evaluate the performance of the newly generated coded aperture. value. Only one element value of the coding aperture matrix is changed at a time, until each element of the coding aperture matrix is traversed, and the result with the optimal aperture performance value is retained. The above iterative process is repeated until the performance evaluation standard index of the coded aperture converges and the aperture does not change any more.

F. Repeat steps A and B until the performance evaluation standard index of the coded aperture converges as the resolution increases, and the coded aperture no longer changes significantly, and the resolution of the coded aperture finally increases from a low resolution of 11×11 and converges to a high resolution 47 x 47. So far the final coded aperture pattern is obtained.

Use a stainless steel sheet to make the coded aperture as shown in Figure 3 according to the pattern, and then place the coded aperture at the end of the camera lens near the detector.

2) Due to the lack of geometric feature information in all directions, it is difficult to estimate a reliable point spread function from the target imaging of the collimator for the traditional four-panel target (Fig. 4a); therefore, CAD (computer-aided design) software is used to design There are targets with significant geometric features in all directions. Various styles of targets are made of stainless steel sheets and placed in front of the image light source for subsequent image capture. The designed patterns are shown in Figure 4b~h. In order to get as large a shooting pattern as possible, the target patterns we designed this time are as close as possible to the edge of the target. These targets will be imaged and point spread function estimates will be performed on the acquisitions.

3) Estimating the point spread function through an improved non-blind method: For the traditional blind estimation method, construct a fuzzy kernel optimization model:

where |||| ₂ is the L2 norm, _f is the blurred image, s is the sampling function of the camera, u is the original image, h is the point spread function, * stands for convolution, and γ is the balance parameter. The result of blur kernel estimation based on the pixel intensity value directly solved by the above formula is inaccurate, so h is estimated in the gradient domain:

where q ₂ =s*u, Indicates the gradient operator, and || _· || ₂ is the L2 norm. The corresponding algorithm is as follows:

A. Input: blurred image f;

B. Let i=1→5

a. to solve Get u, where P(u) is constrained, and λ is a parameter;

b. Solve formula (14) to get the blur kernel h

c.λ←max{λ/1.1,1e ^-4 }

C. Output: The blur kernel h and the underlying original sharp image u produced by the intermediate process.

After the estimated kernel is obtained, the negative value in the kernel is reset to 0, and h is standardized so that the element sum is 1. In the non-blind estimation of the point spread function proposed by the invention, the convergence of light rays is through the lens in front of the focal plane, assuming that the captured image i is planar, then this mapping can be simulated with a planar response w(i), considering that in the real The distortion phenomenon existing in the camera, define d as the geometric distortion function, then this image imaging model can be expressed as the following form:

b=v(d(w(i)))*h+n (8)

b is the captured image, h is the PSF of the lens deviation, v is the halo phenomenon caused by the camera lens, and n is the Gaussian noise. Generating u, the original image to be solved for, by radiation model warping, let

u=v(d(w(i))) (9)

Equation (8) can be changed to:

b=u*h+n (10)

Using the light information of the blur kernel as a priori, then the above problem can be transformed into the following optimization problem:

where _{|||| 2} is the L2 norm, Gradient factor, the first item is the data matching item, λ, μ, γ are parameters, the second and third items are the kernel sparse item and kernel smooth item constrained by the parameters λ and μ respectively. is the ideal blur kernel The magnitude of the spectral density function of ( F ^* (b) is the conjugate of F(b)), which is the spectral density function constraint parameter of PSF. The estimated value of PSF is obtained by optimizing formula (12). For blurred images and clear images of the same scale, the blurred image can be regarded as the result of the convolution of the original image and the blurred kernel. The mathematical language under ideal conditions (without considering noise) is expressed as:

f=u*h (12)

Among them, u is the original image, h is the blur kernel, and f is the observed blurred image. As shown in Figure 5, in order to ensure the accuracy of non-blind convolution kernel estimation, the blurred image and GT image in the project are first preprocessed, including sample-by-sample mean reduction and feature standardization, so as to ensure the stability of the data. Second, the blurred image and the GT image are calibrated through rotation, translation and scaling operations to ensure image consistency. Considering the solution feasibility and algorithm complexity, the fuzzy kernel solution algorithm model is converted to the frequency domain by using Fourier transform, and the transformation from the spatial deconvolution operation to the frequency domain point division operation is completed. The results of the fuzzy kernel solving algorithm model obtained in the domain domain are converted to the time domain to obtain the final point spread function, and the image reconstruction is performed according to the point spread function.

4) Use energy function minimization to solve the maximum a posteriori problem to estimate the original image. The image reconstruction solution model proposed by the present invention is as follows:

in To solve the data item of the model, ||·|| ₂ is the L ₂ norm, J(u) is a smooth item, η is a parameter, Represents finding the minimum value of the energy function. u*h is expressed in matrix form:

u*h=SHu=Au (14)

Where A is equal to SH, and the smoothing term of the solution model, that is, the TV regular term (total variation term) J(u) is expressed as:

_{|||| 2} is the L2 norm, Represents the gradient of u(x), and the gradient Grad J(u) of J(u) can be obtained by solving the Euler-Lagrange equation:

Among them, div( ) means seeking divergence. Since the gradient of the TV regular term is used directly at pixel x is not defined, it is difficult to directly minimize with the TV regular term, so the smooth TV regular term, namely J _ε (u) is used to replace J(u), the formula is:

in is replaced by ε represents the variable value, div(·) represents the divergence, then the gradient Grad J _ε (u) of the replaced TV regular term becomes:

Given a point spread function, the L ₂ norm is used to carry out regularization constraints for image reconstruction, and the solution is obtained according to the Richardon-Lucy (RL) method to obtain the following multiplication iteration formula:

Among them, represents the dot product between elements, t represents the number of iterations, f represents the blurred image, ^AT represents the transposition matrix of A. η is a parameter, and Grad J _ε (u) represents the gradient of J _ε (u). The final original image u is obtained by iterative solution.

In the actual scene, a camera equipped with a coded aperture is used to shoot different target patterns to obtain blurred images, and the image reconstruction results are compared. From the experimental results in FIG. 6 , it can be seen that the non-blind estimation of the point spread function proposed by the present invention can effectively eliminate noise interference and obtain a clearer image reconstruction result. In the image reconstruction results based on the non-blind estimation of the point spread function, it can be seen that the reconstruction results basically restore the geometric features of the clear target image, and the edges are relatively sharp.

Claims (2)

1. An image reconstruction method based on coded aperture and target design, it is characterized in that, comprises the following steps: design coded light and be placed in camera lens near detector end; Design target and be placed in the light tube front end of image light source; The point spread function of the image is obtained by solving the algorithm model of the blur kernel; the L ₂ norm is used as the image regularization constraint, and the blurred image is reconstructed and restored according to the previously solved point spread function. 2. the image reconstruction method based on coded aperture and target design as claimed in claim 1, is characterized in that, design coded aperture and place near detector end of camera lens; Design target and place at the light tube front end of image light source , the specific steps are: 1) The mathematical model of image blur is expressed as the following formula: f=h*u+n (1) Where f is the blurred image, u is the original image, h is the point spread function PSF, n is Gaussian white noise, and * represents convolution. If h is expressed in matrix form H, the formula is expressed as: f＝SHu+n (2) Among them, S is the matrix form of the camera sampling function, which is the downsampling matrix of the camera itself. The defocus deblurring problem is to estimate the original image u by solving a maximum posterior probability problem, and use the minimum value of the energy function E to solve the maximum posterior probability problem. Experimental probability problem to achieve performance optimization for coded apertures: <mrow><mtable><mtr><mtd><mrow><mi>u</mi><mo>=</mo><munder><mrow><mi>arg</mi><mi>min</mi></mrow><mi>u</mi></munder><mi>E</mi><mrow><mo>(</mo><mi>u</mi><mo>|</mo><mi>f</mi><mo>,</mo><mi>h</mi><mo>)</mo></mrow></mrow></mtd></mo>mtr><mtr><mtd><mrow><mo>=</mo><munder><mrow><mi>arg</mi><mi>min</mi></mrow><mi>u</mi></munder><mo>|</mo><mo>|</mo><mi>u</mi><mo>*</mo><mi>h</mi><mo>-</mo><mi>f</mi><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup><mo>+</mo><mo>|</mo><mo>|</mo><mi>C</mi><mo>&amp;CenterDot;</mo><mi>u</mi><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup></mrow></mtd></mtr></mtable><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mrow> in, Indicates the minimum value of the energy function, ||·|| ₂ indicates the L ₂ norm, Represents the noise-to-signal ratio of the image prior, expressed as σ ² /A _ξ , where σ represents the standard deviation of Gaussian white noise, A _ξ represents the average power spectrum of multiple natural images, μ(u) is the original image u in the image space A measurement of , according to the 1/f law of natural images, the average power spectrum expression of multiple natural images is: <mrow><msub><mi>A</mi><mi>&amp;xi;</mi></msub><mo>=</mo><munder><mo>&amp;Integral;</mo><mi>u</mi></munder><mo>|</mo><mi>u</mi><mrow><mo>(</mo><mi>&amp;xi;</mi><mo>)</mo></mrow><msup><mo>|</mo><mn>2</mn></msup><mi>&amp;mu;</mi><mrow><mo>(</mo><mi>u</mi><mo>)</mo></mrow><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>4</mn><mo>)</mo></mrow></mrow> u is the original image, ξ is the frequency, u(ξ) is the representation of u on the frequency spectrum, and the performance evaluation standard of the coded aperture is constructed: <mrow><mi>R</mi><mrow><mo>(</mo><mi>H</mi><mo>)</mo></mrow><mo>=</mo><munder><mo>&amp;Sigma;</mo><mi>&amp;xi;</mi></munder><mfrac><msup><mi>&amp;sigma;</mi><mn>2</mn></msup><mrow><mo>|</mo><msub><mi>H</mi><mi>&amp;xi;</mi></msub><msup><mo>|</mo><mn>2</mn></msup><mo>+</mo><msup><mi>&amp;sigma;</mi><mn>2</mn></msup><mo>/</mo><msub><mi>A</mi><mi>&amp;xi;</mi></msub></mrow></mfrac><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>5</mn><mo>)</mo></mrow></mrow> where H _ξ is the matrix form of the point spread function corresponding to the frequency ξ, for each frequency ξ, It reflects the degree to which the noise is amplified. The optimal aperture has the smallest R(H), which also means that the image reconstruction and restoration results will be more accurate; the genetic algorithm is used to optimize the low-resolution coding aperture, and the gradient descent method is used to improve The resolution of the coded aperture, and then place the designed coded aperture at the near detector end of the camera lens; 2) Design targets with significant geometric features in all directions, respectively place the targets at the front of the light tube of the image light source for subsequent image shooting of different patterns, and use the coded aperture and the target to shoot images to make different patterns Preserve more spectral information of the image. The point spread function of the image is obtained by solving the algorithm model of the blur kernel _; the L2 norm is used as the image regularization constraint, and the blurred image is reconstructed and restored according to the previously solved point spread function. The lens shoots the light source image of the front design target, the captured image i is planar, and then this mapping is simulated with a planar response w(i), considering the distortion phenomenon in the real camera, define d as geometric distortion function, constructing an image imaging model expressed in the following form: b=v(d(w(i)))*h+n (6) b is the captured image, h is the PSF of the lens deviation, v is the halo phenomenon caused by the camera lens, and the original image u to be solved is generated through the distortion of the radiation model, so that u=v(d(w(i))) (7) Formula (6) can be transformed into: b=u*h+n (8) Using the light information of the blur kernel as a priori, the above problem becomes the following optimization problem: <mrow><munder><mrow><mi>m</mi><mi>i</mi><mi>n</mi></mrow><mi>h</mi></munder><mi>E</mi><mrow><mo>(</mo><mi>h</mi><mo>)</mo></mrow><mo>=</mo><mo>|</mo><mo>|</mo><mover><mi>u</mi><mo>^</mo></mover><mi>h</mi><mo>-</mo><mover><mi>b</mi><mo>^</mo></mover><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup><mo>+</mo><mi>&amp;lambda;</mi><mo>|</mo><mo>|</mo><mi>h</mi><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup><mo>+</mo><mi>&amp;mu;</mi><mo>|</mo><mo>|</mo><mo>&amp;dtri;</mo><mi>h</mi><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup><mo>+</mo><mi>&amp;gamma;</mi><mo>|</mo><mo>|</mo><mo>|</mo><mi>F</mi><mrow><mo>(</mo><mi>h</mi><mo>)</mo></mrow><mo>|</mo><mo>-</mo><mo>|</mo><mi>F</mo>mi><mo>(</mo><mover><mi>h</mi><mo>^</mo></mover><mo>)</mo><mo>|</mo><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup><mo>,</mo><mi>s</mi><mo>.</mo><mi>t</mi><mo>.</mo><mi>h</mi><mo>&amp;GreaterEqual;</mo><mn>0</mn><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>9</mn><mo>)</mo></mrow></mrow> where _{|||| 2} is the L2 norm, Gradient operator, the first item is a data matching item, λ, μ, γ are parameters, the second and third items are kernel sparse items and kernel smooth items constrained by parameters λ and μ respectively, is the ideal blur kernel The magnitude of the spectral density function of is the PSF spectral density function constraint parameter, F*(b) is the conjugate of F(b), the estimated value of PSF is obtained by optimizing formula (10), and image reconstruction is performed according to the point spread function. Using the energy function minimization to solve the maximum a posteriori problem to estimate the original image, the image reconstruction solution model is as follows: <mrow><munder><mi>argmin</mi><mi>u</mi></munder><mo>|</mo><mo>|</mo><mi>u</mi><mo>*</mo><mi>h</mi><mo>-</mo><mi>f</mi><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup><mo>+</mo><mi>&amp;eta;</mi><mi>J</mi><mrow><mo>(</mo><mi>u</mi><mo>)</mo></mrow><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>10</mn><mo>)</mo></mrow></mrow> in To solve the data item of the model, ||·|| ₂ is the L ₂ norm, J(u) is a smooth item, η is a parameter, Represents finding the minimum value of the energy function. u*h is expressed in matrix form: u*h=SHu=Au (11) Where A is equal to SH, and the smoothing term of the solution model, that is, the TV regular term, that is, the total variation term J(u) is expressed as: <mrow><mi>J</mi><mrow><mo>(</mo><mi>u</mi><mo>)</mo></mrow><mo>=</mo><mo>&amp;Integral;</mo><mo>|</mo><mo>|</mo><mo>&amp;dtri;</mo><mi>u</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>|</mo><msub><mo>|</mo><mn>2</mn></msub><mi>d</mi><mi>x</mi><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>12</mn><mo>)</mo></mrow></mrow> _{|||| 2} is the L2 norm, Represents the gradient of u(x), and the gradient Grad J(u) of J(u) obtained by solving the Euler-Lagrange equation is: <mrow><mi>G</mi><mi>r</mi><mi>a</mi><mi>d</mi><mi></mi><mi>J</mi><mrow><mo>(</mo><mi>u</mi><mo>)</mo></mrow><mo>=</mo><mi>d</mi><mi>i</mi><mi>v</mi><mrow><mo>(</mo><mfrac><mrow><mo>&amp;dtri;</mo><mi>u</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow></mrow><mrow><mo>|</mo><mo>|</mo><mo>&amp;dtri;</mo><mi>u</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>|</mo><msub><mo>|</mo><mn>2</mn></msub></mrow></mfrac><mo>)</mo></mrow><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>13</mn><mo>)</mo></mrow></mrow> Among them, div(·) means to seek divergence, and replace J(u) with a smooth TV regular term, namely J _ε (u), the formula is: <mrow><msub><mi>J</mi><mi>&amp;epsiv;</mi></msub><mrow><mo>(</mo><mi>u</mi><mo>)</mo></mrow><mo>=</mo><mo>&amp;Integral;</mo><msqrt><mrow><msup><mi>&amp;epsiv;</mi><mn>2</mn></msup><mo>+</mo><mo>|</mo><mo>|</mo><mo>&amp;dtri;</mo><mi>u</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>|</mo><msubsup><mo>|</mo><mn>2</mn><mn>2</mn></msubsup></mrow></msqrt><mi>d</mi><mi>x</mi><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>14</mn><mo>)</mo></mrow></mrow> in is replaced by ε represents the variable value, div(·) represents the divergence, then the gradient Grad J _ε (u) of the replaced TV regular term becomes: <mrow><mi>G</mi><mi>r</mi><mi>a</mi><mi>d</mi><mi></mi><msub><mi>J</mi><mi>&amp;epsiv;</mi></msub><mrow><mo>(</mo><mi>u</mi><mo>)</mo></mrow><mo>=</mo><mi>d</mi><mi>i</mi><mi>v</mi><mrow><mo>(</mo><mfrac><mrow><mo>&amp;dtri;</mo><mi>u</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow></mrow><msqrt><mrow><msup><mi>&amp;epsiv;</mi><mn>2</mn></msup><mo>+</mo><mo>|</mo><mo>|</mo><mo>&amp;dtri;</mo><mi>u</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>|</mo><msub><mo>|</mo><mn>2</mn></msub></mrow></msqrt></mfrac><mo>)</mo></mrow><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>15</mn><mo>)</mo></mrow></mrow> Given a point spread function, the L2 norm is used to carry out regularization constraints for image reconstruction, and the solution is obtained according to the Richardon _- Lucy method to obtain the following multiplication iteration formula: <mrow><msup><mi>u</mi><mrow><mi>t</mi><mo>+</mo><mn>1</mn></mrow></msup><mo>=</mo><msup><mi>u</mi><mi>t</mi></msup><mo>&amp;CenterDot;</mo><msup><mi>A</mi><mi>T</mi></msup><mrow><mo>(</mo><mfrac><mi>f</mi><mrow><msup><mi>Au</mi><mi>t</mi></msup></mrow></mfrac><mo>)</mo></mrow><mfrac><msup><mi>u</mi><mi>t</mi></msup><mrow><mn>1</mn><mo>-</mo><mi>&amp;eta;</mi><mi>G</mi><mi>r</mi><mi>a</mi><mi>d</mi><mi></mi><msub><mi>J</mi><mi>&amp;epsiv;</mi></msub><mrow><mo>(</mo><mi>u</mi><mo>)</mo></mrow></mrow></mfrac><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>16</mn><mo>)</mo></mrow></mrow> Among them, represents the dot product between elements, t represents the number of iterations, f represents the blurred image, ^AT represents the transposition matrix of A. η is a parameter, Grad J _ε (u) represents the gradient of J _ε (u), and the final original image u is obtained through iterative solution.