CN104091312B - A kind of simple lens formation method according to image spectrum information extraction fuzzy core priori - Google Patents
A kind of simple lens formation method according to image spectrum information extraction fuzzy core priori Download PDFInfo
- Publication number
- CN104091312B CN104091312B CN201410331056.5A CN201410331056A CN104091312B CN 104091312 B CN104091312 B CN 104091312B CN 201410331056 A CN201410331056 A CN 201410331056A CN 104091312 B CN104091312 B CN 104091312B
- Authority
- CN
- China
- Prior art keywords
- mrow
- msub
- image
- msup
- msubsup
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000001228 spectrum Methods 0.000 title claims abstract description 16
- 238000000605 extraction Methods 0.000 title claims abstract description 4
- 230000015572 biosynthetic process Effects 0.000 title abstract 2
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 46
- 238000005457 optimization Methods 0.000 claims abstract description 22
- 238000003384 imaging method Methods 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 23
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 8
- 238000009826 distribution Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 13
- 238000012545 processing Methods 0.000 abstract description 6
- 230000004075 alteration Effects 0.000 description 9
- 238000013461 design Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 5
- 230000011514 reflex Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 239000006185 dispersion Substances 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000012634 optical imaging Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013179 statistical model Methods 0.000 description 2
- 206010010071 Coma Diseases 0.000 description 1
- 201000009310 astigmatism Diseases 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The present invention relates to is image processing field, specifically discloses a kind of simple lens formation method according to image spectrum information extraction fuzzy core priori, obtains blurred picture by simple lens camera; L is added in existing objective optimization function
1/ l
2image prior and the fuzzy core priori extracted according to blurred picture self information; Recycle the fuzzy core based on the blind convolution algorithm estimate sheet lens camera of maximum a posteriori probability; In conjunction with estimated by fuzzy core, utilize non-blind convolution algorithm to obtain picture rich in detail.The present invention can realize simple lens high-quality by said method and be calculated to be picture, and the fuzzy core priori adopted can make full use of obscuring image information, therefore has good deblurring effect.
Description
The technical field is as follows:
the invention belongs to the field of image processing, mainly relates to image deblurring, and particularly relates to a single-lens imaging method for extracting fuzzy kernel prior according to image frequency spectrum information.
Background art:
the Single Lens is the most basic device of a modern optical imaging system such as a Single Lens Reflex (SLR) Lens, and as the requirement of people for image quality is continuously improved, the precision of the Lens is also increased. Increasingly complex lens designs aim to compensate for Aberrations (Aberrations) caused by a single lens, i.e. deviations between the actual optical system and the ideal optical system, which result in image blur. Optical aberrations include chromatic aberration, spherical aberration, coma, astigmatism, field curvature, distortion, and the like [1 ]. All single lenses with spherical shapes generate the optical aberration, and cannot be directly applied to high-resolution and high-quality photography, so that the design of an optical imaging system needs to balance parameters such as aperture size, focal length, image distortion limitation, lens cost and the like. The lens of the single lens reflex may include a combination of tens of lenses, but the design of the combined lens greatly increases the cost of the lens while improving the imaging quality, and the lens is the most expensive component of the single lens reflex at present. And the design of the combined lens also increases the volume and the weight of the lens. The single lens reflex camera has the advantages of high price and heavy volume, and the factors influence the popularization of the single lens reflex camera to a large-area user and bring inconvenience to the daily life of people. Therefore, how to reduce the size and manufacturing cost of the slr lens while maintaining high-quality imaging is one of the mainstream directions for the slr lens design in the future.
In recent years, with the development of advanced technologies such as computers, optics, sensors and the like and basic theories such as computer vision and the like, a new photographing technology is gradually formed, namely computational photography. Computational photography combines a large number of calculations, digital sensors, modern optics, actuators, detectors, and smart lighting to get rid of the limitations of traditional cameras and enable the creation of novel image applications. Inspired by computer camera technology, single lens imaging has become a new direction for lens design research in recent years [2 ]. As shown in fig. 1, a common single lens is used to replace a complex optical device, and the image blurring generated by the single lens is eliminated by a mature computational photography technique such as image restoration, so that the imaging effect of the complex lens can be achieved or approached.
As shown in fig. 2, for a blurred image to be deblurred, a flow chart of an existing image restoration algorithm is generally that a blur kernel is estimated through a blind convolution image restoration algorithm, in the process, generally, a relatively simple non-blind convolution image restoration algorithm is used, an iterative optimization method is used for alternately estimating the blur kernel and a potential sharp image, and then a relatively complex non-blind convolution image restoration algorithm is used for obtaining a final deblurred sharp image.
A more commonly used blind convolution image restoration algorithm is a blind convolution image restoration method based on the maximum a posteriori probability, and under the framework of the maximum a posteriori probability, a statistical model of the blind restoration problem can be expressed as:
where b is the blurred image, l is the sharp image, and k is the blur kernel. It is assumed here that l, b, k are independent of each other and that all information is known in view of blurred images, so that
p(k,l|b)∝p(b|l,k)p(l)p(k) (2)
In equation (2), the term on the left is the expected a posteriori estimate. If the negative logarithm is taken at the same time for the left and right sides of the formula (2), then the result is obtained
-logp(k,l|b)＝-logp(b|k,l)-logp(l)-logp(k) (3)
Can be more clearly expressed as in formula (3)
Equation (4) is the target optimization function of the blind convolution image restoration algorithm based on the maximum a posteriori probability. Wherein the first term represents the constraint on image noise, and the second term and the third term represent the prior knowledge of the natural image and the blur kernel, respectively, also referred to as the regularization term. In a general blind convolution image restoration algorithm, a proper image prior and a proper fuzzy kernel prior are added to ensure that an objective optimization function is quickly and accurately converged to an optimal solution.
There are mature image priors, such as Total Variation (TV) image priors [3 ]]hyper-Laplacian image prior [4 [ ]]And l1/l2Image priors [5]And the like.
Wherein l1/l2The image prior effect is better because: the main disadvantage of the image prior in the existing blind convolution recovery algorithm is that the solution of the target optimization function is not consistent with the actual image, and l1/l2Image priors can ensure that an optimized solution close to a real image is obtained. Of dimensional space variation1Norm is often used in the field of image processing to limit image high frequency information, for image denoising, for noise at image high frequencies by reducing l1The norm may remove a portion of the noise. But deblurring the image is not suitable for reducing l1The norm removes the ambiguity. Because the constraint of the blur kernel is loose in the blind convolution image restoration, if l is reduced1The norm will cause the image to be more blurred. The simplest method is to introduce l1/l2Norm, which can be seen as l1Normalization of norm by reducing l1、l2When norm,/2Norm decreasing speed ratio l1The norm decreases more rapidly, so1/l2The norm is increased as a whole, thus ensuring l1/l2The norm scale space is invariant.
Current blur kernel priors mainly include basic assumptions about the blur kernel such as that the blur kernel must be non-negative, energy preserving, and smooth. In addition, or a priori of the blur kernel is obtained according to the image statistical characteristics, such as assuming that the blur kernel follows gaussian distribution or satisfies a certain sparse condition [6 ]. From the above, although there is a good image prior at present, the existing fuzzy kernel prior is a simple basic assumption made on the fuzzy kernel, and it cannot be guaranteed that an accurate fuzzy kernel is estimated quickly. Therefore, the blur kernel priors cannot achieve a good deblurring effect on single-lens computed imaging.
Fuzzy kernel priors (such as sparse priors or Gaussian priors) adopted in the existing blind convolution image restoration algorithm and image priors are mutually independent, and Liu et al [7] researches find that an original fuzzy image contains rich fuzzy kernel information, and the fuzzy kernel priors extracted from the original fuzzy image are more consistent with real fuzzy kernels. The blurred image can be regarded as a result of convolution of the sharp image and the blur kernel, the image is used as a part of convolution operation, and the blur kernel prior is obtained by analyzing and comparing the change situation of the image frequency spectrum before and after image blurring, and the information of the original image can be fully utilized by the blur kernel prior.
At present, the method for extracting the fuzzy kernel prior according to the original fuzzy image information is only applied to the common image deblurring and is not applied to the single-lens computational imaging technology, so the method tries to apply the special fuzzy kernel prior to the single-lens imaging algorithm, and fully considers the influence of lens imaging aberration and dispersion on the image by utilizing the image frequency spectrum information so as to obtain the fuzzy kernel prior which is more accurate than the prior algorithm.
After the fuzzy kernel is estimated by using a blind convolution image restoration algorithm combining image prior and fuzzy kernel prior, a restored image is further estimated by using a non-blind convolution image restoration algorithm. The existing non-blind convolution image restoration algorithm is relatively mature, and the probability model of the non-blind convolution image restoration algorithm can be expressed as:
wherein for filtering f, C is definedfFor corresponding convolution matrices (e.g. for)。CfkConvolution matrices corresponding to the blur kernel, CglAnd CgbRespectively, filtering and corresponding convolution matrix, and eta is a control parameter. From the probabilistic model of equation (6), the image restoration problem of non-blind convolution can be expressed as solving the optimal solution:
l*＝arg max Pk(l|b) (6)
for gaussian distribution, the optimal solution can be approximately converted into the following least square optimization problem:
the optimal solution of equation (7) can be solved by sparse linear representation of Al ═ b
Equation (8) can be solved quickly in the frequency domain.
The invention content is as follows:
aiming at the problem that the existing fuzzy kernel prior cannot meet the requirement of single-lens computational imaging, the invention aims to provide a single-lens imaging method for extracting the fuzzy kernel prior according to image frequency spectrum information, and the method combines a new fuzzy kernel prior and utilizes a non-blind convolution image restoration algorithm to obtain a deblurred clear image so as to obtain a better deblurring effect.
In order to achieve the purpose, the invention adopts the technical scheme that:
a single lens imaging method for extracting fuzzy kernel prior according to image frequency spectrum information is characterized by comprising the following steps:
the method comprises the following steps: obtaining an original blurred image b by a single lens camera;
step two: setting the target optimization function of the blind convolution image restoration algorithm based on the maximum posterior probability as follows:
wherein, l is a clear image, k is a fuzzy kernel corresponding to the clear image, the first term | k | l-b | Y2Representing constraints on image noise, the second term ρl(l) And a third term ρk(k) Respectively representing an image prior and a fuzzy kernel prior;
step three: substituting the target optimization function into the blind convolution image restoration algorithm based on the maximum posterior probability1/l2Image priors and blur kernel priors rho extracted from frequency domain information of blurred imagesk(k) Estimating the single lens camera by using the blind convolution image restoration method based on the maximum posterior probabilityA corresponding blur kernel;
step four: and (4) combining the blurred image in the step one and the blurred kernel estimated in the step three, and obtaining the deblurred clear image by using a non-blind convolution image restoration algorithm.
Further, the fuzzy kernel prior in the third step is rhok(k) The extraction method comprises the following steps: firstly, a fuzzy kernel prior is set, and the fuzzy kernel prior converges to a real fuzzy kernel k0That is, given a blurred image b represented by a certain image feature L, a convex function as shown below is obtained as a blur kernel prior:
wherein h (k) represents fuzzy kernel prior, b is a fuzzy image, L represents an image characteristic used for extracting the fuzzy kernel prior, and m1×m2Representing the size of the blurred image, R representing the real number domain;
for images defined by image features LFirst eigenvalue of its convolutionIs defined as:
wherein,representing imagesThe first feature value of the convolution, L represents the image feature employed,representing a convolution operator, R representing a real number field, s1×s2Representing the sample size employed, X being satisfied with | | X | | luminanceF1-ofAnd (4) matrix. | x | non-conducting phosphorFRepresenting the F norm of a matrix, expressed asThe optimal solution of the above formula is expressed as
The final blur kernel prior can be expressed as:
further, the single lens camera in the first step is different from the mainstream camera currently used, and the lens part of the single lens camera only comprises a simple convex lens instead of a combination of a plurality of different types of lenses. The image blur obtained by a single lens camera is mainly due to aberrations and chromatic dispersion of the single lens imaging.
Further, l added to the objective optimization equation in the third step1/l2Image priors, where1Norm sum l2The norm is not the norm of the image gradient, but the norm for the image itself. If b represents the original blurred image, the specific expression of the image prior in the target optimization equation is | | b | | survival1/||b||2. Wherein | b | purple1L representing image b1Norm, | b | | luminance2L representing image b2And (4) norm. l1The norm represents the number of non-zero elements in the matrix, l of matrix x1Norm expression is | | x | | non-conducting phosphor1＝sum(abs(xi) Sum denotes the sum, abs denotes the absolute value, xiRepresenting the ith element of the matrix x. l2The norm represents the linear distance of two vector matrices in space, l of the matrix2Norm expression is | | x | | non-conducting phosphor2＝sqrt(sum(xi2)), where sqrt represents taking the square root.
Further, when the image prior and the fuzzy kernel prior are added to the objective optimization function in the third step, the following formula is obtained:
when solving, firstly setting an initial fuzzy kernel (such as a delta function, wherein the delta function is a fuzzy kernel instead of a fuzzy kernel prior), obtaining a potential clear image, then estimating a new fuzzy kernel according to the potential clear image and an original fuzzy image, and iterating the loop blind convolution image restoration algorithm in such a way until obtaining a final fuzzy kernel.
Further, in the fourth step, the blind convolution image restoration method based on the maximum posterior probability performs multiple iterations in the scale space in order to further improve the image restoration effect. Blind convolution image restoration methods are described in the background section.
Further, the non-blind convolution image restoration algorithm adopted in the fifth step is the existing non-blind convolution image restoration algorithm, and the specific steps are that the probability model of the non-blind convolution image restoration algorithm is expressed as: <math>
<mrow>
<msub>
<mi>R</mi>
<mi>k</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>l</mi>
<mo>|</mo>
<mi>b</mi>
<mo>)</mo>
</mrow>
<mo>∝</mo>
<mi>exp</mi>
<mrow>
<mo>(</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mfrac>
<mn>1</mn>
<msup>
<mi>η</mi>
<mn>2</mn>
</msup>
</mfrac>
<msup>
<mrow>
<mo>|</mo>
<msub>
<mi>C</mi>
<mi>fk</mi>
</msub>
<mi>l</mi>
<mo>-</mo>
<mi>b</mi>
<mo>|</mo>
</mrow>
<mn>2</mn>
</msup>
<mo>+</mo>
<mi>α</mi>
<msup>
<mrow>
<mo>|</mo>
<msub>
<mi>C</mi>
<mi>gl</mi>
</msub>
<mi>l</mi>
<mo>|</mo>
</mrow>
<mn>2</mn>
</msup>
<mo>+</mo>
<mi>α</mi>
<msup>
<mrow>
<mo>|</mo>
<msub>
<mi>C</mi>
<mi>gb</mi>
</msub>
<mi>b</mi>
<mo>|</mo>
</mrow>
<mn>2</mn>
</msup>
<mo>)</mo>
</mrow>
<mo>)</mo>
</mrow>
</mrow>
</math>
wherein for filtering f, C is definedfFor corresponding convolution matrices, i.e.CfkConvolution moments corresponding to blur kernelsArray, CgxAnd CgyRespectively, filtering and corresponding convolution matrix, and eta is a control parameter. From the above non-blind convolution probability model, the image restoration problem of non-blind convolution can be expressed as solving an optimal solution:
l*＝arg max Pk(l|b)
for gaussian distribution, the above optimal solution is approximately converted into the following least squares optimization problem:
the optimal solution of the above equation is solved by sparse linear representation of Al ═ b:
and finally, rapidly solving a final clear image in a frequency domain.
The method can finally realize the deblurring processing of the single lens calculation imaging, and when the blind convolution image restoration algorithm is used for solving the fuzzy core corresponding to the single lens, the blind convolution image restoration algorithm is used for1/l2The combination of the image prior and the fuzzy kernel prior extracted according to the frequency spectrum information of the image can quickly and accurately estimate the fuzzy kernel, thereby finally obtaining a satisfactory image deblurring effect. The addition of the fuzzy kernel prior enables the imaging effect of the single-lens camera to be better, and has very important significance in the fields of camera lens design and computational photography.
Reference documents:
[1]N.V.Mahajan,N.V.Mahajan,Aberration theory made simple,Bellingham,Washington,USA:SPIEoptical engineering press,1991.
[2]R.Raskar,J.Tumblin,A.Mohan,Computational photography,Proceedings of Eurgraphics:State of the ArtReport(STAR),1(1)(2006)1-20.
[3]T.F.Chan,C.K.Wong,Total variation blind deconvolution,IEEE Transaction on Image Processing7(3)(1998)370-375.
[4]D.Krishnan,R.Fergus,Fast image deconvolution us ing hyper-Laplacian priors,in:NIPS,22(2009)1-9.
[5]D.Krishnan,T.Tay,R.Fergus,Blind deconvolution using a normalized sparsity measure,in:IEEEInternational Conference on C omp uter Vision and Pattern Recogni tion,2008,pp.233-240.
[6]L.Yuan,L.Quan,Image deblurring,PhD thesis,Hong Kong University of Science and Technology,2009.
[7]G.C.Liu,M.Yi,Blind image deblurring by spectral properties of convolution operators,arXiv preprint arXiv:1209.2082(2012).
description of the drawings:
FIG. 1 is a schematic view of a complex combined lens with a single lens replaced by a conventional one;
FIG. 2 is a diagram illustrating a conventional image restoration algorithm;
FIG. 3 is a flow chart of a method of the present invention;
FIG. 4 is an original blurred image taken by a single lens camera;
FIG. 5 is a flow chart of a multi-scale interactive iterative algorithm;
FIG. 6(a) is a deblurring effect obtained by using a prior blur kernel prior;
FIG. 6(b) shows the deblurring effect obtained by the method of the present invention.
The specific implementation mode is as follows:
the design principles of the present invention are further illustrated with reference to the accompanying drawings and specific examples.
As shown in fig. 3, the single-lens imaging method for extracting a blur kernel prior according to image spectrum information includes the following steps:
the method comprises the following steps: the original blurred image is obtained by a single lens camera, which, unlike the mainstream cameras currently used, has a lens portion comprising only a simple convex lens, rather than a combination of many different types of lenses. The single lens camera is obtained by mounting the manufactured single lens on a Canon EOS5D camera. The image blur obtained by a single lens camera is mainly caused by aberrations and dispersion of single lens imaging, and an original blurred image taken by a single lens is shown in fig. 4.
Step two: the original blurred image b is obtained from step one, and now a sharp image l and a corresponding blur kernel k need to be obtained. A blind convolution image restoration algorithm based on the maximum posterior probability is generally adopted to estimate a fuzzy core corresponding to the single lens camera, and corresponding image priors and fuzzy core priors need to be added in a target optimization equation in the process of estimating the fuzzy core.
Under the framework of maximum posterior probability, the statistical model for obtaining the clear image l and the fuzzy kernel k from the fuzzy image b can be expressed as
Assuming that l, b, and k are independent of each other, since the blurred image is directly obtained by a single-lens camera, it can be considered that all information on the blurred image is known, and the following expression can be obtained from the property of the conditional probability formula
p(k,l|b)∝p(b|l,k)p(l)p(k) (2)
In equation (2), the term p (k, l | b) on the left is the expected a posteriori estimate in equation (1). If the negative logarithm is taken at the same time for the left and right sides of the formula (2), then the result is obtained
-logp(k,l|b)＝-logp(b|k,l)-logp(l)-logp(k) (3)
Formula (1) takes the maximum value, and formula (3) is equivalent to formula (1), so formula (3) can be more clearly expressed as
Equation (4) is the target optimization function of the blind convolution image restoration algorithm based on the maximum a posteriori probability. Wherein the first term | | k | | l-b | | ceiling2Representing constraints on image noise, the second term ρl(l) And a third term ρk(k) Respectively, representing a priori knowledge of the natural image and the blur kernel. Selection of image priors and fuzzy kernel priorsThe selection directly influences the accuracy of the objective function solution.
Step three: adding l aiming at the target optimization function of the blind convolution image restoration algorithm in the step two1/l2Image prior,/1/l2The advantages of image priors, as discussed above, are1Norm sum l2Neither is the norm of the image gradient, but the norm of the blurred image itself, in equation (4), ρl(l)＝||b||1/||b||2Where b represents a blurred image directly obtained by a single lens camera. Wherein | b | purple1L representing image b1Norm, | b | | luminance2L representing image b2And (4) norm. l1The norm represents the number of non-zero elements in the matrix, l of matrix x1Norm expression is | | x | | non-conducting phosphor1＝sum(abs(xi) Sum denotes the sum, abs denotes the absolute value, xiRepresenting the ith element of the matrix x. l2The norm represents the linear distance of two vector matrices in space, l of the matrix2Norm expression is | | x | | non-conducting phosphor2＝sqrt(sum(xi2)), where sqrt represents taking the square root.
Step four: and adding a fuzzy kernel prior extracted according to the image frequency spectrum information aiming at the target optimization function of the blind convolution image restoration algorithm in the step two. The image has the following characteristics as part of the convolution operation: for a given image (which can be regarded as a matrix), performing convolution operation on the image and any other image (convolution can be regarded as linear operation), the spectrum information (i.e. the set of matrix eigenvalues) of the convolution operation of the blurred image is much smaller than that of the corresponding sharp image. Therefore, a fuzzy kernel prior can be designed, which converges to the true fuzzy kernel k0. In other words, given a blurred image b represented by a certain image feature L, a function as a blur kernel prior can be obtained as shown below:
in formula (5), h (k) represents fuzzy kernel prior, b is a fuzzy image, L represents an image feature used for extracting the fuzzy kernel prior, and m1×m2Indicating the size of the blurred image and R the real number field.
For images defined by image features LFirst eigenvalue of its convolutionIs defined as:
wherein,representing imagesThe first feature value of the convolution, L represents the image feature employed,representing a convolution operator, R representing a real number field, s1×s2Representing the sample size employed, X being satisfied with | | X | | luminanceF1 matrix. | x | non-conducting phosphorFRepresenting the F norm of a matrix, expressed asThe optimal solution of the above formula is expressed as
Under the ideal assumption that there is no noise interference, the blurred image b can be regarded as being obtained by convolving the sharp image with a blur kernelIs/are as followsL is a predefined characteristic filter. From the definition in equation (6) we can derive:
also because the convolution operation is linear, equation (7) can be written as:
it is noted that <math>
<mrow>
<msub>
<mrow>
<mo>|</mo>
<mo>|</mo>
<mfrac>
<mrow>
<msub>
<mi>k</mi>
<mn>0</mn>
</msub>
<mo>⊗</mo>
<msubsup>
<mi>k</mi>
<mi>i</mi>
<mi>L</mi>
</msubsup>
<mrow>
<mo>(</mo>
<mi>b</mi>
<mo>)</mo>
</mrow>
</mrow>
<msub>
<mrow>
<mo>|</mo>
<mo>|</mo>
<msub>
<mi>k</mi>
<mn>0</mn>
</msub>
<mo>⊗</mo>
<msubsup>
<mi>k</mi>
<mi>i</mi>
<mi>L</mi>
</msubsup>
<mrow>
<mo>(</mo>
<mi>b</mi>
<mo>)</mo>
</mrow>
<mo>|</mo>
<mo>|</mo>
</mrow>
<mi>F</mi>
</msub>
</mfrac>
<mo>|</mo>
<mo>|</mo>
</mrow>
<mi>F</mi>
</msub>
<mo>=</mo>
<mn>1</mn>
<mo>,</mo>
</mrow>
</math> And also
So for the ideal blur kernel k0The constraint of (d) can be expressed as:
the final blur kernel prior can be expressed as:
step five: according to the image prior and the fuzzy kernel prior obtained in the third step and the fourth step, the objective optimization function in the formula (4) in the second step can be finally expressed as
Equation (12) is a specific expression of the blind convolution image restoration algorithm based on the maximum a posteriori probability in this item. The blind restoration algorithm obtains two unknown items, namely a clear image and a fuzzy kernel, from a fuzzy image, so that a general solving method firstly fixes a variable, namely an initial fuzzy kernel (such as a delta function, wherein the delta function is used as the fuzzy kernel instead of the fuzzy kernel prior) is given, a potential clear image is obtained, then a more accurate fuzzy kernel is estimated according to the potential clear image and the original fuzzy image, and iteration is circulated until a satisfactory fuzzy kernel is finally obtained. Moreover, in order to further improve the accuracy of the final solution, blind convolution image restoration algorithms are generally implemented in a multi-scale space. Namely, two pyramids containing blurred images and blurred kernels with different scales are established through down-sampling, the edge and structure information of the images are reserved in the high-level scale of the pyramid, and the detail information of the images is reserved in the low-level scale of the pyramid. The whole algorithm flow is shown in fig. 5. Firstly, the obtained blurred image b and the blurred kernel k are subjected to down sampling to obtain two image pyramids. In the top-level scale of the pyramid, the clear image is obtained by using an image restoration method containing the omnidirectional image regular term in the embodiment. For l +1 layer scale, up-sampling the restored image obtained under l layer scale, performing image restoration as a guide image of the image restoration under the scale, and simultaneously recovering more image details by adopting an iterative residual convolution algorithm.
Step six: and aiming at the estimated fuzzy kernel and the original fuzzy image, obtaining the deblurred clear image by using a non-blind convolution image restoration algorithm. The probability model for non-blind convolution can be expressed as:
wherein for filtering f, C is definedfFor corresponding convolution matrices (e.g. for)。CfkConvolution matrices corresponding to the blur kernel, CglAnd CgbRespectively, filtering and corresponding convolution matrix, and eta is a control parameter. From the probabilistic model of equation (6), the image restoration problem of non-blind convolution can be expressed as solving the optimal solution:
l*＝arg max Pk(l|b) (12)
for gaussian distribution, the optimal solution can be approximately converted into the following least square optimization problem:
the optimal solution of equation (13) can be solved by sparse linear representation of Al ═ b
Equation (14) can be solved quickly in the frequency domain.
In the present embodiment, the deblurring processing of the single-lens imaging can be finally realized by the above method, and the deblurring processing is performed on the blurred image captured by the single-lens camera shown in fig. 4 by using the present invention, and the finally obtained deblurred image is shown in fig. 6(b), where fig. 6(a) is a deblurring effect obtained by using the existing blur kernel prior. As can be seen from a comparison of FIG. 6, the present invention has a good deblurring effect. The method can ensure that the single lens calculation imaging obtains a good deblurring effect, and has very important significance in the fields of camera lens design and calculation photography.
Claims (4)
1. A single lens imaging method for extracting fuzzy kernel prior according to image frequency spectrum information is characterized by comprising the following steps:
the method comprises the following steps: obtaining an original blurred image b by a single lens camera;
step two: setting a target optimization function of a blind convolution image restoration algorithm based on maximum posterior probability as follows:
wherein, l is a clear image, k is a fuzzy kernel corresponding to the clear image, the first term | k | l-b | Y2Representing constraints on image noise, the second term ρl(l) And a third term ρk(k) Respectively representing an image prior and a fuzzy kernel prior;
step three: substituting the target optimization function into the blind convolution image restoration algorithm based on the maximum posterior probability1/l2Image priors and blur kernel priors rho extracted from frequency domain information of blurred imagesk(k) Estimating a fuzzy core corresponding to the single lens camera by using a blind convolution image restoration method based on the maximum posterior probability;
step four: combining the blurred image in the step one and the blurred kernel estimated in the step three, and obtaining a deblurred clear image by using a non-blind convolution image restoration algorithm;
fuzzy kernel prior rho in the third stepk(k) The extraction method comprises the following steps: firstly, a fuzzy kernel prior is set, and the fuzzy kernel prior converges to a real fuzzy kernel k0That is, given a blurred image b represented by a certain image feature L, a convex function as shown below is obtained as a blur kernel prior:
wherein h (k) represents fuzzy kernel prior, b is a fuzzy image, L represents an image characteristic used for extracting the fuzzy kernel prior, and m1×m2Representing the size of the blurred image, R representing the real number domain;
for images defined by image features LFirst eigenvalue of its convolutionIs defined as:
wherein,representing imagesThe first feature value of the convolution, L represents the image feature employed,representing a convolution operator, R representing a real number field, s1×s2Representing the sample size employed, X being satisfied with | | X | | luminanceF| | × | non-conducting phosphor non-phosphor screen | |, matrix of 1 | | x | phosphor non-phosphor screenFRepresenting the F norm of the matrix, with the expression | | | x | | non-conducting phosphorF＝(∑∑aij 2)1/2The optimal solution of the above formula is expressed as
The final blur kernel prior can be expressed as:
in the fourth step, the method for obtaining the deblurred clear image by using the non-blind convolution image restoration algorithm comprises the following steps:
the probability model of the non-blind convolution image restoration algorithm is expressed as:
wherein for filtering f, C is definedfFor corresponding convolution matrices, i.e.CfkConvolution matrices corresponding to the blur kernel, CglAnd CgbFiltering and corresponding convolution matrixes are respectively used, eta is a control parameter, and the non-blind convolution probability model and the non-blind convolution image restoration problem are expressed as solving an optimal solution:
l*＝argmaxPk(l|b)
for gaussian distribution, the above optimal solution is approximately converted into the following least squares optimization problem:
the optimal solution of the above equation is solved by sparse linear representation of Al ═ b:
and finally, rapidly solving a final clear image in a frequency domain.
2. The single-lens imaging method for extracting fuzzy kernel prior according to the image spectrum information as claimed in claim 1, wherein l in the third step1/l2In image priors,/1Norm sum l2The norm is the norm of the blurred image itself, i.e. ρl(l)＝||b||1/||b||2Where b represents a blurred image directly obtained by a single-lens camera, | b | | luminance1L representing image b1Norm, | b | | luminance2L representing image b2Norm,/, of1The norm represents the number of non-zero elements in the matrix, l of matrix x1Norm expression is | | x | | non-conducting phosphor1＝sum(abs(xi) Sum denotes the sum, abs denotes the absolute value, xiRepresents the second of the matrix xi elements, l2The norm represents the linear distance of two vector matrices in space, l of the matrix2Norm expression is | | x | | non-conducting phosphor2＝sqrt(sum(xi2)), where sqrt represents taking the square root.
3. The single-lens imaging method for extracting a blur kernel prior according to the image spectrum information of claim 2, wherein when the image prior and the blur kernel prior are added to the objective optimization function in the third step, the following formula is obtained:
during solving, an initial fuzzy kernel is set firstly, a potential clear image is obtained, then a new fuzzy kernel is estimated according to the potential clear image and an original fuzzy image, and the iterative loop blind convolution image restoration algorithm is carried out in this way until a final fuzzy kernel is obtained.
4. The single-lens imaging method for extracting the blur kernel priors according to the image spectrum information of claim 3, wherein the blind convolution image restoration algorithm is performed in a multi-scale space.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410331056.5A CN104091312B (en) | 2014-07-11 | 2014-07-11 | A kind of simple lens formation method according to image spectrum information extraction fuzzy core priori |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410331056.5A CN104091312B (en) | 2014-07-11 | 2014-07-11 | A kind of simple lens formation method according to image spectrum information extraction fuzzy core priori |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104091312A CN104091312A (en) | 2014-10-08 |
| CN104091312B true CN104091312B (en) | 2015-08-19 |
Family
ID=51639027
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410331056.5A Active CN104091312B (en) | 2014-07-11 | 2014-07-11 | A kind of simple lens formation method according to image spectrum information extraction fuzzy core priori |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104091312B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104599254B (en) * | 2015-02-03 | 2016-03-02 | 中国人民解放军国防科学技术大学 | Simple lens based on combinational fuzzy nuclear structure priori calculates formation method |
| CN104809706B (en) * | 2015-05-05 | 2016-08-17 | 中国人民解放军国防科学技术大学 | A kind of simple lens based on color of image smooth variation priori calculates formation method |
| CN104915937B (en) * | 2015-07-02 | 2017-10-20 | 中国人民解放军国防科学技术大学 | Quick simple lens based on frequency domain matrix decomposition calculates imaging method |
| CN105046659B (en) * | 2015-07-02 | 2017-08-11 | 中国人民解放军国防科学技术大学 | A kind of simple lens based on rarefaction representation is calculated as PSF evaluation methods |
| CN106651811B (en) * | 2017-01-03 | 2018-03-16 | 长沙全度影像科技有限公司 | A kind of simple lens imaging ambiguity removal method of luminance channel guiding |
| CN106651812B (en) * | 2017-01-03 | 2018-06-26 | 长沙全度影像科技有限公司 | A kind of multichannel PSF scaling methods of simple lens imaging |
| CN107680062A (en) * | 2017-10-12 | 2018-02-09 | 长沙全度影像科技有限公司 | A kind of micro- burnt Restoration method of blurred image based on l1/l2 priori combination Gaussian priors |
| CN107749051A (en) * | 2017-10-17 | 2018-03-02 | 长沙全度影像科技有限公司 | A kind of unzoned lens space-variant blur core smoothing method based on mean filter |
| CN108364274B (en) * | 2018-02-10 | 2020-02-07 | 东北大学 | Nondestructive clear reconstruction method of optical image under micro-nano scale |
| CN108694715A (en) * | 2018-05-15 | 2018-10-23 | 清华大学 | One camera RGB-NIR imaging systems based on convolution sparse coding |
| CN108732432B (en) * | 2018-05-30 | 2020-08-21 | 重庆大学 | Constrained noise enhancement signal detection method under maximum posterior probability criterion |
| CN109272456A (en) * | 2018-07-25 | 2019-01-25 | 大连理工大学 | The blurred picture high-precision restoring method of view-based access control model prior information |
| CN113177889B (en) * | 2021-04-27 | 2022-03-01 | 深圳市慧鲤科技有限公司 | Image processing method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063716A (en) * | 2011-01-13 | 2011-05-18 | 耿则勋 | Multiframe iteration blind deconvolution image restoration method based on anisotropic constraint |
| JP2011233058A (en) * | 2010-04-30 | 2011-11-17 | Panasonic Corp | Imaging device, image processor, and image processing method |
| CN102354395A (en) * | 2011-09-22 | 2012-02-15 | 西北工业大学 | Sparse representation-based blind restoration method of broad image |
| CN102393955A (en) * | 2011-07-18 | 2012-03-28 | 西安电子科技大学 | Perfect information non-local constraint total variation method for image recovery |
| EP2535864A1 (en) * | 2010-02-10 | 2012-12-19 | Panasonic Corporation | Image processing device and method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8781250B2 (en) * | 2008-06-26 | 2014-07-15 | Microsoft Corporation | Image deconvolution using color priors |
-
2014
- 2014-07-11 CN CN201410331056.5A patent/CN104091312B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2535864A1 (en) * | 2010-02-10 | 2012-12-19 | Panasonic Corporation | Image processing device and method |
| JP2011233058A (en) * | 2010-04-30 | 2011-11-17 | Panasonic Corp | Imaging device, image processor, and image processing method |
| CN102063716A (en) * | 2011-01-13 | 2011-05-18 | 耿则勋 | Multiframe iteration blind deconvolution image restoration method based on anisotropic constraint |
| CN102393955A (en) * | 2011-07-18 | 2012-03-28 | 西安电子科技大学 | Perfect information non-local constraint total variation method for image recovery |
| CN102354395A (en) * | 2011-09-22 | 2012-02-15 | 西北工业大学 | Sparse representation-based blind restoration method of broad image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104091312A (en) | 2014-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104091312B (en) | A kind of simple lens formation method according to image spectrum information extraction fuzzy core priori | |
| Liu et al. | Attention-guided global-local adversarial learning for detail-preserving multi-exposure image fusion | |
| Li et al. | Blind image deblurring via deep discriminative priors | |
| Jin et al. | Normalized blind deconvolution | |
| CN104008531B (en) | Blurred image blind restoration method based on mixed type Markov expert field | |
| CN104112261A (en) | Norm ratio regularization based rapid image blind deblurring method | |
| CN110796616B (en) | Turbulence degradation image recovery method based on norm constraint and self-adaptive weighted gradient | |
| CN107507135A (en) | Image reconstructing method based on coding aperture and target | |
| Li et al. | A maximum a posteriori estimation framework for robust high dynamic range video synthesis | |
| Sanghvi et al. | Photon limited non-blind deblurring using algorithm unrolling | |
| El-Henawy et al. | A comparative study on image deblurring techniques | |
| Lin et al. | Non-blind optical degradation correction via frequency self-adaptive and finetune tactics | |
| Chen et al. | Deep Richardson–Lucy deconvolution for low-light image deblurring | |
| Li et al. | A computational photography algorithm for quality enhancement of single lens imaging deblurring | |
| Torem et al. | Complex-valued retrievals from noisy images using diffusion models | |
| Wang et al. | Multi-scale blind motion deblurring using local minimum | |
| Zhang et al. | Research on adaptive optics image restoration algorithm based on improved joint maximum a posteriori method | |
| Maik et al. | Blind deconvolution using maximum a posteriori (MAP) estimation with directional edge based priori | |
| Li et al. | Defocus hyperspectral image deblurring with adaptive reference image and scale map | |
| Kheradmand et al. | Motion deblurring with graph Laplacian regularization | |
| Xu et al. | Geometric distortion correction of long-range imaging containing moving objects | |
| Conti et al. | A regularized deep learning approach for image de-blurring | |
| Jain et al. | Evaluation of neural network algorithms for atmospheric turbulence mitigation | |
| Liu et al. | Prior Guided Multi-Scale Dynamic Deblurring Network for Diffraction Image Restoration in Droplet Measurement | |
| Mao | Image Restoration Methods for Imaging Through Atmospheric Turbulence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |