CN106780406A - A kind of quick fish eye images deblurring method - Google Patents

Abstract

The invention discloses a kind of quick fish eye images deblurring method, belong to digital image processing techniques field, the method includes：S1, the projection according to fish eye images and Cylindrical panoramic image and anti-projection relation, project Cylindrical panoramic image by fish eye images are counter, obtain to deserved Cylindrical panoramic image；S2, deblurring is carried out to cylinder panoramic image P using the method based on frequency domain matrix decomposition, obtain the Cylindrical panoramic image P' of deblurring；S3, the Cylindrical panoramic image P ' orthographic projections of deblurring to fish eye images are obtained into clear fish eye images I '.This method proposes that a kind of fish eye lens based on frequency domain matrix decomposition calculates imaging method, fundamentally eliminate conventional images deblurring algorithm successive ignition optimization process, it is changing into a kind of method of linear plus sum, the time required to so as to greatly reduce fish eye images deblurring, it is used to meet the demand of panorama flake video real-time.

Description

A kind of quick fish eye images deblurring method

Technical field

The present invention relates to digital image processing techniques field, more particularly to a kind of quick fish eye images deblurring method.

Background technology

Fish eye lens is different with other camera lenses in shape, and preceding group of eyeglass is outwardly as flake, is a kind of special Wide-angle lens, its angle of visual field close to, equal to even greater than 180 degree, can be by the object in hemisphere spatial domain even hemispherical spatial domain It is imaged on and practises physiognomy in limited scope.

From from the point of view of optical design, the lens that fish eye lens is used have very big sphere radian, and distance into Image plane closer to.This special design feature and imaging characteristicses, on the one hand allow that fisheye camera obtains very big visual field model Enclose, the reality scene of larger field is needed in robot navigation, video conference, monitor in real time, panoramic shooting and astronomical observation etc. In be able to extensive use；On the other hand, due to introducing very big barrel distortion so that the image that fish eye lens is formed is except drawing The scenery at face center keeps constant, and the scenery that other should be horizontally or vertically all there occurs corresponding change, so as to cause flake The resolution ratio of lens imaging face different zones is different, and closer to picture centre, resolution ratio is higher, and detailed information is more, more Deviate picture centre, resolution ratio is lower, and detailed information is fewer, deform more serious.

In order to improve fish eye images definition, make up that the big visual angle of flake brings is image blurring, and conventional way is to use Method of Fuzzy Enhancement.The enhanced fuzzy of image is, using certain uncertainty, i.e. ambiguity present in image, fuzzy set to be managed By a kind of method for image enhaucament.

In conventional images Enhancement Method, directly processed on the fish eye images of distortion, but due to fish eye images data Store in a non-linear manner, it is impossible to directly process, this processing mode cannot obtain preferable image deblurring effect.

In addition, the process that picture rich in detail is tried to achieve in existing fuzzy image enhancement method needs successive ignition, the time of consuming It is more long.It is difficult to meet the demand of panorama flake video real-time.

The content of the invention

The present invention need in existing fish eye images deblurring successive ignition to overcome, and expends time technical problem more long, Aim to provide a kind of quick fish eye images deblurring method for meeting panorama flake video real-time demand.

A kind of quick fish eye images deblurring method, including：

S1, the projection according to fish eye images and Cylindrical panoramic image and anti-projection relation, project cylinder by fish eye images are counter Panorama sketch, obtains to deserved Cylindrical panoramic image；

S2, deblurring is carried out to cylinder panoramic image P using the method based on frequency domain matrix decomposition, obtain the post of deblurring Face panorama sketch P'；Specifically include：

S21, blurred picture y under time domain and corresponding fuzzy core k are transformed into frequency domain, obtain the frequency of corresponding blurred picture The spectral matrix K of spectrum matrix Y and fuzzy core；

S22, by spectral matrix Y with a series of basis representation Y=α₁H₁+α₂H₂+…+α_nH_n；The coefficient a of linear combination_iIt is With in frequency domain matrix Y with base H_iCorresponding that a part of numerical value；

S23, each base H for Y_i, with reference to fuzzy core K, deblurring is carried out using the non-blind convolution algorithm under frequency domain, Obtain corresponding picture rich in detail base X_i；

S24, time domain blurred picture P is transformed into frequency domain blurred picture P₁；

S25, for new frequency domain blurred picture P₁, because picture rich in detail respectively obtains blurred picture Y and blurred picture P₁Institute The fuzzy core K for using is identical, for the spectral matrix P of new blurred picture₁, equally it is split into the linear combination of base P₁=α₁’H₁+α₂’H₂+…+α_n’H_n, then corresponding picture rich in detail can directly be expressed as X=α₁’X₁+α₂’X₂+…+α_n’X_n, by institute The picture rich in detail X reconverts for obtaining obtain the Cylindrical panoramic image P' of deblurring to time domain；

S3, the Cylindrical panoramic image P ' orthographic projections of deblurring to fish eye images are obtained into clear fish eye images I '.

Further, the S1 is specially：

Point I (i, j) back projection in fish eye images finds cylindrical panoramic to point P (u, v) in cylinder panoramic image (u+1, v) with P (u, v+1), fish eye images is found further according to orthographic projection to consecutive points pixel P in image both horizontally and vertically In corresponding pixel I (i+s_1, j+t_1) and I (i+s_2, j+t_2), then by pixel I (i+s_1, j+ in fish eye images T_1) and I (i+s_2, j+t_2) pixel value be imparted to corresponding pixel P in cylindrical picture (u+1, v) and P (u, v+1), according to Secondary iteration, obtains Cylindrical panoramic image P.

Further, in the step S22,

The selection mode of the base of spectral matrix：, as origin, number according to base is by frequency spectrum square for central point with spectral matrix Battle array is divided into the multiple width identical straight-flanked rings equal with the number of base, size and the frequency spectrum square of each base successively from the inside to the outside Battle array is in the same size, and the matrix ring of innermost matrix ring to outermost is set to the first matrix ring, second the n-th square of matrix ring ... successively Matrix ring, by innermost matrix ring, each pixel is set to 1 first, and the pixel of other two straight-flanked ring is set to 0, obtains base H₁；So By the second matrix ring, each pixel is set to 1 afterwards, and the pixel of other two straight-flanked ring is set to 0, obtains base H₂；According to the method according to It is secondary to analogize, finally give all of base.

Further, the method employed in step S33 is the non-blind convolved image restoration algorithm based on L2 norms, algorithm Main flow is as follows：

Blurred picture y is expressed as the convolution y=x*k of picture rich in detail x and fuzzy core k, if based on Maximize Thought, image restoration problem representation is：

X=arg max_x P(x|y)∝P(y|x)P(x) (1)

In formula (1), x represents the picture rich in detail finally tried to achieve；Y represents known blurred picture；P (x | y) represent Know blurred picture, obtain the probability that picture rich in detail is x；P (y | x) represent if it is known that picture rich in detail, obtains correspondence blurred picture It is the probability of y；P (x) is represented to prior probability known to original picture rich in detail；

Assuming that noise Gaussian distributed, and variance is η, then it represents that be：

In formula (2), and P (y | x) represent if it is known that picture rich in detail, it is the probability of y to obtain correspondence blurred picture, by clear The process that image obtains blurred picture is interpreted as with the addition of noise, so this probability is approximately the Gaussian Profile that variance is ηC_fIt is the convolution matrix of N ' N；

Assuming that image prior can use a series of filtering g_kRepresent, and the reaction that image is filtered to priori is as much as possible Small, then image prior is expressed as：

Wherein, P (x) represents the known prior information of picture rich in detail；Horizontal direction is filtered into g_x=[1-1]；Vertical Square To be filtered into g_y=[1-1]^T；ρ represents Prior function；g_i,kRepresent k-th filtering for ith pixel；

Formula (1), (2), the logarithmic form of (3) are gone, then can obtain the object function of image restoration：

||y-C_fx||²+ω∑_{I, k}ρ(g_i,k*x) (4)

W=α η in formula (4)²；Gaussian image priori is taken, and sets ρ (z)=| z |²；Formula (4) derivation, and order are led Number is zero, then can obtain Ax=b, whereinAx=b is transformed under frequency domain Solution is then obtained：

Formula (5) is the non-blind convolved image restoration algorithm final result based on L2 norms, wherein v and ω tables under frequency domain Show the coordinate under frequency domain.

Projection and anti-projection relation of this method according to fish eye images and Cylindrical panoramic image, project post by fish eye images are counter Face panorama sketch, obtains to deserved Cylindrical panoramic image；Then for Cylindrical panoramic image, by the matrix phase of frequency domain hypograph restoration algorithm Multiply and split, propose that a kind of fish eye lens based on frequency domain matrix decomposition calculates imaging method, fundamentally eliminate existing Image deblurring algorithm successive ignition optimization process, directly the base X by being tried to achieve_iWith corresponding linear combination coefficient α_i' pass through phase Multiply addition and obtain final clear cylinder panoramic image P ', then clear cylinder panoramic image P ' orthographic projections are obtained to fish eye images To clear fish eye images I ', the calculating time needed for the method greatly reduces, and can meet the real-time demand of panorama picture of fisheye lens, This method all has very important significance in image procossing and camera design field.

Additional aspect of the invention and advantage will be set forth in part in the description, and will partly become from the following description Obtain substantially, or recognized by practice of the invention.

Brief description of the drawings

In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this Some embodiments of invention, for those of ordinary skill in the art, without having to pay creative labor, may be used also Other accompanying drawings are obtained with according to these accompanying drawings.

Fig. 1 is fish eye images cylinder back projection schematic diagram provided in an embodiment of the present invention；

Fig. 2 is a kind of quick fish eye images deblurring method flow chart provided in an embodiment of the present invention；

Fig. 3 is the selection schematic diagram of frequency domain matrix correspondence base provided in an embodiment of the present invention；

Fig. 4 is blurred picture spectral matrix basis representation schematic diagram provided in an embodiment of the present invention.

Specific embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made Embodiment, belongs to the scope of protection of the invention.

Because there is larger deformation in the fish eye images that fish eye lens has special structure, shooting, the vision with human eye There is larger difference in effect, and view data is stored in a non-linear manner, it is impossible to directly process, therefore firstly the need of by fish Eye image flame detection is linear projection image.

The present invention expends time technical problem more long to solve to need successive ignition in existing fish eye images deblurring, There is provided a kind of quick fish eye images deblurring method, as shown in Fig. 2 the method includes：

S1, the projection according to fish eye images and Cylindrical panoramic image and anti-projection relation, project cylinder by fish eye images are counter Panorama sketch, obtains to deserved Cylindrical panoramic image；

Specifically, cylinder panoramic image is projected by the pixel of fish eye images is counter, according to ladder in Cylindrical panoramic image Degree computational methods determine neighbor pixel both horizontally and vertically；Orthographic projection again finds corresponding pixel points in fish eye images Relevant position, the pixel value of relevant position in fish eye images is imparted to the pixel in cylindrical picture, obtain corresponding post Face panorama sketch；

The step reference picture 1 is further analyzed, and point I (i, the j) back projection in fish eye images is complete to cylinder Point P (u, v) in scape image, find in cylinder panoramic image both horizontally and vertically consecutive points pixel P (u+1, v) and P (u, v+1), corresponding pixel I (i+s_1, j+t_1) and I (i+s_2, j+t_2) in fish eye images are found further according to orthographic projection, Then the pixel value of pixel I (i+s_1, j+t_1) and I (i+s_2, j+t_2) in fish eye images is imparted in cylindrical picture (u+1, v) with P (u, v+1), iteration, obtains Cylindrical panoramic image P to corresponding pixel P successively.

S2, deblurring is carried out to cylinder panoramic image P using the method based on frequency domain matrix decomposition, after obtaining deblurring Cylindrical panoramic image P '；

Specifically include：

S21：Blurred picture y under time domain and corresponding fuzzy core k are transformed into frequency domain, the frequency of corresponding blurred picture is obtained The spectral matrix K of spectrum matrix Y and fuzzy core；

Blurred picture y under time domain and corresponding fuzzy core k are transformed into frequency domain by the ifft2 functions in Matlab.

S22：By spectral matrix Y with a series of basis representation Y=α₁H₁+α₂H₂+…+α_nH_n；Wherein：The choosing of the base of spectral matrix Mode is taken as shown in figure 3, Fig. 3 is with the central point of spectral matrix as origin, the number according to base by spectral matrix from the inside to the outside according to Secondary to be divided into the multiple width identical straight-flanked rings equal with the number of base, the size of each base is in the same size with spectral matrix, Innermost matrix ring to the matrix ring of outermost is set to the first matrix ring successively, and second the n-th matrix ring of matrix ring ... first will Innermost matrix ring each pixel is set to 1, and the pixel of other two straight-flanked ring is set to 0, obtains base H₁；Then by the second matrix Ring each pixel is set to 1, and the pixel of other two straight-flanked ring is set to 0, obtains base H₂；According to the method the like, finally Obtain all of base；The factor alpha of linear combination_iBe with frequency domain matrix Y with base H_iCorresponding that a part of numerical value.At other In case study on implementation, for the base of different numbers, also according to the method, all of base is finally given.Specific real Shi Zhong, the number of the base of spectral matrix elects n=3, selection mode as shown in figure 3, schematic diagram such as Fig. 4 institutes of resulting base Show.

S23：For each base H of Y_i, with reference to fuzzy core K, restored using the non-blind convolved image based on L2 norms and calculated Method carries out deblurring computing, and the restoration result X that will be finally given_iPreserve.Algorithm main flow is as follows：

Blurred picture y is expressed as the convolution y=x*k of picture rich in detail x and fuzzy core k, if based on Maximize Thought, image restoration problem representation is：

X=argmax_x P(x|y)∝P(y|x)P(x)(1)

In formula (1), x represents the picture rich in detail finally tried to achieve；Y represents known blurred picture；P (x | y) represent Know blurred picture, obtain the probability that picture rich in detail is x；P (y | x) represent if it is known that picture rich in detail, obtains correspondence blurred picture It is the probability of y；P (x) is represented to prior probability known to original picture rich in detail；

Assuming that noise Gaussian distributed, and variance is η, then it represents that be：

In formula (2), and P (y | x) represent if it is known that picture rich in detail, it is the probability of y to obtain correspondence blurred picture, by clear The process that image obtains blurred picture is interpreted as with the addition of noise, so this probability is approximately the Gaussian Profile that variance is ηC_fIt is the convolution matrix of N ' N；

Assuming that image prior can use a series of filtering g_kRepresent, and the reaction that image is filtered to priori is as much as possible Small, then image prior is expressed as：

Wherein, P (x) represents the known prior information of picture rich in detail；Horizontal direction is filtered into g_x=[1-1]；Vertical Square To be filtered into g_y=[1-1]^T；ρ represents Prior function；g_i,kRepresent k-th filtering for ith pixel；Go formula (1), (2), the logarithmic form of (3), then can obtain the object function of image restoration：

||y-C_fx||²+ω∑_{I, k}ρ(g_i,k*x) (4)

W=α η in formula (4)²；Gaussian image priori is taken, and sets ρ (z)=| z |²；Formula (4) derivation, and order are led Number is zero, then can obtain Ax=b, whereinAx=b is transformed under frequency domain Solution is then obtained：

Formula (5) is the non-blind convolved image restoration algorithm final result based on L2 norms, wherein v and ω tables under frequency domain Show the coordinate under frequency domain.

S24, time domain blurred picture P is transformed into frequency domain blurred picture P₁。

S25, for new frequency domain blurred picture P₁, because picture rich in detail respectively obtains blurred picture Y and blurred picture P₁Institute The fuzzy core K for using is identical.For the spectral matrix P of new blurred picture₁, equally it is split into the linear combination of base P₁=α₁’H₁+α₂’H₂+…+α_n’H_n, then corresponding picture rich in detail can directly be expressed as X=α₁’X₁+α₂’X₂+…+α_n’X_n, by institute The picture rich in detail X reconverts for obtaining are to obtaining P ' by time domain.The selection mode of the base of new blurred picture and mould in step 2 The selection mode for pasting the base of image is the same, and the image under frequency domain is gone into time domain uses ifft2 letters in Matlab Number.

For the gray level image of 255 × 255 sizes, in image restoration effect in the case of, matrix decomposition institute is used The time for needing is 0.0253s, is 0.0588s without the time needed for the direct Image Restoration Algorithm using L2 norms under frequency domain. If it is 1024 × 1024 that image size increases, the calculating time needed for two kinds of algorithms is respectively 0.2334s and 2.2806s. With reference to the real-time demand that panorama flake is imaged, the image recovery method based on matrix decomposition can substantially meet this demand.

S3 and then P ' orthographic projections to fish eye images are obtained into clear fish eye images I '.

The present invention is based on panorama fish-eye lens imaging system, projection and anti-projection according to fish eye images and Cylindrical panoramic image Relation, projects Cylindrical panoramic image by fish eye images are counter, obtains to deserved Cylindrical panoramic image P；Then for Cylindrical panoramic image, will The matrix multiple of frequency domain hypograph restoration algorithm is split, and proposes that a kind of fish eye lens based on frequency domain matrix decomposition is calculated as Image space method, fundamentally eliminates conventional images deblurring algorithm successive ignition optimization process, directly the base X by being tried to achieve_iWith Corresponding linear combination coefficient α_i' by being multiplied, addition obtains final picture rich in detail P ', then by P ' orthographic projections to fish-eye image As obtaining clear fish eye images I ', the calculating time needed for the method greatly reduces, and can meet the real-time of panorama picture of fisheye lens Demand, this method all has very important significance in image procossing and camera design field.

Above disclosed is only a kind of preferred embodiment of the invention, can not limit the power of the present invention with this certainly Sharp scope, therefore the equivalent variations made according to the claims in the present invention, still belong to the scope that the present invention is covered.

Claims (4)

1. a kind of quick fish eye images deblurring method, it is characterised in that including：

S1, the projection according to fish eye images and Cylindrical panoramic image and anti-projection relation, project cylindrical panoramic by fish eye images are counter Figure, obtains to deserved Cylindrical panoramic image；

S2, deblurring is carried out to cylinder panoramic image P using the method based on frequency domain matrix decomposition, the cylinder for obtaining deblurring is complete Scape figure P'；Specifically include：

S21, blurred picture y under time domain and corresponding fuzzy core k are transformed into frequency domain, obtain the frequency spectrum square of corresponding blurred picture The spectral matrix K of battle array Y and fuzzy core；

S22, by spectral matrix Y with a series of basis representation Y=α₁H₁+α₂H₂+…+α_nH_n；The coefficient a of linear combination_iIt is and frequency domain With base H in matrix Y_iCorresponding that a part of numerical value；

S23, each base H for Y_i, with reference to fuzzy core K, deblurring is carried out using the non-blind convolution algorithm under frequency domain, obtain Corresponding picture rich in detail base X_i；

S24, time domain blurred picture P is transformed into frequency domain blurred picture P₁；

S25, for new frequency domain blurred picture P₁, because picture rich in detail respectively obtains blurred picture Y and blurred picture P₁Used Fuzzy core K be identical, for the spectral matrix P of new blurred picture₁, equally it is split into the linear combination P of base₁= α₁’H₁+α₂’H₂+…+α_n’H_n, then corresponding picture rich in detail can directly be expressed as X=α₁’X₁+α₂’X₂+…+α_n’X_n, will be resulting Picture rich in detail X reconverts the Cylindrical panoramic image P' of deblurring is obtained to time domain；

S3, the Cylindrical panoramic image P ' orthographic projections of deblurring to fish eye images are obtained into clear fish eye images I '.

2. quick fish eye images deblurring method according to claim 1, it is characterised in that：The S1 is specially：

Point I (i, j) back projection in fish eye images finds cylinder panoramic image to point P (u, v) in cylinder panoramic image In consecutive points pixel P both horizontally and vertically (u+1, v) and P (u, v+1), it is right in fish eye images to be found further according to orthographic projection The pixel I (i+s_1, j+t_1) and I (i+s_2, j+t_2) for answering, then by pixel I (i+s_1, j+t_1) in fish eye images (u+1's corresponding pixel P, v) with P (u, v+1), changes successively in being imparted to cylindrical picture with the pixel value of I (i+s_2, j+t_2) In generation, obtain Cylindrical panoramic image P.

3. quick fish eye images deblurring method according to claim 2, it is characterised in that：In the step S22,

The selection mode of the base of spectral matrix：Central point with spectral matrix as origin, number according to base by spectral matrix from In to the multiple width identical straight-flanked rings equal with the number of base are divided into successively outward, the size of each base is big with spectral matrix Small consistent, the matrix ring of innermost matrix ring to outermost is set to the first matrix ring, second the n-th matrix of matrix ring ... successively Ring, by innermost matrix ring, each pixel is set to 1 first, and the pixel of other two straight-flanked ring is set to 0, obtains base H₁；Then By the second matrix ring, each pixel is set to 1, and the pixel of other two straight-flanked ring is set to 0, obtains base H₂；According to the method successively Analogize, finally give all of base.

4. quick fish eye images deblurring method according to claim 3, it is characterised in that：Employed in step S33 Method is the non-blind convolved image restoration algorithm based on L2 norms, and algorithm main flow is as follows：

Blurred picture y is expressed as the convolution y=x*k of picture rich in detail x and fuzzy core k, if the think of based on Maximize Think, image restoration problem representation is：

X=arg max_xP(x|y)∝P(y|x)P(x) (1)

In formula (1), x represents the picture rich in detail finally tried to achieve；Y represents known blurred picture；P (x | y) represent known mould Paste image, obtains the probability that picture rich in detail is x；P (y | x) represent if it is known that picture rich in detail, it is y's to obtain correspondence blurred picture Probability；P (x) is represented to prior probability known to original picture rich in detail；

Assuming that noise Gaussian distributed, and variance is η, then it represents that be：

$P\left(y\right|x)\∝e^{-\frac{1}{2{\mathrm{\η}}^2}\left|\right|x-C_{f}y|{|}^2}---\left(2\right)$

In formula (2), and P (y | x) represent if it is known that picture rich in detail, it is the probability of y to obtain correspondence blurred picture, by picture rich in detail The process for obtaining blurred picture is interpreted as with the addition of noise, so this probability is approximately the Gaussian Profile that variance is ηC_fIt is the convolution matrix of N ' N；

Assuming that image prior can use a series of filtering g_kRepresent, and image is small as much as possible to reaction that priori is filtered, then scheme As priori is expressed as：

$P\left(x\right)=e^{-\mathrm{\α}{\Σ}_{i,k}\mathrm{\ρ}(g_{i,k}*x)}---\left(3\right)$

Wherein, P (x) represents the known prior information of picture rich in detail；Horizontal direction is filtered into g_x=[1-1]；The filter of vertical direction Ripple is g_y=[1-1]^T；ρ represents Prior function；g_i,kRepresent k-th filtering for ith pixel；

Formula (1), (2), the logarithmic form of (3) are gone, then can obtain the object function of image restoration：

||y-C_fx||²+ω∑_{I, k}ρ(g_i,k*x) (4)

W=α η in formula (4)²；Gaussian image priori is taken, and sets ρ (z)=| z |²；To formula (4) derivation, and the derivative is made to be Zero, then Ax=b can be obtained, whereinAx=b is transformed under frequency domain and is solved Then obtain：

$X(v,\mathrm{\ω})=\frac{K(v,\mathrm{\ω})*Y(v,\mathrm{\ω})}{|K(v,,\mathrm{\ω}){|}^2+{\mathrm{\ω\Σ}}_k|G_k(v,\mathrm{\ω}){|}^2}---\left(5\right)$

Formula (5) is the non-blind convolved image restoration algorithm final result based on L2 norms under frequency domain, and wherein v and ω represents frequency Coordinate under domain.