CN117132505A - Motion deblurring method and device based on auto-correlation of texture image - Google Patents
Motion deblurring method and device based on auto-correlation of texture image Download PDFInfo
- Publication number
- CN117132505A CN117132505A CN202311362350.8A CN202311362350A CN117132505A CN 117132505 A CN117132505 A CN 117132505A CN 202311362350 A CN202311362350 A CN 202311362350A CN 117132505 A CN117132505 A CN 117132505A
- Authority
- CN
- China
- Prior art keywords
- image
- variable
- model
- iteration process
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 180
- 230000008569 process Effects 0.000 claims description 98
- 238000004590 computer program Methods 0.000 claims description 10
- 238000011480 coordinate descent method Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 5
- 230000000694 effects Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 238000005286 illumination Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000005311 autocorrelation function Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000002601 radiography Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Mathematical Optimization (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Operations Research (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image processing, and provides a motion deblurring method and a motion deblurring device based on auto-correlation of texture images, wherein an image to be processed is firstly obtained, an auto-correlation image of the texture image corresponding to the image to be processed is calculated, and an initial blur kernel is determined based on the auto-correlation image; then determining a model data item based on the image to be processed and the fuzzy kernel variable and the restored image variable corresponding to the image to be processed, and taking L of the fuzzy kernel variable 2 The square of the norm is the first regularization term, L, to recover the gradient of the image variable 0 The norm is the second regular term to recover the L of the image variable 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built; finally, solving the deblurring model by utilizing the initial blurring kernel to obtain an image to be processed after deblurringIs used for restoring the image. The method can obtain the restored image with clear main structure.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a motion deblurring method and apparatus based on auto-correlation of texture images.
Background
Images are an important way for people to obtain information. However, during the imaging process, the obtained image is often blurred due to the influence of adverse factors such as camera shake, movement of objects in the scene, camera defocus and the like, which greatly hinders the normal use and subsequent processing of the image.
Blind image motion blur removal is a non-negligible technique in computer vision and computational photography, with the goal of recovering blurred images due to motion factors such as camera shake, motion of objects, etc., into sharp images. However, the existing methods for blind deblurring of most images have poor restoration image effect, so that the restored clear image is deformed and ringing artifact is generated, and the maximum image restoration cannot be realized. Moreover, the convergence of the fuzzy core is not strong enough, and the calculation cost is high.
Disclosure of Invention
The invention provides a motion deblurring method and a motion deblurring device based on auto-correlation of texture images, which are used for solving the defects in the prior art.
The invention provides a motion deblurring method based on auto-correlation of texture images, which comprises the following steps:
acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image;
determining a model data item based on the image to be processed and a blur kernel variable and a restored image variable corresponding to the image to be processed, and taking L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built;
and solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after the motion blurring is removed.
According to the motion deblurring method based on the auto-correlation of the texture image, the method for calculating the auto-correlation image of the texture image corresponding to the image to be processed comprises the following steps:
and converting the texture image into a frequency domain based on a fast Fourier transform method, and performing autocorrelation operation on the texture image in the frequency domain to obtain the autocorrelation image.
According to the motion deblurring method based on the auto-correlation of the texture image, the initial blur kernel is determined based on the auto-correlation image, and the motion deblurring method comprises the following steps:
determining a motion mode of an image acquisition device for shooting the image to be processed based on the autocorrelation image;
the initial blur kernel is determined based on the motion pattern.
According to the motion deblurring method based on the auto-correlation of the texture image, the deblurring model is solved based on the initial blur kernel to obtain a restored image of the image to be processed after the motion blur, and the method comprises the following steps:
splitting the deblurring model into a first model corresponding to the restored image variable and a second model corresponding to the fuzzy core variable;
carrying out iterative solution on the first model and the second model based on a coordinate descent method and the initial fuzzy kernel, solving the first model based on an intermediate fuzzy kernel corresponding to the fuzzy kernel variable obtained in the previous iteration process and an intermediate image corresponding to the recovered image variable obtained in the previous iteration process in each iteration process to obtain an intermediate image corresponding to the recovered image variable in the current iteration process, and solving the second model based on the intermediate image corresponding to the recovered image variable in the current iteration process to obtain an intermediate fuzzy kernel corresponding to the fuzzy kernel variable in the current iteration process;
and when the iteration process reaches the preset times, deconvoluting the final image and the final fuzzy core obtained in the last iteration process, and taking the obtained deconvolution result as the restored image.
According to the motion deblurring method based on the auto-correlation of the texture image, the first model is solved based on the intermediate blur kernel corresponding to the blur kernel variable obtained in the previous iteration process and the intermediate image corresponding to the restored image variable obtained in the previous iteration process, so as to obtain the intermediate image corresponding to the restored image variable in the current iteration process, and the motion deblurring method comprises the following steps:
introducing auxiliary variables related to gradients of the restored image variables by using a semi-quadratic splitting method, and updating the first model based on the auxiliary variables to obtain an updated first model;
and converting the updated first model into a first least square problem based on the intermediate image obtained in the previous iteration process, and solving the first least square problem based on the intermediate blur kernel obtained in the previous iteration process to obtain the intermediate image in the current iteration process.
According to the motion deblurring method based on the auto-correlation of the texture image, the first least square problem is solved based on the intermediate blur kernel obtained in the previous iteration process, and an intermediate image in the current iteration process is obtained, and the motion deblurring method comprises the following steps:
and solving the first least square problem based on a fast Fourier transform method and an intermediate fuzzy kernel obtained in the previous iteration process to obtain an intermediate image in the current iteration process.
According to the motion deblurring method based on the auto-correlation of the texture image, the second model is solved based on the intermediate image corresponding to the restored image variable in the current iteration process, so as to obtain the intermediate blur kernel corresponding to the blur kernel variable in the current iteration process, and the motion deblurring method comprises the following steps:
and solving the second model based on a fast Fourier transform method and an intermediate latent image in the current iteration process to obtain an intermediate fuzzy core in the current iteration process.
The invention also provides a motion deblurring device based on the auto-correlation of texture images, which comprises:
the image acquisition module is used for acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image;
the model construction module is used for determining a model data item based on the image to be processed and the blur kernel variable and the restored image variable corresponding to the image to be processed and taking L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built;
and the model solving module is used for solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after deblurring.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a texture image auto-correlation based motion deblurring method as described in any of the above when executing the program.
The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a texture image autocorrelation-based motion deblurring method as described in any one of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a texture image auto-correlation based motion deblurring method as described in any one of the above.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a motion deblurring method and a motion deblurring device based on auto-correlation of a texture image, wherein the method comprises the steps of firstly obtaining an image to be processed, calculating an auto-correlation image of the texture image corresponding to the image to be processed, and determining an initial blur kernel based on the auto-correlation image; then based on the image to be processed and the blur kernel variable and the restored image change corresponding to the image to be processedQuantity, determining model data item and using L of fuzzy kernel variable 2 The square of the norm is the first regularization term, L, to recover the gradient of the image variable 0 The norm is the second regular term to recover the L of the image variable 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built; and finally, solving the deblurring model by utilizing the initial blurring kernel to obtain a restored image of the image to be processed after deblurring. According to the method, the initial fuzzy core is determined through the autocorrelation image, and the de-fuzzy model is solved, so that the fuzzy core estimated by the method is better in convergence, the estimated fuzzy core is more accurate, noise can be processed, the structure is clear, and sharp edges are kept, so that the effect of acquiring more accurate recovered images is achieved. Meanwhile, the method can also eliminate the influence of deformation, noise and ringing artifacts, and can be suitable for blind deblurring of various images such as natural images, face images, low-illumination images and the like to obtain a restored image with clear main structure.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to these drawings without inventive effort.
FIG. 1 is a schematic flow chart of a motion deblurring method based on auto-correlation of texture images according to the present invention;
FIG. 2 is a schematic diagram of a motion deblurring apparatus based on auto-correlation of texture images according to the present invention;
fig. 3 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Image motion deblurring is a non-negligible problem in computer vision and computed radiography. The main factors causing the blurring of the image are camera shake, movement of an object, camera defocus, and the like. The main objective of motion blur estimation is to recover the corresponding sharp image from the acquired blurred image. Most current image deblurring methods solve the problem by imposing various priors on the blur kernel and the potential image. The complexity of the algorithm is improved, the operation speed is reduced, even the fuzzy kernel estimation is inaccurate due to the lack of fuzzy information, and the effect of finally restoring the image is poor.
Moreover, most of the existing blind deblurring methods of images cannot effectively process noise, deformation and ringing artifacts can be generated on the restored clear images, maximum image restoration cannot be realized, convergence of a blur kernel is not strong enough, and calculation cost is high. Based on the above, the embodiment of the invention provides an image blind motion blur removing method, which is characterized in that a high-quality initial blur kernel is directly obtained on an image to be processed by researching a frequency domain problem, the initial blur kernel is refined, and a more accurate final blur kernel is estimated. The final blur kernel estimated by the motion deblurring method based on the texture image autocorrelation provided by the embodiment of the invention has better convergence, can process noise, and has clear structure of the obtained restored image, can keep sharp edges and has better effect of restoring the image.
Fig. 1 is a schematic flow chart of a motion deblurring method based on auto-correlation of a texture image according to an embodiment of the present invention, as shown in fig. 1, the method includes:
s1, acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image;
s2, based on the image to be processed and fuzzy kernel variables and corresponding fuzzy kernel variables of the image to be processedRestoring image variables, determining model data items, and using L of the fuzzy kernel variables 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built;
and S3, solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after deblurring.
Specifically, in the motion deblurring method based on auto-correlation of texture images provided in the embodiment of the present invention, the execution subject is a processor, and the processor may be a local computer or a cloud computer, and the local computer may be a computer, a tablet, or the like, which is not limited herein.
Firstly, step S1 is executed to acquire an image to be processed, and an autocorrelation image of a texture image corresponding to the image to be processed is determined. The image to be processed refers to a blurred image which has motion blur and needs to be subjected to motion blur removal operation, and the blurred image can be a face image, a large-size substrate image or an image in other fields, and is not particularly limited herein. The image to be processed can be obtained by shooting through image acquisition equipment such as a camera.
The method belongs to a blind deblurring method, and aims to recover clear restored images and blur kernels from blurred images to be processed. Regardless of noise, the blurring process of the restored image can be expressed as a convolution of the restored image with a blurring kernel:
； （1）
where B represents a known image to be processed, I represents a restored image variable, k represents a blur kernel variable,is a convolution operator. This blurring problem is highly uncertain because the different restored images and blurring kernels can beThe same blurred image can be generated.
The aim of the embodiment of the invention is to directly obtain high-quality initial blur kernels from the image to be processed by researching the problem in the frequency domain. It is necessary to provide motion patterns, i.e. behavior information, about the image acquisition device causing the blur to obtain the initial blur kernel. The motion pattern may include a motion direction and a motion amplitude of the image capturing device.
In order to obtain the motion mode of the image acquisition device, a texture image of the image to be processed can be introduced, wherein the texture image can be obtained by adopting an iterative least square method, and can be a one-dimensional image or a two-dimensional image.
The texture image is used for representing texture characteristics of an image to be processed, is obtained by carrying out statistical calculation in an area containing a plurality of pixel points, has rotation invariance and has stronger resistance to noise. Texture features are a visual feature reflecting homogeneity in the image to be processed, which represents structural organization arrangement properties of the object surface. Texture features have three major markers: some local sequence is repeated continuously, non-randomly arranged, and the texture area is a uniform entity. The texture features are different from image features such as gray level, color and the like, are represented by radian distribution of pixels and surrounding spatial neighbors thereof, have local properties, and represent global properties when the local texture features repeatedly appear.
When the texture features represent global properties, the surface properties of the scene to which the image or image region to be processed corresponds may be described. However, since texture is only a characteristic of the surface of an object, and the intrinsic properties of the object cannot be fully reflected, high-level image contents cannot be obtained by using only texture features.
Since the autocorrelation function has a good effect in calculating the texture period, in the embodiment of the invention, the autocorrelation image of the texture image corresponding to the image to be processed needs to be determined. The gray values of each point on the autocorrelation image can represent the horizontal movement information and the vertical movement information of the image, namely the motion mode of the image acquisition equipment required by the fuzzy kernel estimation can be displayed.
Here, the texture image may be converted into a frequency domain by a fast fourier transform method, and an autocorrelation operation is performed on the texture image in the frequency domain, to obtain an autocorrelation image.
The autocorrelation image can be calculated by the following formula:
； （2）
wherein,is an autocorrelation image, L is a texture image, < ->For the fast Fourier transform function, < >>Is complex conjugate operator>As an inverse fast fourier function->Is an autocorrelation operation symbol.
Multiple copies of the restored image I are displayed in the texture image L, the most obvious repeated edges being due to the fact that the main peak of k indicates the start and end of movement of the image acquisition device.
Auto-correlation image of texture image LThe composition of one main peak plus two side peaks separated by the range and direction of the blur kernel will show several bright spots indicating the motion pattern of the image acquisition device. Thus, the autocorrelation image of the texture image L +.>The represented information can record the motion mode of the image acquisition device and provide reliable information for obtaining the blur kernel.
For the simple linearityMotion-generated blur by computing an autocorrelation image of a texture image LThe motion pattern of the image acquisition device can be obtained. And then determining an initial blur kernel according to the extracted motion mode, wherein the motion amplitude determines the size of the initial blur kernel, and the non-zero blur kernel values are uniformly distributed along the motion direction of the image acquisition equipment.
Then, step S2 is executed to determine the model data item by using the image B to be processed and the blur kernel variable k and the restored image variable I corresponding to the image B to be processed, and the model data item is represented by L of the blur kernel variable k 2 The square of the norm is the first regularization term, L, to recover the gradient of the image variable I 0 The norm is the second regular term to recover the L of the image variable I 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built.
In general, in the case of spatially invariant (uniform) nonlinear blurring, the blurring process can be represented by equation (1), and in the case of noise consideration, the deblurring problem with known blurring kernel can be represented as finding. However, in most cases, blurring is the loss of high frequency information, which acts as a form of low pass filter, a problem that is not benign. The convolution of the known blur kernel k is regarded as a linear operator, with a characteristic value close to zero, corresponding to the high frequency component of the signal (image). The deblurring process is to recover the high frequency components lost in the image blurring process. If the high frequency components are over emphasized during deblurring, the resulting restored image will have noise or edge artifacts. For this purpose, in the embodiment of the invention, a second regularization term, namely an image gradient regularization term, is added>To block excessive high frequency components.
To avoid that the minima of the cost function obtained in many common image prior forms do not correspondThe problem of true clear solution. In the embodiment of the invention, the image is adoptedNorms and +.>The ratio of norms serves as a third regularization term providing the lowest cost for restoring the image. The third regularization term is a regularization term with unchanged scale, can compensate high-frequency attenuation, and greatly stabilizes the estimation process of the fuzzy core.
Therefore, the model data item can restrict the recovered convolution result of the value of the restored image variable and the value of the fuzzy core variable to be similar to the image to be processed, the first regularization item is used for regularizing the value of the fuzzy core variable, the second regularization item is used for reserving a large gradient and sharp edges by adopting image gradients and removing tiny details, the third regularization item is used for guaranteeing sparsity, and therefore the value of the fuzzy core variable is estimated stably.
In order to maintain the proper high frequency components, the deblurring model may be characterized based on the following formula:
； （3）
wherein,to restore the gradient of the image variable I +.>For model data item->For the first regularization term,/>For the second regularization term,/->For the third regularization term,/->、/>And->The weight parameters of the first regularization term, the second regularization term and the third regularization term are respectively.
And finally, executing step S3, solving the deblurring model by utilizing an initial blur kernel to obtain a restored image of the image to be processed after motion blurring, namely, solving the formula (3) to obtain a final value of the I, then continuously calculating the final value of a blur kernel variable, and deconvoluting the final value of the I and the final value of the blur kernel variable to obtain a result serving as the restored image. The restored image is the restored clear image.
According to the motion deblurring method based on the auto-correlation of the texture image, firstly, an image to be processed is obtained, an auto-correlation image of the texture image corresponding to the image to be processed is calculated, and an initial blur kernel is determined based on the auto-correlation image; then determining a model data item based on the image to be processed and the fuzzy kernel variable and the restored image variable corresponding to the image to be processed, and taking L of the fuzzy kernel variable 2 The square of the norm is the first regularization term, L, to recover the gradient of the image variable 0 The norm is the second regular term to recover the L of the image variable 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built; and finally, solving the deblurring model by utilizing the initial blurring kernel to obtain a restored image of the image to be processed after deblurring. According to the method, the initial fuzzy core is determined through the autocorrelation image, and the de-fuzzy model is solved, so that the fuzzy core estimated by the method is better in convergence, the estimated fuzzy core is more accurate, noise can be processed, the structure is clear, and sharp edges are kept, so that the effect of acquiring more accurate recovered images is achieved. At the same time, the method can eliminate the influence of deformation, noise and ringing artifacts, and is suitable for various images such as natural images, face images, low-illumination images and the likeBlind deblurring of the main structure to obtain a restored image with clear main structure.
On the basis of the above embodiment, the motion deblurring method based on auto-correlation of a texture image provided in the embodiment of the present invention solves the deblurring model based on the initial blur kernel to obtain a restored image of the image to be processed after deblurring, including:
splitting the deblurring model into a first model corresponding to the restored image variable and a second model corresponding to the fuzzy core variable;
carrying out iterative solution on the first model and the second model based on a coordinate descent method and the initial fuzzy kernel, solving the first model based on an intermediate fuzzy kernel corresponding to the fuzzy kernel variable obtained in the previous iteration process and an intermediate image corresponding to the recovered image variable obtained in the previous iteration process in each iteration process to obtain an intermediate image corresponding to the recovered image variable in the current iteration process, and solving the second model based on the intermediate image corresponding to the recovered image variable in the current iteration process to obtain an intermediate fuzzy kernel corresponding to the fuzzy kernel variable in the current iteration process;
and when the iteration process reaches the preset times, deconvoluting the final image and the final fuzzy core obtained in the last iteration process, and taking the obtained deconvolution result as the restored image.
Specifically, in the embodiment of the present invention, when the deblurring model is solved, the deblurring model may be split into the first model corresponding to the restored image variable and the second model corresponding to the blur kernel variable.
The first model may be expressed as:
； （4）
the second model may be expressed as:
； （5）
here, the first model and the second model may be alternately solved using an initial blur kernel and a coordinate descent method. In each iteration process, the intermediate fuzzy core obtained in the previous iteration process and the intermediate image obtained in the previous iteration process are utilized to solve the first model, the intermediate image in the current iteration process is obtained, and the second model is solved based on the intermediate image in the current iteration process, so that the intermediate fuzzy core in the current iteration process is obtained.
The intermediate image is the value of the restored image variable obtained in the current iteration process, and the intermediate fuzzy kernel is the value of the fuzzy kernel variable obtained in the current iteration process.
And when the iteration process reaches the preset times, deconvoluting the final image and the final fuzzy core obtained in the last iteration process, and taking the obtained deconvolution result as a restored image.
The above process, when solving the first model in each iteration process, is due toThe presence of norms, equation (4), is computationally difficult. Therefore, in order to solve->The problem that the norm is difficult to calculate is that a semi-quadratic splitting method is used, auxiliary variables related to gradients of restored image variables are introduced, and the first model is updated by using the auxiliary variables, so that an updated first model is obtained.
The updated first model may be expressed as:
； （6）
wherein u is an auxiliary variable, and there are，/>Representing the gradient of the restored image variable in the horizontal direction,gradient in vertical direction representing restored image variable, +.>Is a random integer>Is a penalty parameter. When->Approaching infinity, the solution of equation (6) approaches the solution of equation (4). At this time, equation (6) may be solved by alternately minimizing I and u while fixing other variables.
Based on the intermediate image obtained in the previous iteration process, the updated first model is converted into a first least square problem.
The first least squares problem can be expressed as:
； （7）
wherein,is a convolution (Toeplitz) matrix of k. />Is a vector form of u. The generation of matrix vectors for the convolution matrix may be achieved by using a Fast Fourier Transform (FFT). Here, the solution of I, i.e. the intermediate image obtained in the current iteration process, can be obtained by solving equation (7) according to the existing method.
The solution of equation (7) can be expressed as:
；（8）
wherein,and->Respectively->And->Is a vector representation of (c). And has the following steps: />C takes a fixed value in each iteration process.
Given I, a sub-problem with the auxiliary variable u can be solved, which can be expressed as:
； （9）
solving the formula (9) to obtain an auxiliary variable u as follows:
； （10）
and (3) obtaining the value of I, namely a final image, when the iteration times reach the preset times through alternative optimization solution of the formula (8) and the formula (10).
On the basis of the above embodiment, the motion deblurring method based on auto-correlation of texture images provided in the embodiment of the present invention solves the second model based on an intermediate image corresponding to the restored image variable in the current iteration process, to obtain an intermediate blur kernel corresponding to the blur kernel variable in the current iteration process, including:
and solving the second model based on a fast Fourier transform method and an intermediate latent image in the current iteration process to obtain an intermediate fuzzy core in the current iteration process.
Specifically, when the intermediate blur kernel is solved in each iteration process, since the intermediate image is known, the solution formula (5) can be regarded as solving the least square problem, and then the second model can be solved in the fourier domain by using the fast fourier transform method and the intermediate latent image in the current iteration process to obtain the intermediate blur kernel in the current iteration process, and the solution method can obtain a closed solution, namely:
； （11）
wherein the division and multiplication in equation (11) are performed by point.
In each iteration process, after the estimated intermediate fuzzy core is obtained, non-negativity and normalization processing can be carried out on the intermediate fuzzy core, so that the intermediate fuzzy core obtained by processing can meet the fuzzy core characteristics.
As shown in fig. 2, based on the above embodiment, an embodiment of the present invention provides a motion deblurring device based on auto-correlation of a texture image, including:
an image acquisition module 21, configured to acquire an image to be processed, calculate an autocorrelation image of a texture image corresponding to the image to be processed, and determine an initial blur kernel based on the autocorrelation image;
a model construction module 22 for determining a model data item based on the image to be processed and the blur kernel variable and the restored image variable corresponding to the image to be processed, and using L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built;
and the model solving module 23 is configured to solve the deblurring model based on the initial blur kernel, so as to obtain a restored image after the motion blur of the image to be processed is removed.
On the basis of the above embodiment, the motion deblurring device based on auto-correlation of texture images provided in the embodiment of the present invention is an image acquisition module, specifically configured to:
and converting the texture image into a frequency domain based on a fast Fourier transform method, and performing autocorrelation operation on the texture image in the frequency domain to obtain the autocorrelation image.
On the basis of the above embodiment, the motion deblurring device based on auto-correlation of texture images provided in the embodiment of the present invention is an image acquisition module, specifically configured to:
determining a motion mode of an image acquisition device for shooting the image to be processed based on the autocorrelation image;
the initial blur kernel is determined based on the motion pattern.
On the basis of the above embodiment, the motion deblurring device based on auto-correlation of texture images provided in the embodiment of the present invention is a model solving module, which is specifically configured to:
splitting the deblurring model into a first model corresponding to the restored image variable and a second model corresponding to the fuzzy core variable;
carrying out iterative solution on the first model and the second model based on a coordinate descent method and the initial fuzzy kernel, solving the first model based on an intermediate fuzzy kernel corresponding to the fuzzy kernel variable obtained in the previous iteration process and an intermediate image corresponding to the recovered image variable obtained in the previous iteration process in each iteration process to obtain an intermediate image corresponding to the recovered image variable in the current iteration process, and solving the second model based on the intermediate image corresponding to the recovered image variable in the current iteration process to obtain an intermediate fuzzy kernel corresponding to the fuzzy kernel variable in the current iteration process;
and when the iteration process reaches the preset times, deconvoluting the final image and the final fuzzy core obtained in the last iteration process, and taking the obtained deconvolution result as the restored image.
On the basis of the above embodiment, the motion deblurring device based on auto-correlation of texture images provided in the embodiment of the present invention is a model solving module, which is specifically configured to:
introducing auxiliary variables related to gradients of the restored image variables by using a semi-quadratic splitting method, and updating the first model based on the auxiliary variables to obtain an updated first model;
and converting the updated first model into a first least square problem based on the intermediate image obtained in the previous iteration process, and solving the first least square problem based on the intermediate blur kernel obtained in the previous iteration process to obtain the intermediate image in the current iteration process.
On the basis of the above embodiment, the motion deblurring device based on auto-correlation of texture images provided in the embodiment of the present invention is a model solving module, which is specifically configured to:
and solving the first least square problem based on a fast Fourier transform method and an intermediate fuzzy kernel obtained in the previous iteration process to obtain an intermediate image in the current iteration process.
On the basis of the above embodiment, the motion deblurring device based on auto-correlation of texture images provided in the embodiment of the present invention is a model solving module, which is specifically configured to:
and solving the second model based on a fast Fourier transform method and an intermediate latent image in the current iteration process to obtain an intermediate fuzzy core in the current iteration process.
Specifically, the functions of each module in the motion deblurring device based on the auto-correlation of the texture image provided in the embodiment of the present invention are in one-to-one correspondence with the operation flow of each step in the above method embodiment, and the achieved effects are consistent.
Fig. 3 illustrates a physical schematic diagram of an electronic device, as shown in fig. 3, where the electronic device may include: processor (Processor) 310, communication interface (Communications Interface) 320, memory (Memory) 330 and communication bus 340, wherein Processor 310, communication interface 320, memory 330 accomplish communication with each other through communication bus 340. Processor 310 may invoke the saveLogic instructions in the memory 330 to perform the texture image autocorrelation-based motion deblurring method provided in the embodiments described above, the method comprising: acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image; determining a model data item based on the image to be processed and a blur kernel variable and a restored image variable corresponding to the image to be processed, and taking L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built; and solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after the motion blurring is removed.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program storable on a non-transitory computer readable storage medium, the computer program being executable by a processor to perform the texture image autocorrelation based provided in the above embodimentsA method of motion deblurring, the method comprising: acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image; determining a model data item based on the image to be processed and a blur kernel variable and a restored image variable corresponding to the image to be processed, and taking L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built; and solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after the motion blurring is removed.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the texture image autocorrelation-based motion deblurring method provided in the above embodiments, the method comprising: acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image; determining a model data item based on the image to be processed and a blur kernel variable and a restored image variable corresponding to the image to be processed, and taking L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built; and solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after the motion blurring is removed.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A motion deblurring method based on auto-correlation of a texture image, comprising:
acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image;
determining a model data item based on the image to be processed and a blur kernel variable and a restored image variable corresponding to the image to be processed, and taking L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built;
and solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after the motion blurring is removed.
2. The motion deblurring method based on auto-correlation of texture images according to claim 1, wherein calculating an auto-correlation image of a texture image corresponding to the image to be processed comprises:
and converting the texture image into a frequency domain based on a fast Fourier transform method, and performing autocorrelation operation on the texture image in the frequency domain to obtain the autocorrelation image.
3. The texture image auto-correlation based motion deblurring method according to claim 1, wherein determining an initial blur kernel based on the auto-correlation image comprises:
determining a motion mode of an image acquisition device for shooting the image to be processed based on the autocorrelation image;
the initial blur kernel is determined based on the motion pattern.
4. A motion deblurring method based on auto-correlation of a texture image according to any one of claims 1-3, wherein solving the deblurring model based on the initial blur kernel to obtain a restored image of the image to be processed after the motion blur comprises:
splitting the deblurring model into a first model corresponding to the restored image variable and a second model corresponding to the fuzzy core variable;
carrying out iterative solution on the first model and the second model based on a coordinate descent method and the initial fuzzy kernel, solving the first model based on an intermediate fuzzy kernel corresponding to the fuzzy kernel variable obtained in the previous iteration process and an intermediate image corresponding to the recovered image variable obtained in the previous iteration process in each iteration process to obtain an intermediate image corresponding to the recovered image variable in the current iteration process, and solving the second model based on the intermediate image corresponding to the recovered image variable in the current iteration process to obtain an intermediate fuzzy kernel corresponding to the fuzzy kernel variable in the current iteration process;
and when the iteration process reaches the preset times, deconvoluting the final image and the final fuzzy core obtained in the last iteration process, and taking the obtained deconvolution result as the restored image.
5. The motion deblurring method based on auto-correlation of texture images according to claim 4, wherein solving the first model based on an intermediate blur kernel corresponding to the blur kernel variable obtained in a previous iteration process and an intermediate image corresponding to the restored image variable obtained in a previous iteration process to obtain an intermediate image corresponding to the restored image variable in a current iteration process comprises:
introducing auxiliary variables related to gradients of the restored image variables by using a semi-quadratic splitting method, and updating the first model based on the auxiliary variables to obtain an updated first model;
and converting the updated first model into a first least square problem based on the intermediate image obtained in the previous iteration process, and solving the first least square problem based on the intermediate blur kernel obtained in the previous iteration process to obtain the intermediate image in the current iteration process.
6. The motion deblurring method based on auto-correlation of texture images according to claim 5, wherein solving the first least squares problem based on the intermediate blur kernel obtained in the previous iteration process to obtain an intermediate image in the current iteration process comprises:
and solving the first least square problem based on a fast Fourier transform method and an intermediate fuzzy kernel obtained in the previous iteration process to obtain an intermediate image in the current iteration process.
7. The motion deblurring method based on auto-correlation of texture images according to claim 4, wherein solving the second model based on the intermediate image corresponding to the restored image variable in the current iteration process to obtain the intermediate blur kernel corresponding to the blur kernel variable in the current iteration process comprises:
and solving the second model based on a fast Fourier transform method and an intermediate latent image in the current iteration process to obtain an intermediate fuzzy core in the current iteration process.
8. A motion deblurring apparatus based on auto-correlation of a texture image, comprising:
the image acquisition module is used for acquiring an image to be processed, calculating an autocorrelation image of a texture image corresponding to the image to be processed, and determining an initial blur kernel based on the autocorrelation image;
the model construction module is used for determining a model data item based on the image to be processed and the blur kernel variable and the restored image variable corresponding to the image to be processed and taking L of the blur kernel variable 2 The square of the norm is the first regularization term, L, with the gradient of the restored image variable 0 The norm is the second regular term, and the L of the image variable is restored by the second regular term 1 Norms and L 2 The ratio of norms is a third regular term, and a deblurring model is built;
and the model solving module is used for solving the deblurring model based on the initial blurring kernel to obtain a restored image of the image to be processed after deblurring.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a texture image autocorrelation-based motion deblurring method as claimed in any one of claims 1 to 7 when the program is executed.
10. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a texture image autocorrelation based motion deblurring method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311362350.8A CN117132505B (en) | 2023-10-20 | 2023-10-20 | Motion deblurring method and device based on auto-correlation of texture image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311362350.8A CN117132505B (en) | 2023-10-20 | 2023-10-20 | Motion deblurring method and device based on auto-correlation of texture image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117132505A true CN117132505A (en) | 2023-11-28 |
| CN117132505B CN117132505B (en) | 2023-12-22 |
Family
ID=88854827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311362350.8A Active CN117132505B (en) | 2023-10-20 | 2023-10-20 | Motion deblurring method and device based on auto-correlation of texture image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117132505B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104112261A (en) * | 2014-07-17 | 2014-10-22 | 五邑大学 | Norm ratio regularization based rapid image blind deblurring method |
| CN108376393A (en) * | 2018-03-16 | 2018-08-07 | 华南理工大学 | A kind of blurred picture blind restoration method towards high-speed straight-line Moving Objects |
| CN109767404A (en) * | 2019-01-25 | 2019-05-17 | 重庆电子工程职业学院 | Infrared image deblurring method under a kind of salt-pepper noise |
-
2023
- 2023-10-20 CN CN202311362350.8A patent/CN117132505B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104112261A (en) * | 2014-07-17 | 2014-10-22 | 五邑大学 | Norm ratio regularization based rapid image blind deblurring method |
| CN108376393A (en) * | 2018-03-16 | 2018-08-07 | 华南理工大学 | A kind of blurred picture blind restoration method towards high-speed straight-line Moving Objects |
| CN109767404A (en) * | 2019-01-25 | 2019-05-17 | 重庆电子工程职业学院 | Infrared image deblurring method under a kind of salt-pepper noise |
Non-Patent Citations (2)
| Title |
|---|
| 余义斌: "凹凸范数比值正则化的快速图像盲去模糊", 电子学报, vol. 44, no. 5, pages 1168 - 1172 * |
| 电子学报: "基于深度先验的盲图像去模糊算法", 白勇强, vol. 51, no. 4, pages 1051 - 1055 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117132505B (en) | 2023-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhu et al. | Removing atmospheric turbulence via space-invariant deconvolution | |
| CN111275626B (en) | Video deblurring method, device and equipment based on ambiguity | |
| Ren et al. | Image deblurring via enhanced low-rank prior | |
| US9692939B2 (en) | Device, system, and method of blind deblurring and blind super-resolution utilizing internal patch recurrence | |
| TW200840365A (en) | Motion-blur degraded image restoration method | |
| CN110675347A (en) | Image blind restoration method based on group sparse representation | |
| Sada et al. | Image deblurring techniques—a detail review | |
| Anger et al. | Efficient blind deblurring under high noise levels | |
| Laroche et al. | Deep model-based super-resolution with non-uniform blur | |
| Caliskan et al. | Atmospheric turbulence mitigation using optical flow | |
| CN110852947B (en) | Infrared image super-resolution method based on edge sharpening | |
| Mahalakshmi et al. | A survey on image deblurring | |
| CN117132505B (en) | Motion deblurring method and device based on auto-correlation of texture image | |
| Aswathi et al. | A review on image restoration in medical images | |
| Goilkar et al. | Implementation of blind and non-blind deconvolution for restoration of defocused image | |
| Nunes et al. | A novel approach for mitigating atmospheric turbulence using weighted average Sobolev gradient and Laplacian | |
| Sola et al. | Blind image deblurring based on multi-resolution ringing removal | |
| He et al. | Joint motion deblurring and superresolution from single blurry image | |
| CN113393408A (en) | Multi-focus image processing method based on dual-tree complex wavelet and convolutional neural network | |
| Chen et al. | Image deblurring method based on fourier transform | |
| El Abbadi et al. | A Survey on Blind De-Blurring of Digital Image. | |
| Latha et al. | Simultaneous denoising and deblurring by non-convex regularization: alternating minimization framework | |
| CN117710250B (en) | Method for eliminating honeycomb structure imaged by fiberscope | |
| CN117115044A (en) | Image blind motion blur removing method and device, electronic equipment and storage medium | |
| Abdulkhaleq et al. | A Survey on Blind De-Blurring of Digital Image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |