CN106651790B

CN106651790B - Image deblurring method, device and equipment

Info

Publication number: CN106651790B
Application number: CN201611032330.4A
Authority: CN
Inventors: 陶鑫; 贾佳亚; 鲁亚东
Original assignee: Hangzhou Huawei Digital Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2016-11-22
Filing date: 2016-11-22
Publication date: 2020-01-31
Anticipated expiration: 2036-11-22
Also published as: CN106651790A

Abstract

The embodiment of the invention provides image deblurring methods, devices and equipmentA blur kernel of the layer; according to the formula

Determining a background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_R. The method of the embodiment of the invention respectively considers the fuzzy kernel of the background layer and the fuzzy kernel of the reflecting layer of the image to be processed, and separates and deblurs the background layer and the reflecting layer based on the two fuzzy kernels, thereby not only improving the deblurring accuracy of the image, but also enabling the restored image to be clearer.

Description

Image deblurring method, device and equipment

Technical Field

The embodiment of the invention relates to an image processing technology, in particular to image deblurring methods, devices and equipment.

Background

In the process of actually taking a picture, because of the shake of a camera or the motion of an object, the motion blur of the taken picture often occurs, especially under the conditions that the ambient light is dark and the time is is long, the probability of the image blur is greatly improved.

In the prior art, an image Deblurring algorithm is proposed by Xu & jiaa in 2010 ECCV conference article "Two-phase Kernel Estimation for Robust Motion Deblurring", and specifically, starting from the bottom layer of an image pyramid, a multi-scale (multi-scale) iteration is used for optimizing a Motion blur kernel of an image from small to large, then an ISD technology is used for carrying out -step optimization on the estimated Motion blur kernel, and finally a clear result image is obtained after deconvolution is carried out by adopting a fast TV-L1 deconvolution algorithm based on the estimated Motion blur kernel.

However, for a picture with motion blur and light reflection, the part of the strongest edge in the picture belongs to a background layer, but part of the strongest edge belongs to a light reflection layer, namely the strong edges are not generated by fuzzy cores, while the image deblurring method in the prior art is based on a prior condition that no light reflection exists in the picture, so the prior condition is not satisfied in the picture with the light reflection layer, and the image deblurring method in the prior art cannot obtain an accurate result.

Disclosure of Invention

The embodiment of the invention provides image deblurring methods, devices and equipment, which are used for solving the technical problem that when an image with a reflective layer is deblurred in the prior art, the obtained deblurring result is low in accuracy.

, embodiments of the present invention provide a method for deblurring images, comprising:

acquiring a fuzzy kernel of a background layer and a fuzzy kernel of a reflecting layer of an image to be processed;

according to the formula

Determining a background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_R(ii) a Wherein, B is a pixel matrix of an image to be processed, and K is_BIs a blur kernel of the background layer, the K_RIs a blur kernel of the light-reflecting layer, the K_iIs K_BOr K_RSaid L is_iIs L_BOr L_RSaid α_lThe factor for characterizing the degree of gradient distribution of the original sharp image, the μ_dAnd mu_lAnd adjusting factors for adjusting the smoothness degree of the restored clear image.

The image demould provided by the embodiment of the inventionThe blurring method comprises the steps of obtaining a blurring kernel of a background layer and a blurring kernel of a reflecting layer of an image to be processed, and then obtaining a blurring kernel of a background layer and a blurring kernel of a reflecting layer of the image to be processed according to a formula

Determining a background layer L of an image to be processed_BAnd a light-reflecting layer L of the image to be processed_RTherefore, the image with the definition meeting the requirements of the user is restored. Because the fuzzy kernel of the background layer and the fuzzy kernel of the reflecting layer of the image to be processed are respectively considered in the embodiment of the invention, and the background layer and the reflecting layer are separated and deblurred simultaneously by adopting the formula 1 based on the two fuzzy kernels, the deblurring accuracy of the image is improved, and the restored image is clearer.

In possible designs, the acquiring the blur kernel of the background layer and the blur kernel of the light reflection layer of the image to be processed specifically includes:

clustering each image block in the image to be processed to obtain th background layer L_B' and th light-reflecting layer L_R'；

According to the formula

And said th background layer L_B', obtaining a blur kernel K for the background layer_B(ii) a Wherein, 0 is<λ_k≤1；

According to the formula

And said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_R。

In possible designs, clustering each image block in the image to be processed to obtain a th background layer L_B' and th light-reflecting layer L_R', specifically includes:

according to the formula

For each image block in the image to be processedFourier transform is carried out to obtain a plurality of th image blocks, wherein P is_xIs an image block of the image to be processed, the

A Fourier transform of a 5-point Laplace kernel;

according to the formula

Determining a clustering distance between any two th image blocks, and clustering the image blocks of the image to be processed according to the clustering distance to obtain a th cluster and a second cluster, wherein the th cluster comprises at least pixel values of the image blocks which belong to a background layer, the second cluster comprises at least pixel values of the image blocks which belong to the background layer, and the P cluster comprises at least pixel values of the image blocks which belong to the background layer_yThe image blocks of the image to be processed are obtained;

obtaining the background layer L according to the th cluster_B'；

Obtaining the th light-reflecting layer L according to the second polymer_R'。

In possible designs, the μ _d1 and μ_l＝1e-1。

The method for deblurring the image, provided by the embodiment of the invention, obtains the th background layer L by clustering each image block in the image to be processed_B' and th light-reflecting layer L_R', then according to the formula

And th background layer L_B', obtaining a blur kernel K for the background layer_BAnd according to a formula

And th light-reflecting layer L_R', obtaining a fuzzy kernel K of the light-reflecting layer_RAnd further determining the background layer L of the image to be processed according to the formula 1_BAnd a reflective layer of the image to be processed, so that the image with the definition meeting the requirements of the user is restored. Book (I)Fuzzy kernel K of background layer determined by embodiment of the invention_BAnd a blurring kernel K of the light-reflecting layer_RIn addition, because the fuzzy kernel of the background layer of the image to be processed and the fuzzy kernel of the reflecting layer are respectively considered in the embodiment of the invention, and the background layer and the reflecting layer are simultaneously separated and deblurred by adopting the formula 1 based on the two fuzzy kernels, the deblurring accuracy of the image is improved, and the restored image is clearer.

In possible designs, the formula is used

And said th background layer L_B', obtaining a blur kernel K for the background layer_BAnd, according to the formula

And said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_RThe method specifically comprises the following steps:

step A: according to the formula

And said th background layer L_B' obtaining th initialization blur kernel K of the background layer_B'; and, according to the formulaAnd said th light-reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R'；

And B: according to the formula

The th initialization blur kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R"; wherein, 0 is<λ_l≤1；

And C: will be describedThe second background layer L_B"as a new th background layer L_B', and combining said second light-reflecting layer L_R"as a new th light-reflecting layer L_R' executing the step A until the iteration number reaches a preset number, and taking the th initialized fuzzy kernel obtained in the last iterations as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R。

Of the possible designs, the α_lEqual to 0.8.

In the method for deblurring an image, which is provided by the embodiment of the invention, the th initialization blur kernel K of the background layer is obtained through the step A_B' and second initialization blur kernel of light-reflecting layer K_R' and then a second background layer L is determined by the above step B_BAnd a second light-reflecting layer L_R", and then a second background layer L_B"as a new th background layer L_B', and a second light-reflecting layer L_R"as a new th light-reflecting layer L_R' executing the step A until the iteration number reaches a preset number, and taking the th initialized fuzzy core acquired in the last iterations as the fuzzy core K of the background layer_BAnd taking the second initialized fuzzy kernel obtained in the last iterations as the fuzzy kernel K of the light reflecting layer_R. The method of the embodiment of the invention determines the fuzzy kernel K of the background layer of the image to be processed through multiple iterations_BAnd a blurring kernel K of the light-reflecting layer_RSo that the two obtained fuzzy kernels are more accurate, and the finally determined background layer L of the image to be processed is greatly improved_BAnd a light-reflecting layer L of the image to be processed_RThe restored image is clearer due to the accuracy of the image restoration method.

In a second aspect, an embodiment of the present invention provides apparatus for deblurring an image, including:

the acquisition module is used for acquiring a fuzzy kernel of a background layer and a fuzzy kernel of a reflecting layer of the image to be processed;

a determination module for determining the formulaDetermining a background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_R(ii) a Wherein, B is a pixel matrix of an image to be processed, and K is_BIs a blur kernel of the background layer, the K_RIs a blur kernel of the light-reflecting layer, the K_iIs K_BOr K_RSaid L is_iIs L_BOr L_RSaid α_lThe factor for characterizing the degree of gradient distribution of the original sharp image, the μ_dAnd mu_lAnd adjusting factors for adjusting the smoothness degree of the restored clear image.

In possible designs, the obtaining module includes:

a clustering unit for clustering each image block in the image to be processed to obtain th background layer L_B' and th light-reflecting layer L_R'；

An acquisition unit for acquiring the formula

And said th background layer L_B', obtaining a blur kernel K for the background layer_B(ii) a And, according to the formula

And said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_R(ii) a Wherein, 0 is<λ_k≤1。

In possible designs, the obtaining unit is specifically configured to obtain the formula

And said th background layer L_B' obtaining th initialization blur kernel K of the background layer_B'; and, according to the formula

And said thLight reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R'; and according to the formula

The th initialization blur kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R", and applying said second background layer L_B"as a new th background layer L_B', and combining said second light-reflecting layer L_R"as a new th light-reflecting layer L_R', return to determining a new th initialization fuzzy kernel K_B' and a new second initialization fuzzy core K_R', until the iteration number reaches the preset number, and the -th initialized fuzzy kernel obtained in the last iterations is used as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R(ii) a Wherein, 0 is<λ_l≤1。

Of the possible designs, the α_lEqual to 0.8.

In possible designs, the clustering unit includes:

a processing subunit for processing according to a formula

Fourier transform is carried out on each image block in the image to be processed to obtain a plurality of th image blocks, wherein P is_xIs an image block of the image to be processed, the

A Fourier transform of a 5-point Laplace kernel;

a clustering subunit for clustering according to a formulaDetermining a clustering distance between any two th image blocks, and mapping the to-be-processed image according to the clustering distanceClustering image blocks of the images to obtain th cluster and a second cluster, wherein the th cluster comprises at least pixel values of the image blocks which belong to the background layer, the second cluster comprises at least pixel values of the image blocks which belong to the background layer, and the P cluster comprises a plurality of P clusters_yThe image blocks of the image to be processed are obtained;

an obtaining subunit, configured to obtain the th background layer L according to the th cluster_B' and, obtaining said th retroreflective layer L from said second polymer_R'。

In possible designs, the μ _d1 and μ_l＝1e-1。

The advantages of the image deblurring apparatus provided by the second aspect and the possible designs of the second aspect can refer to the advantages brought by the aspect and the aspect, and are not described herein again.

In a third aspect, embodiments of the invention provide image deblurring devices, including an input device and a processor;

the processor is used for acquiring the fuzzy kernel of the background layer and the fuzzy kernel of the reflecting layer of the image to be processed input by the input equipment and according to a formula

In possible designs, the processor is specifically configured to perform clustering on each image block in the image to be processed to obtain a clusterTo th background layer L_B' and th light-reflecting layer L_RAccording to formula

In possible designs, the processor is specifically configured to be based on a formula

And said th background layer L_B' obtaining th initialization blur kernel K of the background layer_B'; and, according to the formulaAnd said th light-reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R'; and according to the formula

Of the possible designs, the α_lEqual to 0.8.

In possible designs, the processor is specifically configured to be based on a formulaFourier transform is carried out on each image block in the image to be processed to obtain a plurality of th image blocks, and the th image blocks are obtained according to a formula

Determining the clustering distance between any two th image blocks, clustering the image blocks of the image to be processed according to the clustering distance to obtain a th cluster and a second cluster, and obtaining the th background layer L according to the th cluster_B', and obtaining said th retroreflective layer L from said second polymer_R'; wherein, the P_xIs an image block of the image to be processed, the

A Fourier transform of a 5-point Laplacian kernel, said th cluster comprising pixel values of at least image blocks belonging to the background layer, said second cluster comprising pixel values of at least image blocks belonging to the background layer, said P_yIs an image block of the image to be processed.

In possible designs, the μ _d1 and μ_l＝1e-1。

The beneficial effects of the image deblurring device provided by the above possible designs of the third aspect and the third aspect can refer to the beneficial effects brought by the th aspect and the th aspect, and are not described herein again.

Drawings

FIG. 1 is a schematic diagram of a blurred image with reflection according to an embodiment of the present invention;

FIG. 2 is a flow chart illustrating an embodiment of a method for deblurring an image according to an embodiment of the present invention;

FIG. 3 is a comparison diagram illustrating the deblurring effect of an image according to an embodiment of the present invention;

FIG. 4 is a second comparison diagram illustrating the deblurring effect of an image according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating a second method for deblurring an image according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating a third method for deblurring an image according to an embodiment of the present invention;

FIG. 7 is a flowchart illustrating a fourth embodiment of a method for deblurring an image according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an embodiment of an apparatus for deblurring an image according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a second embodiment of an apparatus for deblurring an image according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of a third embodiment of an apparatus for deblurring an image according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of an image deblurring apparatus provided in an embodiment of the present invention.

Detailed Description

The method, the device and the equipment for deblurring the image provided by the embodiment of the invention can be suitable for scenes which do not meet the definition requirement of a shot picture and need to be subjected to image restoration due to camera shake, object movement, a reflective surface of an object and the like when the shot picture is shot.

The image processing device according to the embodiment of the present invention may be a computer, a mobile terminal, and the like having an image processing function, and the mobile terminal may be a mobile phone, a tablet computer, a personal digital assistant, and the like of a user.

However, for a motion-blurred and reflective picture, the strongest edge in the picture is subordinate to a background layer, but some strongest edges belong to a reflective layer, i.e., the strong edges are not generated by types of blur kernels, for example, referring to the label shown in fig. 1, the strong edge marked with the symbol "1 #" is subordinate to the background layer, and the strong edge marked with the symbol "2 #" is subordinate to the reflective layer.

Therefore, the image deblurring method provided by the embodiment of the invention aims to solve the above technical problems in the prior art.

The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 2 is a schematic flowchart of an embodiment of a method for deblurring an image according to an embodiment of the present invention, in this embodiment, an image processing apparatus separates a background layer and a reflective layer based on an image imaging model, where B is L_B*K_B+L_R*K_R+ n; wherein L is_BFor a clear background layer to be estimated (i.e. clear to be separated out)Background layer), L_RIs the clear reflector to be estimated (i.e. the clear reflector desired to be separated), K_BIs a fuzzy kernel of the background layer of the current image to be processed, K_RThe method is characterized in that the method is a fuzzy kernel of a reflecting layer of a current image to be processed, B is a actually shot fuzzy picture with reflection (namely the image to be processed), and n is noise. In the imaging model, the shot picture B is a known quantity, and the separation of the background layer and the light reflecting layer of the image to be processed is actually to solve the L in the imaging model of the image_BAnd L_RThe process of (1).

It should be noted that, in the image imaging model, B is actually a pixel matrix of the image to be processed in the formula, and K is described above_B、K_R、L_B、L_RAre all in the form of a matrix.

As shown in fig. 2, the method comprises the steps of:

s101: and acquiring a fuzzy kernel of a background layer and a fuzzy kernel of a reflecting layer of the image to be processed.

Optionally, the image processing device may iteratively optimize the motion blur kernel of the image to be processed from a bottom layer of an image pyramid in a multi-scale (multi-scale) from small to large manner, and then optimize the estimated motion blur kernel by steps using an ISD technology to obtain a blur kernel K of the background layer_B(ii) a Optionally, the image processing apparatus may also obtain the blur kernel K of the reflective layer by applying different prior constraints to different image layers_R. In summary, the embodiment of the present invention does not limit how to obtain the blur kernel of the background layer and the blur kernel of the reflective layer of the image to be processed, as long as the image processing apparatus can respectively consider and obtain the blur kernels of the two layers when determining that the image to be processed has both the background layer and the reflective layer.

S102: according to the formula

Determining a background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_R。

Specifically, when the image processing apparatus obtains the blur kernel K of the background layer_BAnd a blurring kernel K of the light-reflecting layer_RThen, the image processing apparatus calculates the formula

(equation 1) solving the background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_RI.e. L which minimizes the value to the right of the equation of equation 1 during the solution process_BAnd L_RThen is L to be finally obtained_BAnd L_R. Wherein, B is the pixel matrix of the image to be processed, K_BBlur kernel for background layer obtained at S101, K_RK in equation 1 for the blurring kernel of the light-reflecting layer obtained in the above-described S101_iIs K_BOr K_R，L_iIs L_BOr L_Rα as described above_lThe above-mentioned μ is a factor characterizing the degree of gradient distribution of the original sharp image_dAnd mu_lFor adjusting the adjustment factor for the smoothness of the restored sharp image, the above

Is the differential operator of the image along the horizontal and vertical directions, optionally α, above_lThe separation effect of the background layer and the reflective layer is more obvious by taking 0.8; alternatively, the above μ_dCan take on a value of 1, mu_lThe value of (1 e-1) can be set, so that the smoothness degree of the finally restored clear image can better meet the requirements of users.

It should be noted that, in the prior art, when performing image deblurring, only the situation that an image has a background layer is considered, that is, the prior art performs deblurring based on a priori condition that the image does not have reflection, and for the image with both the background layer and the reflection layer, the priori condition of the prior art is not true, so that the prior art cannot accurately restore both the background layer and the reflection layerAn image of the optical layer; in the embodiment of the invention, the imaging model B is L based on the image_B*K_B+L_R*K_R+ n, it can be seen that the method of the embodiment of the present invention considers the blur kernel of the background layer and the blur kernel of the reflective layer of the image to be processed, and based on the two blur kernels, the separation and deblurring of the background layer and the reflective layer are performed simultaneously by using the above formula 1, which not only improves the accuracy of deblurring the image, but also makes the restored image clearer.

To better illustrate the effect of the embodiment of the present invention in deblurring an image, refer to a comparison graph of restoration effects shown in fig. 3 and fig. 4, where (a) in fig. 3 and (a) in fig. 4 are images to be processed, fig. 3 (b) (c) and fig. 4 (b) (c) are image deblurring effects in the prior art, and fig. 3 (d) and fig. 4 (d) are image deblurring effects by using the method of the embodiment of the present invention.

Of course, the method provided by the embodiment of the invention is also suitable for deblurring of the image without reflection, as long as the K in the image imaging model is enabled_RIt is sufficient if 0.

The image deblurring method provided by the embodiment of the invention obtains the fuzzy kernel of the background layer and the fuzzy kernel of the reflecting layer of the image to be processed, and then obtains the fuzzy kernels according to a formulaDetermining a background layer L of an image to be processed_BAnd a light-reflecting layer L of the image to be processed_RTherefore, the image with the definition meeting the requirements of the user is restored. Because the fuzzy kernel of the background layer and the fuzzy kernel of the reflecting layer of the image to be processed are respectively considered in the embodiment of the invention, and the background layer and the reflecting layer are separated and deblurred simultaneously by adopting the formula 1 based on the two fuzzy kernels, the deblurring accuracy of the image is improved, and the restored image is clearer.

Fig. 5 is a flowchart of a second embodiment of a method for deblurring an image according to an embodiment of the present invention, where this embodiment relates to a specific process of an image processing device acquiring a blur kernel of a background layer and a blur kernel of a reflective layer of an image to be processed, and on the basis of the foregoing embodiment, step , where step S101 may specifically include the following steps:

s201, clustering each image block in the image to be processed to obtain th background layer L_B' and th light-reflecting layer L_R′。

Specifically, after the image processing device acquires the image to be processed and determines that the image to be processed has the background layer and the reflection layer, the image processing device performs clustering on each image block of the image to be processed, that is, the image block belonging to the background layer and the image block belonging to the reflection layer in the image to be processed are classified, and a specific clustering algorithm is not limited in the embodiment of the present invention.

After clustering image blocks of an image to be processed by the image processing equipment, obtaining image blocks of a background layer and image blocks of a reflecting layer, wherein the image blocks of the background layer comprise all image blocks belonging to the background layer in the image to be processed, and the image blocks of the reflecting layer comprise all image blocks belonging to the reflecting layer in the image to be processed_B' and th reflecting layer L is obtained according to the image block type of the reflecting layer_R'. Note that the background layer L_B' in the form of a matrix, each elements in the matrix are the pixel values of each image block in the image block class of the background layer, the th light-reflecting layer L_R' also in the form of a matrix, where each elements in the matrix are the pixel values of each image block in the image block class of the light-reflective layer.

Optionally, the third embodiment shown in fig. 6 provides obtaining background layers L_B' and th light-reflecting layer L_RIn a possible embodiment of the' method, as shown in fig. 6, the method comprises:

s301: according to the formula

And performing Fourier transform on each image block in the image to be processed to obtain a plurality of th image blocks.

Wherein, the P_xIs an image block of the image to be processed, the

Is a fourier transform of a 5-point laplacian kernel.

S302: according to the formula

And determining a clustering distance between any two th image blocks, and clustering the image blocks of the image to be processed according to the clustering distance to obtain a th cluster and a second cluster.

Wherein the th cluster comprises pixel values of at least image blocks belonging to the background layer, the second cluster comprises pixel values of at least image blocks belonging to the background layer, the P_yIs an image block of the image to be processed.

Specifically, when the image processing apparatus is based on the above formula

After each image block in the image to be processed is subjected to Fourier transform to obtain a plurality of th image blocks, the image processing equipment adopts a formula

Optionally, the image processing device may set clustering distance thresholds, and set an original image block (i.e., an image block not subjected to fourier transform) corresponding to the -th image block of which the clustering distance is smaller than the threshold to belong to the -th cluster, and set an original image block (i.e., an image block not subjected to fourier transform) corresponding to the -th image block of which the clustering distance is greater than or equal to the threshold to belong to the second cluster.

S303, obtaining the background layer L according to the th cluster_B′。

S304, obtaining the th light reflecting layer L according to the second polymer_R′。

Specifically, after the image processing apparatus obtains the th cluster and the second cluster, since the th cluster includes at least pixel values of image blocks belonging to the background layer, and the second cluster includes at least pixel values of image blocks belonging to the background layer, the image processing apparatus can obtain the th background layer L according to the th cluster_B' obtaining a th retroreflective layer L according to the second polymerization_R'。

S202: according to the formula

And said th background layer L_B', obtaining a blur kernel K for the background layer_B。

Wherein, 0 is<λ_k≤1。

S203: according to the formula

Specifically, the embodiment of the invention determines the fuzzy kernel K of the background layer_BThe time is determined in consideration of the light-reflecting layer, that is, the image processing apparatus of the embodiment of the present invention is based on the general formula (4):

to determine K_iWherein K is_iBlur kernel K as background layer_BOr the fuzzy core K of the light-reflecting layer_R0 to said<λ_k≤1。

At the solution of K_iThen, the above equation 4 can be used to obtain

Due to the data fidelity term in equation 5

And the regularization term

All are L2 norms, so that closed solving can be carried out by utilizing a Fourier transform algorithm to obtain

Therefore, the th background layer L is obtained based on the image processing apparatus in the above-described S201_B' and th light-reflecting layer L_R', the image processing device may be based on the th background layer L_B' and

obtaining a blur kernel K for a background layer_BAnd the light reflecting layer L can also be according to the th light reflecting layer L_R' sum formula

Obtaining a fuzzy kernel K of a light-reflecting layer_R。

When the image processing device obtains the fuzzy kernel K of the background layer_BAnd a blurring kernel K of the light-reflecting layer_RThen, the image processing device can calculate the clear background layer L of the image to be processed according to the above formula 1_BAnd a light-reflecting layer L of the image to be processed_R。

The method for deblurring the image, provided by the embodiment of the invention, obtains the th background layer L by clustering each image block in the image to be processed_B' and th light-reflecting layer L_R', then according to the formulaAnd th background layer L_B', obtaining a blur kernel K for the background layer_BAnd according to a formula

And th light-reflecting layer L_R', obtaining a fuzzy kernel K of the light-reflecting layer_RAnd further determining the background layer L of the image to be processed according to the formula 1_BAnd a reflective layer of the image to be processed, so that the image with the definition meeting the requirements of the user is restored. Fuzzy kernel K of background layer determined by embodiment of the invention_BAnd a blurring kernel K of the light-reflecting layer_RIn addition, because the fuzzy kernel of the background layer of the image to be processed and the fuzzy kernel of the reflecting layer are respectively considered in the embodiment of the invention, and the background layer and the reflecting layer are simultaneously separated and deblurred by adopting the formula 1 based on the two fuzzy kernels, the deblurring accuracy of the image is improved, and the restored image is clearer.

Fig. 7 is a flowchart illustrating a fourth embodiment of a method for deblurring an image according to an embodiment of the present invention, where the embodiment relates to a specific process in which an image processing apparatus obtains a blur kernel of a background layer and a blur kernel of a light reflection layer of an image to be processed in an iterative manner, so that the two blur kernels have better convergence, and the effects of separation and deblurring of the background layer and the light reflection layer are more obvious, and based on the above embodiment, further , the above S202 and S203 may specifically include the following steps:

step A: according to the formula

And said th light-reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R′。

And B: according to the formula

The th pointInitialized fuzzy kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R”。

Specifically, the above-mentioned 0<λ_lLess than or equal to 1. Based on the above formula (4), it can be known

K here_iNamely, the th initialization fuzzy core K calculated in the step A_B' and a second initialization fuzzy core K_R', L on the left side of the equation_iL to the right of the equation for the unknowns to be solved_iL to the left of the equation for the variable for which the equation is directed_iThe value of (a) is a value that can minimize the right side of the equation, so to distinguish from the parameter identifier of the formula in the above embodiment, the formula 9 may be specifically written as:

where X and Y replace variables on the right side of the equation of equation 9, { L_B”,L_R"} replaces the unknown number L on the left side of the equation of equation 9_i。

Therefore, th initialization blur kernel K when the image processing apparatus acquires the background layer of the image to be processed_B' and second initialization blur kernel of light-reflecting layer K_RAfter that, the image processing apparatus determines the second background layer L according to equation 10_BAnd a second light-reflecting layer L_R”。

And C: applying the second background layer L_B"as a new th background layer L_B', and combining said second light-reflecting layer L_R"as a new th light-reflecting layer L_R' executing the step A until the iteration number reaches a preset number, and taking the th initialized fuzzy kernel obtained in the last iterations as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R。

Specifically, when the image processing apparatus initializes the mode according to Paste kernel K_B', second initial blurring kernel of light-reflecting layer K_R' and the above equation 10 to obtain the second background layer L_BAnd a second light-reflecting layer L_R"thereafter, the second background layer L_B"as a new th background layer L_B', and the second light-reflecting layer L_R"as a new th light-reflecting layer L_R' are respectively substituted into the above-mentioned formula 7 and formula 8 to respectively obtain the new th initialization fuzzy kernel K_B' and a new second initialization fuzzy core K_R' then the new th initialization fuzzy kernel K_B' and a new second initialization fuzzy core K_R' the above formula 10 is substituted again to obtain a new second background layer L_B"and a new second light-reflecting layer L_R", and then the new second background layer L is applied again_B"and a new second light-reflecting layer L_R"as a new th background layer L_B' and a new th light-reflecting layer L_R' sequentially circulating until the iteration times reach the preset times, and taking the -th initialized fuzzy kernel obtained in the last iterations as the fuzzy kernel K of the background layer of the image to be processed_BAnd taking the second initialized fuzzy kernel obtained in the last iterations as the fuzzy kernel K of the light reflecting layer of the image to be processed_R。

It should be noted that α in the above equation 10_lAnd the separation effect of the background layer and the light reflecting layer of the image to be processed is more obvious, and the iterative process is specifically solved by an iterative weighted least squares (IRLS) method.

In the method for deblurring an image, which is provided by the embodiment of the invention, the th initialization blur kernel K of the background layer is obtained through the step A_B' and second initialization blur kernel of light-reflecting layer K_R' and then a second background layer L is determined by the above step B_BAnd a second light-reflecting layer L_R", and then a second background layer L_B"as a new th background layer L_B', and a second light-reflecting layer L_R"as a new th light-reflecting layer L_R' go back to the above step A until the number of iterations reaches the preset number, and go back to the end iterationsObtained th initialized blur kernel as blur kernel K of background layer_BAnd taking the second initialized fuzzy kernel obtained in the last iterations as the fuzzy kernel K of the light reflecting layer_R. The method of the embodiment of the invention determines the fuzzy kernel K of the background layer of the image to be processed through multiple iterations_BAnd a blurring kernel K of the light-reflecting layer_RSo that the two obtained fuzzy kernels are more accurate, and the finally determined background layer L of the image to be processed is greatly improved_BAnd a light-reflecting layer L of the image to be processed_RThe restored image is clearer due to the accuracy of the image restoration method.

Fig. 8 is a schematic structural diagram of an embodiment of an apparatus for image deblurring according to an embodiment of the present invention, where the apparatus for image deblurring can be integrated into the above-mentioned image deblurring device, or can be a stand-alone image deblurring device, and the apparatus for image deblurring can be implemented by software, hardware, or a combination of software and hardware, as shown in fig. 7, the apparatus includes an obtaining module 11 and a determining module 12.

Specifically, the obtaining module 11 is configured to obtain a blur kernel of a background layer and a blur kernel of a reflective layer of the image to be processed;

a determination module 12 for determining the formula

The image deblurring device provided by the embodiment of the invention can execute the method embodiment, has similar realization principle and technical effect, and is not repeated herein.

Fig. 9 is a schematic structural diagram of a second embodiment of the image deblurring apparatus according to the present invention, and based on the embodiment shown in fig. 8, further , the obtaining module 11 may include a clustering unit 111 and an obtaining unit 112.

Specifically, the clustering unit 111 is configured to perform clustering on each image block in the image to be processed to obtain an th background layer L_B' and th light-reflecting layer L_R'；

An obtaining unit 112 for obtaining the formula

, the obtaining unit 112 is specifically configured to obtain the formula

And said th light-reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R'; and according to the formulaThe th initialization blur kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R", and applying said second background layer L_B"as a new th background layer L_B', and willThe second light reflecting layer L_R"as a new th light-reflecting layer L_R', return to determining a new th initialization fuzzy kernel K_B' and a new second initialization fuzzy core K_R', until the iteration number reaches the preset number, and the -th initialized fuzzy kernel obtained in the last iterations is used as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R(ii) a Wherein, 0 is<λ_l≤1。

Optionally, said α_lEqual to 0.8.

Alternatively, the μ _d1 and μ_l＝1e-1。

Fig. 10 is a schematic structural diagram of a third embodiment of the image deblurring apparatus according to the embodiment of the present invention, and based on the embodiment shown in fig. 9, further , the clustering unit 111 may specifically include a processing subunit 201, a clustering subunit 202, and an obtaining subunit 203.

In particular, the processing subunit 201 is configured to perform the following operations according to a formula

A Fourier transform of a 5-point Laplace kernel;

a clustering subunit 202 for calculating a formula

Determining a clustering distance between any two th image blocks, and clustering the image blocks of the image to be processed according to the clustering distanceClass, obtaining th cluster and second cluster, wherein, the th cluster comprises at least pixel values of image blocks subordinating to the background layer, the second cluster comprises at least pixel values of image blocks subordinating to the background layer, the P cluster is a cluster of the image blocks subordinating to the background layer_yThe image blocks of the image to be processed are obtained;

an obtaining subunit 203, configured to obtain the th background layer L according to the th cluster_B' and, obtaining said th retroreflective layer L from said second polymer_R'。

Fig. 11 is a schematic structural diagram of an image deblurring device according to an embodiment of the present invention, as shown in fig. 11, the image deblurring device may include an input device 20, a processor 21, for example, a CPU, a memory 22, and at least communication buses 23, and optionally, may further include a display device 24, the communication buses 23 are used to implement communication connections between elements, the memory 22 may include a high-speed RAM memory, and may also include a non-volatile memory NVM, for example, at least disk memories, and various programs may be stored in the memory 22 for performing various processing functions and implementing the method steps of the embodiment, and the input device 20 is used to input an image to be processed to the processor 21.

Specifically, in this embodiment, the processor 21 is configured to obtain a blur kernel of a background layer and a blur kernel of a reflective layer of the image to be processed input by the input device 20, and according to a formula

, the processor 21 is specifically configured to perform clustering on each image block in the image to be processed to obtain a th background layer L_B' and th light-reflecting layer L_RAccording to formula

And said th background layer L_B', obtaining a blur kernel K for the background layer_B(ii) a And, according to the formulaAnd said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_R(ii) a Wherein, 0 is<λ_k≤1。

Further , the processor 21 is specifically configured to calculate a formulaAnd said th background layer L_B' obtaining th initialization blur kernel K of the background layer_B'; and, according to the formula

And said th light-reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R'; and according to the formulaThe th initialization blur kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R", and applying said second background layer L_B"as a new th background layer L_B', and combining said second light-reflecting layer L_R"as a new th light-reflecting layer L_R', return to determining a new th initialization fuzzy kernel K_B' and a new second initialization fuzzy core K_R', until the iteration number reaches the preset number, and the -th initialized fuzzy kernel obtained in the last iterations is used as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R(ii) a Wherein, 0 is<λ_l≤1。

Optionally, said α_lEqual to 0.8.

Alternatively, the μ _d1 and μ_l＝1e-1。

, the processor 21 is specifically configured to calculate a formula

Fourier transform is carried out on each image block in the image to be processed to obtain a plurality of th image blocks, and the th image blocks are obtained according to a formula

It will be understood by those skilled in the art that all or a portion of the steps of implementing the various method embodiments described above may be performed by hardware associated with program instructions, and that the program may be stored in a computer readable storage medium, which when executed performs the steps comprising the various method embodiments described above, including ROM, RAM, magnetic or optical disks, among various media capable of storing program code.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims

A method of deblurring an image of the type , comprising:

acquiring a fuzzy kernel of a background layer and a fuzzy kernel of a reflecting layer of an image to be processed;

according to the formula
Determining a background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_R(ii) a Wherein, B is a pixel matrix of an image to be processed, and K is_BIs a blur kernel of the background layer, the K_RIs a blur kernel of the light-reflecting layer, the K_iIs K_BOr K_RSaid L is_iIs L_BOr L_RSaid α_lThe factor for characterizing the degree of gradient distribution of the original sharp image, the μ_dAnd mu_lAnd adjusting factors for adjusting the smoothness degree of the restored clear image.
2. The method according to claim 1, wherein the acquiring of the blur kernel of the background layer and the blur kernel of the reflective layer of the image to be processed specifically comprises:

clustering each image block in the image to be processed to obtain th background layer L_B' and th light-reflecting layer L_R'；

According to the formula
And said th background layer L_B', obtaining a blur kernel K for the background layer_B(ii) a Wherein, 0 is<λ_k≤1；

According to the formula
And said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_R。
3. The method of claim 2, wherein the equation is based on
And said th background layer L_B', obtaining a blur kernel K for the background layer_BAnd, according to the formula
And said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_RThe method specifically comprises the following steps:

step A: according to the formulaAnd said th background layer L_B' obtaining th initialization blur kernel K of the background layer_B'; and, according to the formula
And said th light-reflecting layer L_R', obtaining said inverseSecond initialization blur kernel K of the optical layer_R'；

And B: according to the formula
The th initialization blur kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R"; wherein, 0 is<λ_l≤1；

And C: applying the second background layer L_B"as a new th background layer L_B', and combining said second light-reflecting layer L_R"as a new th light-reflecting layer L_R' executing the step A until the iteration number reaches a preset number, and taking the th initialized fuzzy kernel obtained in the last iterations as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R。
4. The method of claim 3, wherein said α_lEqual to 0.8.
5. The method according to claim 2, wherein the clustering process is performed on each image block in the image to be processed to obtain th background layer L_B' and th light-reflecting layer L_R', specifically includes:

according to the formula
Fourier transform is carried out on each image block in the image to be processed to obtain a plurality of th image blocks, wherein P is_xIs an image block of the image to be processed, the
A Fourier transform of a 5-point Laplace kernel;

according to the formula
Determining a clustering distance between any two th image blocks, and clustering the image blocks of the image to be processed according to the clustering distance to obtain a th cluster and a second cluster, wherein the th cluster comprises at least pixel values of the image blocks which belong to a background layer, the second cluster comprises at least pixel values of the image blocks which belong to the background layer, and the P cluster comprises at least pixel values of the image blocks which belong to the background layer_yThe image blocks of the image to be processed are obtained;

obtaining the background layer L according to the th cluster_B'；

Obtaining the th light-reflecting layer L according to the second polymer_R'。
6. The method of claim 1, wherein said μ_d1 and μ_l＝1e-1。
An apparatus for deblurring an image of the type 7, , comprising:

the acquisition module is used for acquiring a fuzzy kernel of a background layer and a fuzzy kernel of a reflecting layer of the image to be processed;

a determination module for determining the formula
Determining a background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_R(ii) a Wherein, B is a pixel matrix of an image to be processed, and K is_BIs a blur kernel of the background layer, the K_RIs a blur kernel of the light-reflecting layer, the K_iIs K_BOr K_RSaid L is_iIs L_BOr L_RSaid α_lThe factor for characterizing the degree of gradient distribution of the original sharp image, the μ_dAnd mu_lAnd adjusting factors for adjusting the smoothness degree of the restored clear image.
8. The apparatus of claim 7, wherein the obtaining module comprises:

a clustering unit for clustering each image block in the image to be processed to obtain th background layer L_B' and th light-reflecting layer L_R'；

An acquisition unit for acquiring the formula
And said th background layer L_B', obtaining a blur kernel K for the background layer_B(ii) a And, according to the formula
And said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_R(ii) a Wherein, 0 is<λ_k≤1。
9. Device according to claim 8, wherein the obtaining unit is specifically configured to obtain the data according to a formula
And said th background layer L_B' obtaining th initialization blur kernel K of the background layer_B'; and, according to the formulaAnd said th light-reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R'; and according to the formula
The th initialization blur kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R", and applying said second background layer L_B"as a new th background layer L_B', and inverting said secondOptical layer L_R"as a new th light-reflecting layer L_R', return to determining a new th initialization fuzzy kernel K_B' and a new second initialization fuzzy core K_R', until the iteration number reaches the preset number, and the -th initialized fuzzy kernel obtained in the last iterations is used as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R(ii) a Wherein, 0 is<λ_l≤1。
10. The apparatus of claim 9, wherein said α_lEqual to 0.8.
11. The apparatus of claim 8, wherein the clustering unit comprises:

a processing subunit for processing according to a formula
Fourier transform is carried out on each image block in the image to be processed to obtain a plurality of th image blocks, wherein P is_xIs an image block of the image to be processed, the
A Fourier transform of a 5-point Laplace kernel;

a clustering subunit for clustering according to a formula
Determining a clustering distance between any two th image blocks, and clustering the image blocks of the image to be processed according to the clustering distance to obtain a th cluster and a second cluster, wherein the th cluster comprises at least pixel values of the image blocks which belong to a background layer, the second cluster comprises at least pixel values of the image blocks which belong to the background layer, and the P cluster comprises at least pixel values of the image blocks which belong to the background layer_yThe image blocks of the image to be processed are obtained;

an obtaining subunit, configured to obtain the th background layer L according to the th cluster_B' and, obtaining said th retroreflective layer L from said second polymer_R'。
12. The apparatus of claim 7, wherein μ_d1 and μ_l＝1e-1。
An image deblurring apparatus of the kind , comprising an input device and a processor;

the processor is used for acquiring the fuzzy kernel of the background layer and the fuzzy kernel of the reflecting layer of the image to be processed input by the input equipment and according to a formula
Determining a background layer L of the image to be processed_BAnd a light-reflecting layer L of the image to be processed_R(ii) a Wherein, B is a pixel matrix of an image to be processed, and K is_BIs a blur kernel of the background layer, the K_RIs a blur kernel of the light-reflecting layer, the K_iIs K_BOr K_RSaid L is_iIs L_BOr L_RSaid α_lThe factor for characterizing the degree of gradient distribution of the original sharp image, the μ_dAnd mu_lAnd adjusting factors for adjusting the smoothness degree of the restored clear image.
14. The apparatus according to claim 13, wherein the processor is specifically configured to perform clustering on each image block in the image to be processed to obtain an th background layer L_B' and th light-reflecting layer L_RAccording to formula
And said th background layer L_B', obtaining a blur kernel K for the background layer_B(ii) a And, according to the formula
And said th light-reflecting layer L_R', obtaining a blur kernel K of said light-reflecting layer_R(ii) a Wherein, 0 is<λ_k≤1。
15. Device according to claim 14, wherein the processor is specifically configured to operate according to a formula
And said th background layer L_B' obtaining th initialization blur kernel K of the background layer_B'; and, according to the formulaAnd said th light-reflecting layer L_R' obtaining a second initialized blur kernel K of the light reflecting layer_R'; and according to the formula
The th initialization blur kernel K_B' and the second initialization fuzzy core K_R', determining a second background layer L_BAnd a second light-reflecting layer L_R", and applying said second background layer L_B"as a new th background layer L_B', and combining said second light-reflecting layer L_R"as a new th light-reflecting layer L_R', return to determining a new th initialization fuzzy kernel K_B' and a new second initialization fuzzy core K_R', until the iteration number reaches the preset number, and the -th initialized fuzzy kernel obtained in the last iterations is used as the fuzzy kernel K of the background layer_BAnd taking a second initialized fuzzy kernel obtained in the last iterations as a fuzzy kernel K of the light reflecting layer_R(ii) a Wherein, 0 is<λ_l≤1。
16. The apparatus of claim 15, wherein said α_lEqual to 0.8.
17. Device according to claim 16, wherein the processor is specifically configured to operate according to a formula
Fourier transform is carried out on each image block in the image to be processed to obtain a plurality of th image blocks, and the th image blocks are obtained according to a formula
Determining the clustering distance between any two th image blocks, clustering the image blocks of the image to be processed according to the clustering distance to obtain a th cluster and a second cluster, and obtaining the th background layer L according to the th cluster_B', and obtaining said th retroreflective layer L from said second polymer_R'; wherein, the P_xIs an image block of the image to be processed, theA Fourier transform of a 5-point Laplacian kernel, said th cluster comprising pixel values of at least image blocks belonging to the background layer, said second cluster comprising pixel values of at least image blocks belonging to the background layer, said P_yIs an image block of the image to be processed.
18. The apparatus of claim 13, wherein μ_d1 and μ_l＝1e-1。