CN114078096A - Image deblurring method, device and equipment - Google Patents
Image deblurring method, device and equipment Download PDFInfo
- Publication number
- CN114078096A CN114078096A CN202111322563.9A CN202111322563A CN114078096A CN 114078096 A CN114078096 A CN 114078096A CN 202111322563 A CN202111322563 A CN 202111322563A CN 114078096 A CN114078096 A CN 114078096A
- Authority
- CN
- China
- Prior art keywords
- image
- network model
- deep learning
- learning network
- blurred image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013135 deep learning Methods 0.000 claims abstract description 55
- 238000004364 calculation method Methods 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000012935 Averaging Methods 0.000 claims abstract description 7
- 238000000605 extraction Methods 0.000 claims description 29
- 238000010586 diagram Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims 1
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 238000003672 processing method Methods 0.000 abstract 1
- 238000004422 calculation algorithm Methods 0.000 description 13
- 238000013527 convolutional neural network Methods 0.000 description 9
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000017105 transposition Effects 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration by non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention discloses an image processing method, an image processing device, an image processing medium and image processing equipment, which relate to the field of artificial intelligence and are used for removing a moving target in an image, and the method specifically comprises the following steps: extracting original image information of three channels from the blurred image; respectively carrying out Laplace transform and mean square error removal calculation on the original image information of the three channels to obtain intermediate image information respectively corresponding to the three channels; averaging the intermediate image information respectively corresponding to the three channels to obtain the fuzziness of the blurred image; and selecting a corresponding deep learning network model to perform deblurring operation on the blurred image according to the blurring degree of the blurred image.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image deblurring method, apparatus, and device.
Background
In recent years, the current smart camera with two lenses or even multiple lenses is widely applied, such as a smart phone, an unmanned vehicle, an unmanned aerial vehicle, etc., however, the problem of out-of-focus blur occurs inevitably in many scenes, such as image blur caused by shaking, focusing inaccuracy or rapid movement of an object of the device in the shooting process. This will not only affect the quality of the picture taken, but also have a severe impact on the subsequent processing of the picture. The traditional deblurring algorithms are many and can be divided into a blind deconvolution algorithm and a non-blind deconvolution algorithm according to whether a prior model exists or not. The former has a certain priori assumption on a convolution kernel or an image, and the latter assumes that a fuzzy kernel function is known, both of the two traditional algorithms have the problem of very difficult solution, and a satisfactory effect is often difficult to obtain under the conditions of unknown fuzzy type and deeper fuzzy degree.
The current popular image deblurring algorithm usually needs a large convolutional neural network, and although the convolutional neural network can process fuzzy images of various distortion types or different degrees, the network model with higher complexity is selected to bring larger calculation overhead. In addition, because all the algorithms use the same neural network structure to process all the images, the partial images are still difficult to achieve satisfactory results after being processed.
Therefore, how to select an algorithm with smaller computational cost to perform the deblurring operation of the image becomes an urgent problem to be solved in the industry.
Disclosure of Invention
The embodiment of the invention provides an image deblurring method, an image deblurring device and electronic equipment, which are used for selecting different deep learning network models to perform deblurring operation according to the image blurring degree, so that the calculation overhead is reduced, and meanwhile, images with better quality are obtained.
In a first aspect, the present invention provides an image deblurring method, including the steps of:
extracting original image information of three channels from the blurred image; respectively carrying out Laplace transform and mean square error removal calculation on the original image information of the three channels to obtain intermediate image information respectively corresponding to the three channels; averaging the intermediate image information respectively corresponding to the three channels to obtain the fuzziness of the blurred image; and selecting a corresponding deep learning network model to perform deblurring operation on the blurred image according to the blurring degree of the blurred image.
The image deblurring method provided by the invention has the beneficial effects that: on one hand, the method adopts a smaller network model to estimate the fuzziness of the blurred image, avoids image preprocessing, greatly reduces the calculation cost and simultaneously reserves the original information of the image as much as possible; on the other hand, different methods for deblurring the image are selected according to different blurriness degrees, so that some images with lower blurriness degrees can be deblurred by using smaller depth learning models compared with images with higher blurriness degrees, and the calculation cost is further reduced.
In a possible embodiment, selecting a corresponding deep learning network model to perform a deblurring operation on the blurred image according to the blur degree of the blurred image includes:
when the fuzziness of the blurred image is greater than or equal to a set threshold value, performing deblurring operation on the blurred image by adopting a gradual up-sampling deep learning network model; and when the fuzziness of the blurred image is lower than the set threshold, performing deblurring operation on the blurred image by adopting a double-layer parallel deep learning network model.
In the method, because the operation of deblurring the image with higher blurring degree consumes more network resources, the deblurring operation is carried out by adopting a double-layer parallel deep learning network model, and more color and image detail information of the image can be kept as far as possible under the condition of meeting the basic requirement of deblurring.
In a possible embodiment, the gradual up-sampling deep learning network model comprises a first feature extraction branch, a second feature extraction branch, a full connection module and an image reconstruction branch, wherein each feature extraction branch is composed of different network models, and the first feature extraction branch is used for extracting a first feature map from the blurred image; the second feature extraction branch is used for extracting a second feature map from the blurred image;
the full-connection module is used for fully connecting the first characteristic diagram and the second characteristic diagram to obtain a third characteristic diagram;
the image reconstruction branch is used for adding the blurred image and the first feature map to obtain a first intermediate image of the next level; and adding the intermediate image and the second characteristic image to obtain a second intermediate image of the next stage, and finally adding a third characteristic image obtained after full connection and the second intermediate image to obtain a clear image.
In one possible embodiment, the two-layer parallel deep learning network model comprises a first layer deep learning network model and a second layer deep learning network model;
the first layer of deep learning network model is used for reserving color information and information of an image to obtain a first characteristic diagram;
the first layer of deep learning network model is used for deblurring operation to obtain a second feature map;
wherein the superposition result of the first feature map and the second feature map is equal to a clear image.
In a second aspect, embodiments of the present invention further provide an image deblurring apparatus, which includes a module/unit for performing the method of any one of the possible embodiments of the first aspect. These modules/units may be implemented by hardware, or by hardware executing corresponding software.
In a third aspect, an embodiment of the present invention provides an electronic device, which includes a processor and a memory. Wherein the memory is used to store one or more computer programs; the memory stores one or more computer programs that, when executed by the processor, enable the server to implement the method of any one of the possible embodiments of the first aspect described above.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which includes a computer program and when the computer program is run on an electronic device, the electronic device is caused to perform the method of any one of the possible embodiments of any one of the above aspects.
In a fifth aspect, the present invention further provides a computer program product, which when run on a terminal, causes the electronic device to perform the method of any one of the possible embodiments of any one of the above aspects.
As for the advantageous effects of the above second to fifth aspects, reference may be made to the description in the above first and second aspects.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present invention;
FIG. 2 is a schematic flow chart of an image deblurring method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of another image deblurring method according to an embodiment of the present invention;
fig. 4A is a schematic flowchart of a method for deblurring an image based on a gradual upsampling deep learning network model according to an embodiment of the present invention;
FIG. 4B is a schematic diagram of a gradual upsampling deep learning network model according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart of a method for deblurring an image by using a two-layer parallel deep learning network model according to an embodiment of the present invention;
FIG. 6 is a schematic flow chart of another image deblurring method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to overcome the defects of the existing image deblurring, the invention provides an image deblurring method which can select different deep learning network models to perform deblurring operation based on the image blurring degree, and obtain images with better quality while reducing the calculation cost.
Some terms used in the embodiments of the present invention are explained below to facilitate understanding by those skilled in the art.
1. Convolutional neural network
Convolutional neural networks are a class of feed forward neural networks (feedforward neural networks) that contain convolutional calculations and have deep structures, and are one of the algorithms that represent deep learning (deep learning). The convolutional neural network has the characteristic learning ability and can carry out translation invariant classification on input information according to the hierarchical structure of the convolutional neural network. The convolutional neural network is a kind of neural network, and is inspired by the research of biological neuroscience, and the convolutional neural network is originally proposed to process data with a network-like structure, for example, an image can be regarded as a two-dimensional network consisting of pixels. The general network structure of the convolutional neural network comprises a data input layer, a convolutional layer, a data excitation layer, a pooling layer, a full connection layer and a data output layer.
The embodiment of the invention relates to Artificial Intelligence (AI) and machine learning technologies, which are designed based on a deep learning network (ML) in the AI.
With the research and progress of artificial intelligence technology, artificial intelligence is researched and applied in a plurality of fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, robots, smart medical treatment and the like.
Machine learning is a multi-field cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and the like.
In describing embodiments of the present invention, the terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the description of the invention and the appended claims, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of the present invention, "at least one", "one or more" means one or more than two (including two). The term "and/or" is used to describe an association relationship that associates objects, meaning that three relationships may exist; for example, a and/or B, may represent: a alone, both A and B, and B alone, where A, B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise. The term "coupled" includes both direct and indirect connections, unless otherwise noted. "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated.
In the embodiments of the present invention, words such as "exemplary" or "for example" are used to mean serving as examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "e.g.," an embodiment of the present invention is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.
The image deblurring method provided in the embodiment of the present invention may be applied to an application scenario as shown in fig. 1, where the application scenario includes a server 100 and a terminal device 200.
In one possible design, the server 100 is configured to obtain a blurred image from the terminal device 200, calculate a blur degree of the blurred image, and select a corresponding deep learning network model based on the blur degree to perform a deblurring operation.
In another possible design, after the terminal device 200 generates the blurred image, the blur degree of the blurred image may be calculated, and then the corresponding deep learning network model is selected according to the blur degree to perform the deblurring operation, or the corresponding deep learning network model in the server is used to perform the deblurring operation.
The server 100 and the terminal device 200 may be connected via a wireless network, and the terminal device 200 is a terminal device with network communication capability, which may be a smart phone, a tablet computer, a portable personal computer, or the like. The server 100 may be a server, or a server cluster or a cloud computing center composed of several servers.
Based on the application scenario diagram shown in fig. 1, the embodiment of the present invention provides a flow of an image deblurring method, as shown in fig. 2, the flow of the method may be executed by the server 100 or the terminal device 200, and the server 100 is taken as an example for description below. The method comprises the following steps:
s301, original image information of three channels is extracted from the blurred image.
S302, performing Laplace transform and mean square error removal calculation on the original image information of the three channels respectively to obtain intermediate image information corresponding to the three channels respectively.
And S303, averaging the intermediate image information respectively corresponding to the three channels to obtain the fuzziness of the blurred image.
S304, according to the fuzziness of the blurred image, selecting a corresponding deep learning network model to perform deblurring operation on the blurred image.
With reference to fig. 3, after the server 100 acquires the blurred image from the terminal device 200, first extracting original image information of the R channel, original image information of the G channel, and original image information of the B channel, and then performing laplace transform and mean square error removal calculation on the original image information of the R channel, respectively, to obtain intermediate image information of the R channel; performing Laplace transform and mean square error removal calculation on original image information of the G channel to obtain intermediate image information of the G channel; and carrying out Laplace transform and mean square error removal calculation on the original image information of the B channel to obtain the intermediate image information of the B channel. And averaging the intermediate image information respectively corresponding to the three channels to obtain the fuzziness of the blurred image. If the fuzziness of the blurred image is greater than or equal to a set threshold value, performing deblurring operation on the blurred image by adopting a gradual up-sampling deep learning network model; and if the fuzziness of the blurred image is lower than a set threshold value, performing deblurring operation on the blurred image by adopting a double-layer parallel deep learning network model.
It should be noted that, in the above method, the smaller network model is used to estimate the blurring degree of the image, so that the image preprocessing can be directly skipped, and the original information of the image can be retained as much as possible while the calculation overhead is greatly reduced. Compared with the traditional deblurring algorithm, the traditional deblurring algorithm needs to preprocess an image in the first step, and needs a user to select an image block in order to reduce the calculated amount; secondly, estimating a fuzzy kernel function by using a Bayesian algorithm, so that the local optimization is easy to fall into; and thirdly, reconstructing a clear image by using a standard deblurring algorithm. Thus, the whole calculation is complex, and a satisfactory result is often difficult to obtain under the condition that the fuzzy type is unknown and the fuzzy degree is deep.
In one possible embodiment, as shown in fig. 4, the gradual upsampling deep learning network model provided by the present embodiment includes: a first feature extraction branch 401, a second feature extraction branch 402, a full connectivity module 403 and an image reconstruction branch 404, each feature extraction branch consisting of a different network model. The first feature extraction branch 401 is configured to extract a first feature map from the blurred image; the second feature extraction branch 402 is configured to extract a second feature map from the blurred image; a full connection module 403, configured to perform full connection on the first feature map and the second feature map to obtain a third feature map;
the image reconstruction branch 404 is configured to add the blurred image and the first feature map to obtain a first intermediate image of a next stage; and adding the intermediate image and the second characteristic image to obtain a second intermediate image of the next stage, and finally adding a third characteristic image obtained after full connection and the second intermediate image to obtain a clear image.
The first feature extraction branch 401 uses a residual error structure in a ResNet network for reference to obtain image features, and is composed of a plurality of different convolutional layers, each convolutional layer obtains nonlinear feature mapping, and finally, transpose convolution operation is performed on each layer to obtain the improved image features. 4041 in fig. 4A shows the image feature model obtained from different convolution layers, and the feature model is subjected to a transposition convolution operation to obtain image features, where the transposition convolution is used to better extract the image features. The second feature extraction branch 402 mirrors the VGGNet network block construction and is mainly used for extracting details of the image.
In addition, fig. 4B specifically shows a feature extraction schematic block diagram of the first feature extraction branch 401 and the second feature extraction branch 402, and the extraction process uses an upsampling step, which mainly includes: convolution, feature embedding, feature upsampling, and parameter sharing operations.
It is noted that the number of convolutional layers may be one or more. In this embodiment, the convolutional layer may include a plurality of convolution operators, which are also called kernels, and act as a filter for extracting specific information from the input image matrix in the image processing, and the convolution operator may be essentially a weight matrix, which is usually predefined, and during the convolution operation on the image, the weight matrix is usually processed on the input image pixel by pixel (or two pixels by two pixels) along the horizontal direction, so as to complete the task of extracting the specific image feature from the image.
In a possible embodiment, as shown in fig. 5, for a blurred image with a larger degree of blur, the two-layer parallel deep learning network model designed by the present invention includes a first-layer deep learning network model and a second-layer deep learning network model; the first layer of deep learning network model is used for reserving color information and information of an image to obtain a first characteristic diagram; the first layer of deep learning network model is used for deblurring operation to obtain a second feature map; wherein the superposition result of the first feature map and the second feature map is equal to a clear image.
As shown in fig. 5, the first deep learning network model extracts the three color channels, performs convolution operations respectively, then fuses the features of the three color channels through localization (connection), and finally obtains a feature map of color information through simple feature extraction and fusion operations. The second layer deep learning network model adopts a common pyramid-like structure, and aims to retain useful information, remove fuzzy information and finally obtain a feature map. And adding the information of the first layer and the second layer to obtain a final clear image.
Based on the image deblurring model, in some embodiments of the present invention, an embodiment of the present invention discloses an image deblurring apparatus, as shown in fig. 6, which is configured to implement the method described in the above method embodiments, and includes: an extraction unit 601, a calculation unit 602, and a processing unit 603. Wherein:
an extracting unit 601, configured to extract original image information of three channels from the blurred image;
a calculating unit 602, configured to perform laplacian transform and mean square error removal calculation on the original image information of the three channels, respectively, to obtain intermediate image information corresponding to the three channels, respectively; averaging the intermediate image information respectively corresponding to the three channels to obtain the fuzziness of the blurred image;
and the processing unit 603 is configured to select a corresponding deep learning network model according to the blur degree of the blurred image, and perform a deblurring operation on the blurred image.
All relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional module, and are not described herein again.
The image deblurring device may be a chip or a chip module. Each module/unit included in each apparatus and product described in the above embodiments may be a software module/unit, or may also be a hardware module/unit, or may also be a part of a software module/unit and a part of a hardware module/unit.
In other embodiments of the present invention, an embodiment of the present invention discloses an electronic device, as shown in fig. 7, the electronic device may include: one or more processors 701; a memory 702; a display 703; one or more application programs (not shown); and one or more computer programs 704, which may be connected via one or more communication buses 705. Wherein the one or more computer programs 704 are stored in the memory 702 and configured to be executed by the one or more processors 701, the one or more computer programs 704 comprising instructions.
The invention also provides a computer-readable medium, on which a computer program is stored, which, when executed by a computer, implements the method of the above-described method embodiments. Specific beneficial effects can be seen in the method embodiments.
The invention also provides a computer program product which, when executed by a computer, implements the method of the above method embodiments. Specific beneficial effects can be seen in the method embodiments.
Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions. For the specific working processes of the system, the apparatus and the unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described here again.
Each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be implemented in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: flash memory, removable hard drive, read only memory, random access memory, magnetic or optical disk, and the like.
The above description is only a specific implementation of the embodiments of the present invention, but the scope of the embodiments of the present invention is not limited thereto, and any changes or substitutions within the technical scope disclosed by the embodiments of the present invention should be covered within the scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method of deblurring an image, the method comprising:
extracting original image information of three channels from the blurred image;
respectively carrying out Laplace transform and mean square error removal calculation on the original image information of the three channels to obtain intermediate image information respectively corresponding to the three channels;
averaging the intermediate image information respectively corresponding to the three channels to obtain the fuzziness of the blurred image;
and selecting a corresponding deep learning network model to perform deblurring operation on the blurred image according to the blurring degree of the blurred image.
2. The method according to claim 1, wherein, according to the degree of blur of the blurred image, selecting a corresponding deep learning network model to perform deblurring operation on the blurred image comprises:
when the fuzziness of the blurred image is greater than or equal to a set threshold value, performing deblurring operation on the blurred image by adopting a gradual up-sampling deep learning network model;
and when the fuzziness of the blurred image is lower than the set threshold, performing deblurring operation on the blurred image by adopting a double-layer parallel deep learning network model.
3. The method of claim 2, wherein the gradual upsampling deep learning network model comprises a first feature extraction branch, a second feature extraction branch, a full connectivity module, and an image reconstruction branch, each feature extraction branch consisting of a different network model;
the first feature extraction branch is used for extracting a first feature map from the blurred image;
the second feature extraction branch is used for extracting a second feature map from the blurred image;
the full-connection module is used for fully connecting the first characteristic diagram and the second characteristic diagram to obtain a third characteristic diagram;
the image reconstruction branch is used for adding the blurred image and the first feature map to obtain a first intermediate image of the next level; and adding the intermediate image and the second characteristic image to obtain a second intermediate image of the next stage, and finally adding a third characteristic image obtained after full connection and the second intermediate image to obtain a clear image.
4. The method of claim 2, wherein the two-tier parallel deep learning network model comprises a first tier deep learning network model and a second tier deep learning network model;
the first layer of deep learning network model is used for reserving color information and information of an image to obtain a first characteristic diagram;
the first layer of deep learning network model is used for deblurring operation to obtain a second feature map;
wherein the superposition result of the first feature map and the second feature map is equal to a clear image.
5. An image deblurring apparatus, comprising:
the extraction unit is used for extracting original image information of three channels from the blurred image;
the calculation unit is used for respectively carrying out Laplace transformation and mean square error removal calculation on the original image information of the three channels to obtain intermediate image information respectively corresponding to the three channels; averaging the intermediate image information respectively corresponding to the three channels to obtain the fuzziness of the blurred image;
and the processing unit is used for selecting a corresponding deep learning network model to carry out deblurring operation on the blurred image according to the blurring degree of the blurred image.
6. The apparatus according to claim 5, wherein the processing unit is specifically configured to:
when the fuzziness of the blurred image is greater than or equal to a set threshold value, performing deblurring operation on the blurred image by adopting a gradual up-sampling deep learning network model;
and when the fuzziness of the blurred image is lower than the set threshold, performing deblurring operation on the blurred image by adopting a double-layer parallel deep learning network model.
7. The apparatus of claim 6, wherein the gradual upsampling deep learning network model comprises a first feature extraction branch, a second feature extraction branch, a full connectivity module, and an image reconstruction branch, each feature extraction branch consisting of a different network model,
the first feature extraction branch is used for extracting a first feature map from the blurred image;
the second feature extraction branch is used for extracting a second feature map from the blurred image;
the full-connection module is used for fully connecting the first characteristic diagram and the second characteristic diagram to obtain a third characteristic diagram;
the image reconstruction branch is used for adding the blurred image and the first feature map to obtain a first intermediate image of the next level; and adding the intermediate image and the second characteristic image to obtain a second intermediate image of the next stage, and finally adding a third characteristic image obtained after full connection and the second intermediate image to obtain a clear image.
8. The apparatus of claim 6, wherein the two-tier parallel deep learning network model comprises a first tier deep learning network model and a second tier deep learning network model;
the first layer of deep learning network model is used for reserving color information and information of an image to obtain a first characteristic diagram;
the first layer of deep learning network model is used for deblurring operation to obtain a second feature map;
wherein the superposition result of the first feature map and the second feature map is equal to a clear image.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 4.
10. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program operable on the processor, the computer program, when executed by the processor, causing the electronic device to carry out the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111322563.9A CN114078096A (en) | 2021-11-09 | 2021-11-09 | Image deblurring method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111322563.9A CN114078096A (en) | 2021-11-09 | 2021-11-09 | Image deblurring method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114078096A true CN114078096A (en) | 2022-02-22 |
Family
ID=80284100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111322563.9A Pending CN114078096A (en) | 2021-11-09 | 2021-11-09 | Image deblurring method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114078096A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116205822A (en) * | 2023-04-27 | 2023-06-02 | 荣耀终端有限公司 | Image processing method, electronic device and computer readable storage medium |
-
2021
- 2021-11-09 CN CN202111322563.9A patent/CN114078096A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116205822A (en) * | 2023-04-27 | 2023-06-02 | 荣耀终端有限公司 | Image processing method, electronic device and computer readable storage medium |
| CN116205822B (en) * | 2023-04-27 | 2023-10-03 | 荣耀终端有限公司 | Image processing method, electronic device and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210350168A1 (en) | Image segmentation method and image processing apparatus | |
| CN112233038B (en) | True image denoising method based on multi-scale fusion and edge enhancement | |
| Lv et al. | Attention guided low-light image enhancement with a large scale low-light simulation dataset | |
| Pan et al. | Physics-based generative adversarial models for image restoration and beyond | |
| Li et al. | Learning a discriminative prior for blind image deblurring | |
| Tsai et al. | BANet: A blur-aware attention network for dynamic scene deblurring | |
| Lu et al. | Deep texture and structure aware filtering network for image smoothing | |
| CN112446380A (en) | Image processing method and device | |
| KR102311796B1 (en) | Method and Apparatus for Deblurring of Human Motion using Localized Body Prior | |
| CN112581379A (en) | Image enhancement method and device | |
| Agrawal et al. | Distortion-free image dehazing by superpixels and ensemble neural network | |
| CN113673545A (en) | Optical flow estimation method, related device, equipment and computer readable storage medium | |
| CN114627034A (en) | Image enhancement method, training method of image enhancement model and related equipment | |
| Syed et al. | Addressing image and Poisson noise deconvolution problem using deep learning approaches | |
| CN114170290A (en) | Image processing method and related equipment | |
| Jam et al. | Symmetric skip connection Wasserstein GAN for high-resolution facial image inpainting | |
| CN114078096A (en) | Image deblurring method, device and equipment | |
| Zhao et al. | Saliency map-aided generative adversarial network for raw to rgb mapping | |
| CN113284055A (en) | Image processing method and device | |
| CN116703768A (en) | Training method, device, medium and equipment for blind spot denoising network model | |
| CN112509144A (en) | Face image processing method and device, electronic equipment and storage medium | |
| Zheng et al. | Joint residual pyramid for joint image super-resolution | |
| CN116309158A (en) | Training method, three-dimensional reconstruction method, device, equipment and medium of network model | |
| CN116977683A (en) | Object recognition method, apparatus, computer device, storage medium, and program product | |
| Zheng et al. | Memory-efficient multi-scale residual dense network for single image rain removal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |