CN113269695A - Image deblurring method, system, device and storage medium - Google Patents
Image deblurring method, system, device and storage medium Download PDFInfo
- Publication number
- CN113269695A CN113269695A CN202110699448.7A CN202110699448A CN113269695A CN 113269695 A CN113269695 A CN 113269695A CN 202110699448 A CN202110699448 A CN 202110699448A CN 113269695 A CN113269695 A CN 113269695A
- Authority
- CN
- China
- Prior art keywords
- image
- standard
- native
- standard image
- sample set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000012549 training Methods 0.000 claims abstract description 81
- 238000012545 processing Methods 0.000 claims abstract description 52
- 230000005477 standard model Effects 0.000 claims abstract description 23
- 230000015654 memory Effects 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 238000005468 ion implantation Methods 0.000 claims description 2
- 230000009467 reduction Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 16
- 238000003384 imaging method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000002146 bilateral effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005316 response function Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image deblurring method, a system, a device and a storage medium, wherein the method comprises the following steps: acquiring a standard image sample set, wherein the standard image sample set at least comprises a clear standard image sample; converting each standard image sample in the standard image sample set into a corresponding native image sample, and performing fuzzy processing on each native image sample to generate a fuzzy native image; restoring the blurred native image into a blurred standard image, and constructing a standard training sample set according to the blurred standard image and the clear standard image sample; and the standard training sample set is used for training the standard model so as to perform deblurring processing on the standard image to be processed through the trained standard model. The technical scheme provided by the invention can improve the precision and efficiency of deblurring processing.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image deblurring method, system, device and storage medium.
Background
In the current image shooting scene, if the ambient illumination is dark or the object moves too fast, the obvious motion blur in the shot image can be caused.
With the continuous development of Convolutional Neural Networks (CNN) and generation of countermeasure networks (GAN), learning-based image deblurring methods are emerging. In the prior art, an image pair consisting of a clear image and a blurred image can be acquired as training data, in addition, the problem of misalignment between the clear image and the blurred image caused by camera shake and the like is relieved by using a series of alignment algorithms, and a neural network model can be trained based on the image pair consisting of the clear image and the blurred image, so that the trained neural network model is used for deblurring the image.
However, the training data obtained by the method in the prior art is relatively limited, and a large amount of manpower and material resources are needed to collect the data, so that the overall efficiency and accuracy of the deblurring processing are low.
Disclosure of Invention
In view of this, embodiments of the present invention provide an image deblurring method, system, apparatus, and storage medium, which can improve the precision and efficiency of deblurring processing.
One aspect of the present invention provides an image deblurring method, including: acquiring a standard image sample set, wherein the standard image sample set at least comprises a clear standard image sample; converting each standard image sample in the standard image sample set into a corresponding native image sample, and performing fuzzy processing on each native image sample to generate a fuzzy native image; restoring the blurred native image into a blurred standard image, and constructing a standard training sample set according to the blurred standard image and the clear standard image sample; and the standard training sample set is used for training the standard model so as to perform deblurring processing on the standard image to be processed through the trained standard model.
In another aspect, the present invention provides an image deblurring system, which includes: the system comprises a sample set acquisition unit, a processing unit and a processing unit, wherein the sample set acquisition unit is used for acquiring a standard image sample set, and the standard image sample set at least comprises a clear standard image sample; the blurred native image generation unit is used for converting each standard image sample in the standard image sample set into a corresponding native image sample and performing blurring processing on each native image sample to generate a blurred native image; the standard training sample set constructing unit is used for restoring the fuzzy original image into a fuzzy standard image and constructing a standard training sample set according to the fuzzy standard image and the clear standard image sample; and the standard training sample set is used for training the standard model so as to perform deblurring processing on the standard image to be processed through the trained standard model.
In another aspect, the present invention further provides an image deblurring method, including: acquiring a standard image sample set, wherein the standard image sample set at least comprises a clear standard image sample; converting each standard image sample in the standard image sample set into a corresponding native image sample, and performing fuzzy processing on each native image sample to generate a fuzzy native image; constructing a primary training sample set according to a clear primary image sample obtained by converting the fuzzy primary image and the clear standard image sample; the native training sample set is used for training the native model so as to deblur the native image to be processed through the trained native model.
In another aspect, the present invention provides an image deblurring system, which includes: the system comprises a sample set acquisition unit, a processing unit and a processing unit, wherein the sample set acquisition unit is used for acquiring a standard image sample set, and the standard image sample set at least comprises a clear standard image sample; the blurred native image generation unit is used for converting each standard image sample in the standard image sample set into a corresponding native image sample and performing blurring processing on each native image sample to generate a blurred native image; the fuzzy training sample set construction unit is used for constructing a native training sample set according to a clear native image sample obtained by converting the fuzzy native image and the clear standard image sample; the native training sample set is used for training the native model so as to deblur the native image to be processed through the trained native model.
In another aspect, the present invention further provides an image deblurring apparatus, which includes a processor and a memory, where the memory is used for storing a computer program, and the computer program, when executed by the processor, implements the image deblurring apparatus method described above.
In another aspect, the present invention further provides a computer-readable storage medium for storing a computer program, which when executed by a processor, implements the image deblurring method described above.
According to the technical scheme, after the standard image sample set is obtained, the standard image sample can be converted into the native image sample. The format of the native image sample may be the format of the image captured by the camera during the exposure process. The original image sample is subjected to fuzzy processing, and the obtained fuzzy original image is restored into the fuzzy standard image, and the fuzzy standard image can better accord with the actual image generation mode of the camera, so that the training sample set constructed on the basis of the fuzzy standard image and the clear standard image sample in the standard image sample set can have better training effect.
Therefore, in the technical scheme provided by the application, the fuzzy standard image is not required to be acquired, but can be processed based on the standard image sample set to obtain the fuzzy standard image, so that the acquisition difficulty of the image sample is reduced, the training precision of the model can be improved, and the precision and the efficiency of deblurring processing are improved.
Drawings
The features and advantages of the present invention will be more clearly understood by reference to the accompanying drawings, which are illustrative and not to be construed as limiting the invention in any way, and in which:
FIG. 1 is a schematic diagram illustrating the steps of an image deblurring method in one embodiment of the present invention;
FIG. 2 illustrates a schematic diagram of image transformation in one embodiment of the present invention;
FIG. 3 is a flow diagram illustrating a method for deblurring an image in accordance with an embodiment of the present invention;
FIG. 4 shows a functional block diagram of an image deblurring system in one embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating the steps of an image deblurring method in another embodiment of the present invention;
FIG. 6 shows a functional block diagram of an image deblurring system in another embodiment of the present invention;
fig. 7 shows a schematic configuration diagram of an image deblurring apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings of the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of the present invention.
When an image is taken by a conventional camera, a plurality of sampling frames in a RAW format are generally continuously acquired during exposure, and then the sampling frames are sequentially overlapped and averaged. After exposure is finished, the result of the superposition averaging is processed by the nonlinear response function of the camera, so that the result is converted into the pixel value of each pixel point. The pixel points with the pixel values can finally form a standard image shot by the camera. The format of the standard image can be determined by the image output format of the camera. For example, the standard image may be an RGB image, a JPEG image, a PNG image, or the like. As can be seen, the existing camera imaging process involves two different types of images. The RAW images (images in RAW format) generated during the exposure period of the camera may be superimposed and averaged, and then converted into a standard image (an image in RGB format, as described above) output by the camera.
Based on the existing camera imaging process, if an object moves rapidly in the camera exposure period, the situation that the object is fuzzy can be presented when a plurality of original images are overlapped and averaged and converted into a standard image. Generally, in a standard image normally captured by a camera, a moving object is blurred, but the background is clear (since the background is usually still, the background is still clear after being averaged by multiple overlapping).
In the prior art, in order to construct rich training data, a common method is to acquire a large number of clear standard images shot by a camera, and then perform blurring processing on a part of the standard images by using a blurring kernel, so that the clear standard images and the blurred standard images can be obtained. However, this approach has two problems: 1) the accuracy of the fuzzy core determines the real degree of the sample, and in a real scene, the correct fuzzy core is difficult to estimate; 2) in the blurred standard image obtained by the blur kernel processing, the whole image is often blurred to different degrees, but this is inconsistent with the above-mentioned real imaging process (object blurring, clear background) of the camera. Therefore, the existing method for constructing the training sample has lower precision.
In view of this, an embodiment of the present application provides an image deblurring method, based on which a training sample that conforms to a real imaging process of a camera can be constructed, so that a deblurring process has higher accuracy.
Specifically, referring to fig. 1 to 3, an image deblurring method provided in an embodiment of the present application may include the following steps.
S1: a set of standard image samples is obtained, the set of standard image samples including at least a clear standard image sample.
In this embodiment, a camera may be used to collect a large number of standard images of different scenes and illumination, and then a target detection algorithm may be used to identify a standard image including a target object such as a human face, a vehicle, or a license plate from the collected standard images, and these standard images including the target object may form a standard image sample set. The target detection algorithm can comprise various algorithms such as R-CNN, Fast R-CNN, Region generation Network (RPN) and the like. In practical application, a corresponding algorithm or a target detector generated according to the algorithm can be flexibly selected according to different actual requirements and application scenes.
In this embodiment, in order to subsequently construct a clear and fuzzy training sample, the obtained standard image sample set needs to include a clear standard image sample, where the requirement for the definition of the target object in the clear standard image sample is high. In practical application, an edge detection algorithm may be adopted to detect the contour of the target object in each standard image sample, and the standard image sample where the target object with the clearest contour is located is used as the above-mentioned clear standard image sample.
In one embodiment, there may be a high demand on the performance of the camera in view of the large number of clear standard images acquired by the camera. In order to reduce the requirement on the performance of the camera, in a preferred embodiment, the above standard image sample set may be generated based on a limited number of standard image frames by using a video frame interpolation algorithm.
In particular, in the preferred embodiment, a camera may be used to capture a large number of image frame sequences of different scenes and lighting, the standard images in which are not required to be all clear standard images, and thus the performance requirements on the camera may be reduced. In the acquired image frame sequence, a target detection algorithm may be employed to identify a target image frame containing a target object. Of the identified target image frames, only two adjacent target image frames may need to be extracted. Of course, in order to construct a training sample with higher precision, the definition of the target object in the two adjacent target image frames may be higher. Compared with the foregoing embodiment, in the preferred embodiment, only two adjacent target image frames with a relatively clear target object need to be extracted from the acquired image frame sequence, and each acquired image frame does not need to have a relatively high definition. The two adjacent target image frames may be regarded as adjacent standard image frames containing the target object.
In the preferred embodiment, after the adjacent standard image frames containing the target object are acquired, a video frame interpolation algorithm may be used to interpolate an intermediate standard image frame between the adjacent standard image frames. The adjacent standard image frames show closer contents, so the frame interpolation effect is better. In practical application, frame interpolation can be performed between adjacent standard image frames for multiple times, so that multiple intermediate standard image frames are obtained. In a specific application scenario, BMBC (Bilateral Motion Estimation with Bilateral Cost estimate based on Bilateral Cost) or other similar algorithms may be adopted to perform multiple frame interpolation between adjacent standard image frames. In this way, the combination of the adjacent standard image frame and the one or more intermediate standard image frames obtained by frame interpolation can be used as the acquired standard image sample set.
In this embodiment, the standard image samples at the designated positions in the standard image sample set obtained through frame interpolation may be used as the clear standard image samples. The specified position may be, for example, the most central position in the standard image sample set. For example, in fig. 2, the standard image sample set obtained through frame interpolation contains 5 standard image samples in total, and then the 3 rd standard image sample (the standard image sample within the dashed box) can be used as the sharp standard image sample.
By means of the method of inserting frames between adjacent standard image frames for multiple times, the image frames can be guaranteed to be sufficiently sampled, and the problem that undersampling exists in subsequently synthesized blurred images due to the fact that the sampling rate is too low is solved.
S3: and converting each standard image sample in the standard image sample set into a corresponding native image sample, and performing blurring processing on each native image sample to generate a blurred native image.
Referring to fig. 2, in the present embodiment, in order to simulate a blurred image generated in a real imaging process of a camera, each standard image sample in a standard image sample set may be first converted into a corresponding native image sample. In one specific application example, the standard image sample in RGB format can be converted into a native image sample in RAW format by using cyclelsp or other similar algorithm. In this way, an image sequence consisting of standard image samples can be converted into an image sequence consisting of native image samples.
In the embodiment, a real camera imaging process can be simulated, and each raw image sample is subjected to blurring processing to generate a blurred raw image. Specifically, for any pixel point in the native image sample, the pixel mean value of the pixel point may be calculated according to the pixel value of the pixel point in each native image sample, and finally, an image formed by each pixel point having the pixel mean value may be used as the blurred native image after the blurring process. By such an algorithm of superposition averaging, a blurred native image can be obtained. The generation mode of the fuzzy original image is consistent with the real imaging process of the camera, so that the image with the fuzzy target object and clear background can be generated.
In a specific application example, the blurred native image can be calculated by the following formula:
wherein S isStandard of merit[i]Representing a standard image sample of number i in said set of standard image samples, fStandard->Native to the originalRepresenting a conversion function of converting standard image samples into native image samples, M representing the total number of standard image samples contained in the set of standard image samples.
It should be noted that, in the prior art, a usable video frame interpolation algorithm does not exist for RAW-format native images, but a video frame interpolation algorithm (BMBC or other similar algorithms) is provided for RGB-format standard images. Therefore, in the above-mentioned embodiment of the present application, the adjacent standard image samples may be interpolated first, and then the standard image sample set obtained by interpolation is converted into the corresponding native image sample set, thereby solving the problem that the native image cannot be interpolated in the prior art.
S5: restoring the blurred native image into a blurred standard image, and constructing a standard training sample set according to the blurred standard image and the clear standard image sample; and the standard training sample set is used for training the standard model so as to perform deblurring processing on the standard image to be processed through the trained standard model.
In the present embodiment, a blurred native image can be generated through the processing of step S3. As shown in fig. 2, the blurred native image may be format-converted by cyclelsp or other similar algorithms to be restored to the corresponding blurred standard image. Thus, the acquired standard image sample is processed to generate a fuzzy standard image which accords with the real imaging process of the camera. Compared with a mode of directly utilizing the fuzzy core to perform fuzzy processing on the standard image, the method has the advantages that the reality degree and the precision of the generated fuzzy standard image are higher, and the characteristics of fuzzy object and clear background can be embodied.
In a specific application example, the restored blurred standard image is calculated by the following formula:
wherein S isStandard of merit[i]Representing a standard image sample of number i in said set of standard image samples, fStandard->Native to the originalRepresenting a conversion function for converting a standard image sample into a native image sample, fPrimary ion-implantation>Standard of meritRepresenting a reduction function for reducing the blurred native image to a blurred standard image, M representing the total number of standard image samples contained in the set of standard image samples.
Referring to fig. 3, in the present embodiment, the blurred standard image and the sharp standard image sample in the standard image sample set may form a pair of blurred and sharp standard training samples. By performing the above processing on a large number of standard images, a plurality of pairs of such standard training samples can be generated, and these standard training samples can be finally constructed into a standard training sample set.
In the present embodiment, the standard model can be trained using the constructed standard training sample set. The standard model can select a corresponding neural network model according to actual scene requirements. For example, the standard model may be the SRN-DeblurNet model. In the training process, the fuzzy standard image can be input into the standard model, and in order to extract multi-scale information, the standard model can perform down-sampling on the input fuzzy standard image and extract different scale features, so that clear standard images under different resolutions are output. Then, combining the clear standard images in the training samples, loss functions under different scales can be calculated by using Euclidean distances to optimize the standard model, and finally, the standard model capable of defuzzifying the clear standard images is obtained through training of a large number of training samples. In practical applications, the loss function L calculated when the standard model is optimized can be represented as:
wherein, IiAnd
respectively representing the output standard image corresponding to the ith scale and the clear standard image in the training sample, NiIs represented byiThe number of pixels in.
The trained standard model can perform deblurring processing on the input standard image to be processed, so that clear standard images under different resolutions are output.
In one embodiment, considering that in some special scenarios, it may be necessary to directly deblur the native image, in this embodiment, a native training sample may be constructed and the native model may be trained using the native training sample, so that the input native image may be deblurred using the trained native model.
Specifically, referring to fig. 3, after obtaining the blurred native image in step S3, the blurred native image and the clear native image sample obtained by converting the sharp standard image sample may be directly used as a pair of training samples. Several such training samples may constitute a native training sample set. In a similar manner, the constructed native training sample set may be used to train a native model. The corresponding neural network model can be selected according to actual scene requirements. For example, the native model may also be the SRN-DeblurNet model. In the training process, the fuzzy native image can be input into the native model, and in order to extract multi-scale information, the native model can perform downsampling on the input fuzzy native image and extract different scale features, so that clear native images under different resolutions are output. Then, combining the clear native images in the training samples, the Euclidean distance can be used for calculating loss functions under different scales to optimize the native model, and finally, the native model capable of defuzzifying the blurred native images is obtained through training of a large number of training samples.
Therefore, the method and the device can perform deblurring processing on the standard image and the native image, and accordingly universality of deblurring processing is improved.
Referring to fig. 4, an embodiment of the present application further provides an image deblurring system, including:
the system comprises a sample set acquisition unit, a processing unit and a processing unit, wherein the sample set acquisition unit is used for acquiring a standard image sample set, and the standard image sample set at least comprises a clear standard image sample;
the blurred native image generation unit is used for converting each standard image sample in the standard image sample set into a corresponding native image sample and performing blurring processing on each native image sample to generate a blurred native image;
the standard training sample set constructing unit is used for restoring the fuzzy original image into a fuzzy standard image and constructing a standard training sample set according to the fuzzy standard image and the clear standard image sample; and the standard training sample set is used for training the standard model so as to perform deblurring processing on the standard image to be processed through the trained standard model.
Referring to fig. 5, an embodiment of the present application further provides an image deblurring method, where the method includes:
s2: a set of standard image samples is obtained, the set of standard image samples including at least a clear standard image sample.
S4: and converting each standard image sample in the standard image sample set into a corresponding native image sample, and performing blurring processing on each native image sample to generate a blurred native image.
S6: constructing a primary training sample set according to a clear primary image sample obtained by converting the fuzzy primary image and the clear standard image sample; the native training sample set is used for training the native model so as to deblur the native image to be processed through the trained native model.
Referring to fig. 6, an embodiment of the present application further provides an image deblurring system, including:
the system comprises a sample set acquisition unit, a processing unit and a processing unit, wherein the sample set acquisition unit is used for acquiring a standard image sample set, and the standard image sample set at least comprises a clear standard image sample;
the blurred native image generation unit is used for converting each standard image sample in the standard image sample set into a corresponding native image sample and performing blurring processing on each native image sample to generate a blurred native image;
the fuzzy training sample set construction unit is used for constructing a native training sample set according to a clear native image sample obtained by converting the fuzzy native image and the clear standard image sample; the native training sample set is used for training the native model so as to deblur the native image to be processed through the trained native model.
Referring to fig. 7, an embodiment of the present application further provides an image deblurring apparatus, which includes a processor and a memory, where the memory is used for storing a computer program, and the computer program, when executed by the processor, implements the image deblurring method described above.
The processor may be a Central Processing Unit (CPU). The Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or a combination thereof.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the data tracing method in the embodiments of the present invention. The processor executes various functional applications and data processing of the processor by executing non-transitory software programs, instructions and modules stored in the memory, that is, the method in the above method embodiment is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be coupled to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
An embodiment of the present application further provides a computer-readable storage medium for storing a computer program, which when executed by a processor, implements the image deblurring method described above.
According to the technical scheme, after the standard image sample set is obtained, the standard image sample can be converted into the native image sample. The format of the native image sample may be the format of the image captured by the camera during the exposure process. The original image sample is subjected to fuzzy processing, and the obtained fuzzy original image is restored into the fuzzy standard image, and the fuzzy standard image can better accord with the actual image generation mode of the camera, so that the training sample set constructed on the basis of the fuzzy standard image and the clear standard image sample in the standard image sample set can have better training effect.
Therefore, in the technical scheme provided by the application, the fuzzy standard image is not required to be acquired, but can be processed based on the standard image sample set to obtain the fuzzy standard image, so that the acquisition difficulty of the image sample is reduced, the training precision of the model can be improved, and the precision and the efficiency of deblurring processing are improved.
It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (12)
1. A method of deblurring an image, the method comprising:
acquiring a standard image sample set, wherein the standard image sample set at least comprises a clear standard image sample;
converting each standard image sample in the standard image sample set into a corresponding native image sample, and performing fuzzy processing on each native image sample to generate a fuzzy native image;
restoring the blurred native image into a blurred standard image, and constructing a standard training sample set according to the blurred standard image and the clear standard image sample; and the standard training sample set is used for training the standard model so as to perform deblurring processing on the standard image to be processed through the trained standard model.
2. The method of claim 1, wherein the standard image sample set is generated as follows:
acquiring adjacent standard image frames containing a target object, and inserting an intermediate standard image frame between the adjacent standard image frames, wherein the combination of the adjacent standard image frames and the intermediate standard image frame is used as the acquired standard image sample set.
3. The method of claim 2, wherein acquiring adjacent standard image frames containing a target object comprises:
and identifying a target image frame containing a target object from the acquired image frame sequence, and taking two adjacent target image frames in the target image frame as the acquired adjacent standard image frames.
4. The method of claim 1, wherein blurring each of the native image samples to generate a blurred native image comprises:
aiming at any pixel point in the native image samples, calculating the pixel mean value of the pixel point according to the pixel value of the pixel point in each native image sample;
and taking an image formed by each pixel point with the pixel mean value as the blurred original image after the blurring processing.
5. The method according to claim 1 or 4, characterized in that the blurred native image is calculated by the following formula:
wherein S isStandard of merit[i]Representing a standard image sample of number i in said set of standard image samples, fStandard->Native to the originalRepresenting a conversion function of converting standard image samples into native image samples, M representing the total number of standard image samples contained in the set of standard image samples.
6. The method of claim 1, wherein the blurred standard image is calculated by the following formula:
wherein S isStandard of merit[i]Representing a standard image sample of number i in said set of standard image samples, fStandard->Native to the originalRepresenting a conversion function for converting a standard image sample into a native image sample, fPrimary ion-implantation>Standard of meritRepresenting a reduction function for reducing the blurred native image to a blurred standard image, M representing the total number of standard image samples contained in the set of standard image samples.
7. The method of claim 1, further comprising:
constructing a primary training sample set according to a clear primary image sample obtained by converting the fuzzy primary image and the clear standard image sample; the native training sample set is used for training the native model so as to deblur the native image to be processed through the trained native model.
8. An image deblurring system, the system comprising:
the system comprises a sample set acquisition unit, a processing unit and a processing unit, wherein the sample set acquisition unit is used for acquiring a standard image sample set, and the standard image sample set at least comprises a clear standard image sample;
the blurred native image generation unit is used for converting each standard image sample in the standard image sample set into a corresponding native image sample and performing blurring processing on each native image sample to generate a blurred native image;
the standard training sample set constructing unit is used for restoring the fuzzy original image into a fuzzy standard image and constructing a standard training sample set according to the fuzzy standard image and the clear standard image sample; and the standard training sample set is used for training the standard model so as to perform deblurring processing on the standard image to be processed through the trained standard model.
9. A method of deblurring an image, the method comprising:
acquiring a standard image sample set, wherein the standard image sample set at least comprises a clear standard image sample;
converting each standard image sample in the standard image sample set into a corresponding native image sample, and performing fuzzy processing on each native image sample to generate a fuzzy native image;
constructing a primary training sample set according to a clear primary image sample obtained by converting the fuzzy primary image and the clear standard image sample; the native training sample set is used for training the native model so as to deblur the native image to be processed through the trained native model.
10. An image deblurring system, the system comprising:
the system comprises a sample set acquisition unit, a processing unit and a processing unit, wherein the sample set acquisition unit is used for acquiring a standard image sample set, and the standard image sample set at least comprises a clear standard image sample;
the blurred native image generation unit is used for converting each standard image sample in the standard image sample set into a corresponding native image sample and performing blurring processing on each native image sample to generate a blurred native image;
the fuzzy training sample set construction unit is used for constructing a native training sample set according to a clear native image sample obtained by converting the fuzzy native image and the clear standard image sample; the native training sample set is used for training the native model so as to deblur the native image to be processed through the trained native model.
11. An image deblurring apparatus, comprising a processor and a memory for storing a computer program which, when executed by the processor, implements the image deblurring method of any one of claims 1 to 7 and 9.
12. A computer-readable storage medium for storing a computer program which, when executed by a processor, implements the image deblurring method of any one of claims 1 to 7 and 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110699448.7A CN113269695A (en) | 2021-06-23 | 2021-06-23 | Image deblurring method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110699448.7A CN113269695A (en) | 2021-06-23 | 2021-06-23 | Image deblurring method, system, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113269695A true CN113269695A (en) | 2021-08-17 |
Family
ID=77235751
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110699448.7A Pending CN113269695A (en) | 2021-06-23 | 2021-06-23 | Image deblurring method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113269695A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127336A (en) * | 2019-11-18 | 2020-05-08 | 复旦大学 | Image signal processing method based on self-adaptive selection module |
| KR20200087340A (en) * | 2018-12-31 | 2020-07-21 | 아주대학교산학협력단 | Method and device for object recognition using deep learning |
| CN111695421A (en) * | 2020-04-30 | 2020-09-22 | 北京迈格威科技有限公司 | Image recognition method and device and electronic equipment |
| CN112950503A (en) * | 2021-02-26 | 2021-06-11 | 北京小米松果电子有限公司 | Training sample generation method and device and truth value image generation method and device |
-
2021
- 2021-06-23 CN CN202110699448.7A patent/CN113269695A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20200087340A (en) * | 2018-12-31 | 2020-07-21 | 아주대학교산학협력단 | Method and device for object recognition using deep learning |
| CN111127336A (en) * | 2019-11-18 | 2020-05-08 | 复旦大学 | Image signal processing method based on self-adaptive selection module |
| CN111695421A (en) * | 2020-04-30 | 2020-09-22 | 北京迈格威科技有限公司 | Image recognition method and device and electronic equipment |
| CN112950503A (en) * | 2021-02-26 | 2021-06-11 | 北京小米松果电子有限公司 | Training sample generation method and device and truth value image generation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110428366B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN108694705B (en) | Multi-frame image registration and fusion denoising method | |
| Chen et al. | Camera lens super-resolution | |
| WO2020259118A1 (en) | Method and device for image processing, method and device for training object detection model | |
| CN109190581B (en) | Image sequence target detection and identification method | |
| CN110334635B (en) | Subject tracking method, apparatus, electronic device and computer-readable storage medium | |
| CN110473185B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN111968064B (en) | Image processing method and device, electronic equipment and storage medium | |
| EP3798975B1 (en) | Method and apparatus for detecting subject, electronic device, and computer readable storage medium | |
| CN114764868A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN110136055B (en) | Super resolution method and device for image, storage medium and electronic device | |
| CN112602088B (en) | Method, system and computer readable medium for improving quality of low light images | |
| CN106169173B (en) | Image interpolation method | |
| CN112435223B (en) | Target detection method, device and storage medium | |
| CN110766153A (en) | Neural network model training method and device and terminal equipment | |
| JP7543080B2 (en) | Trained model and data processing device | |
| CN112085768A (en) | Optical flow information prediction method, optical flow information prediction device, electronic device, and storage medium | |
| CN116486288A (en) | Aerial target counting and detecting method based on lightweight density estimation network | |
| Zhang et al. | Deep motion blur removal using noisy/blurry image pairs | |
| CN110365897B (en) | Image correction method and device, electronic equipment and computer readable storage medium | |
| Chang et al. | Beyond camera motion blur removing: How to handle outliers in deblurring | |
| CN114511702A (en) | Remote sensing image segmentation method and system based on multi-scale weighted attention | |
| CN111539975B (en) | Method, device, equipment and storage medium for detecting moving object | |
| CN115830064B (en) | Weak and small target tracking method and device based on infrared pulse signals | |
| CN111160340A (en) | Moving target detection method and device, storage medium and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |