CN112950496A - Image sample set generation method, image sample set generation device and non-volatile computer-readable storage medium - Google Patents

Image sample set generation method, image sample set generation device and non-volatile computer-readable storage medium Download PDF

Info

Publication number
CN112950496A
CN112950496A CN202110172289.5A CN202110172289A CN112950496A CN 112950496 A CN112950496 A CN 112950496A CN 202110172289 A CN202110172289 A CN 202110172289A CN 112950496 A CN112950496 A CN 112950496A
Authority
CN
China
Prior art keywords
image
blur kernel
blur
sub
kernel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110172289.5A
Other languages
Chinese (zh)
Inventor
胡杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202110172289.5A priority Critical patent/CN112950496A/en
Publication of CN112950496A publication Critical patent/CN112950496A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/02—Affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

本申请提供一种图像样本集的生成方法、生成装置及计算机可读存储介质。生成方法包括：获取第一图像，第一图像具有第一分辨率；对预设的第一模糊核进行处理以获取第二模糊核，第二模糊核与第一模糊核的类型或模糊程度不同；利用第一模糊核或第二模糊核对第一图像进行卷积操作，以获取与第一图像对应的第二图像，第二图像具有第二分辨率，第一分辨率大于第二分辨率，第一图像与第二图像构成图像样本对，循环执行卷积操作以获取多个图像样本对，并将多个图像样本对集合形成图像样本集。本申请的图像样本集的生成方法、生成装置及计算机可读存储介质中，对第一图像执行循环卷积操作可获取大量的图像样本对，降低图像样本对的获取成本。

Figure 202110172289

The present application provides a method for generating an image sample set, a generating device, and a computer-readable storage medium. The generating method includes: acquiring a first image, the first image having a first resolution; processing a preset first blur kernel to obtain a second blur kernel, the second blur kernel and the first blur kernel are different in type or blur degree ; use the first blur kernel or the second blur kernel to perform a convolution operation on the first image to obtain a second image corresponding to the first image, the second image has a second resolution, and the first resolution is greater than the second resolution, The first image and the second image form an image sample pair, a convolution operation is performed cyclically to obtain multiple image sample pairs, and the multiple image sample pairs are assembled to form an image sample set. In the method for generating an image sample set, the generating device and the computer-readable storage medium of the present application, a circular convolution operation is performed on the first image to obtain a large number of image sample pairs, thereby reducing the acquisition cost of the image sample pairs.

Figure 202110172289

Description

Image sample set generation method, image sample set generation device and non-volatile computer-readable storage medium
Technical Field
The present application relates to the field of deblurring, and more particularly, to a method and an apparatus for generating an image sample set, and a non-volatile computer-readable storage medium.
Background
At present, in the field of deblurring, with the popularization of deep learning algorithms, people begin to use the deep learning algorithms to solve the problem of image blurring. The key point for the successful training of the current deep learning model is on one hand the effective model structure and on the other hand the requirement of a large amount of training sample data. When an effective strong supervision training mode is adopted to train the fuzzy model, paired images (a clear image and a corresponding fuzzy image) are required to be used as a training sample pair, and the two images are required to be aligned pixel by pixel on the content details, so that the acquisition cost of the sample pair is increased. When a large number of image samples meeting requirements do not exist for training the fuzzy model, the fuzzy model is over-fitted, and the trained fuzzy model is difficult to apply.
Disclosure of Invention
The embodiment of the application provides a method and a device for generating an image sample set and a nonvolatile computer readable storage medium.
The method for generating the image sample set according to the embodiment of the application comprises the following steps: acquiring a first image, the first image having a first resolution; processing a preset first fuzzy core to obtain a second fuzzy core, wherein the second fuzzy core is different from the first fuzzy core in type or fuzzy degree; and performing a convolution operation on the first image by using the first blur kernel or the second blur kernel to obtain a second image corresponding to the first image, wherein the second image has a second resolution, the first resolution is higher than the second resolution, the first image and the second image form an image sample pair, the convolution operation is performed in a circulating mode to obtain a plurality of image sample pairs, and the image sample pairs are collected to form an image sample set.
The generation device of the embodiment of the present application includes one or more processors configured to: acquiring a first image, the first image having a first resolution; processing a preset first fuzzy core to obtain a second fuzzy core, wherein the second fuzzy core is different from the first fuzzy core in type or fuzzy degree; and performing a convolution operation on the first image by using the first blur kernel or the second blur kernel to obtain a second image corresponding to the first image, wherein the second image has a second resolution, the first resolution is higher than the second resolution, the first image and the second image form an image sample pair, the convolution operation is performed in a circulating mode to obtain a plurality of image sample pairs, and the image sample pairs are collected to form an image sample set.
The non-transitory computer-readable storage medium of the embodiment of the present application includes a computer program that, when executed by a processor, causes the processor to execute a processing method of: acquiring a first image, the first image having a first resolution; processing a preset first fuzzy core to obtain a second fuzzy core, wherein the second fuzzy core is different from the first fuzzy core in type or fuzzy degree; and performing a convolution operation on the first image by using the first blur kernel or the second blur kernel to obtain a second image corresponding to the first image, wherein the second image has a second resolution, the first resolution is higher than the second resolution, the first image and the second image form an image sample pair, the convolution operation is performed in a circulating mode to obtain a plurality of image sample pairs, and the image sample pairs are collected to form an image sample set.
In the method and the device for generating the image sample set and the non-volatile computer-readable storage medium according to the embodiments of the present application, a convolution operation is performed on a first image by using a first blur kernel or a second blur kernel to generate a second image corresponding to the first image, the first image and the second image form an image sample pair, and a cyclic convolution operation is performed on the first image to obtain a plurality of image sample pairs based on the first image, so that the acquisition cost of the image sample pairs is reduced, and the number of the image sample sets can be increased.
Additional aspects and advantages of embodiments of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of embodiments of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow chart of a method of generating a sample set of images according to some embodiments of the present application;
FIG. 2 is a schematic diagram of the structure of a generating device according to some embodiments of the present application;
FIG. 3 is a schematic diagram of a first blur kernel according to some embodiments of the present application;
FIG. 4 is a schematic illustration of a pair of image samples according to some embodiments of the present application;
FIG. 5 is a flow chart of a method of generating a sample set of images according to some embodiments of the present application;
FIG. 6 is a diagram illustrating a first blur kernel undergoing a scaling transformation to obtain a second blur kernel according to some embodiments of the present application;
FIG. 7 is a diagram illustrating a first blur kernel undergoing a rotation transformation to obtain a second blur kernel according to some embodiments of the present application;
FIG. 8 is a flow chart of a method of generating a sample set of images according to some embodiments of the present application;
FIG. 9 is a schematic diagram of a connection between a non-volatile computer readable storage medium and a processor according to some embodiments of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of explaining the embodiments of the present application, and are not to be construed as limiting the embodiments of the present application.
Referring to fig. 1 to 4, an embodiment of the present application provides a method for generating an image sample set, where the method includes:
01: acquiring a first image P1, the first image P1 having a first resolution;
03: processing a preset first fuzzy core H1 to obtain a second fuzzy core H2, wherein the second fuzzy core H2 is different from the first fuzzy core H1 in type or fuzzy degree; and
05: convolving the first image P1 with the first blur kernel H1 or the second blur kernel H2 to obtain a second image P2 corresponding to the first image P1, the second image P2 having a second resolution, the first resolution being greater than the second resolution, the first image P1 and the second image P2 forming pairs of image samples, circularly performing the convolution operation to obtain a plurality of pairs of image samples, and forming the plurality of sets of pairs of image samples into a set of image samples.
Referring to fig. 2, the present application further provides a generating apparatus 100, where the generating apparatus 100 includes one or more processors 10, and the generating method of the embodiment of the present application can be applied to the generating apparatus 100 of the embodiment of the present application, where one or more processors 10 are configured to execute the method in 01, 03, and 05, that is, one or more processors 10 are configured to: acquiring a first image P1, the first image P1 having a first resolution; processing a preset first fuzzy core H1 to obtain a second fuzzy core H2, wherein the second fuzzy core H2 is different from the first fuzzy core H1 in type or fuzzy degree; and performing a convolution operation on the first image P1 by using the first blurring kernel H1 or the second blurring kernel H2 to obtain a second image P2 corresponding to the first image P1, wherein the second image P2 has a second resolution, the first resolution is higher than the second resolution, the first image P1 and the second image P2 form an image sample pair, the convolution operation is performed in a loop to obtain a plurality of image sample pairs, and the plurality of image sample pairs are collected to form an image sample set.
The image sample set generated herein can be applied to deblurring processing in the image field or other processing that needs to be applied to a pair of sharp and blurred images. The generating apparatus 100 may be any device that needs to use the image sample set, such as a mobile phone, a watch, a notebook computer, a tablet computer, a desktop computer, an intelligent eye, an intelligent helmet, an unmanned aerial vehicle, and the like, without limitation. Referring to fig. 2, the present application only illustrates that the generating device 100 is a mobile phone, and the situation when the generating device 100 is another type of device is similar to that when it is a mobile phone, and is not described in detail. The processor 10 may be integrated in the generating device 100.
At present, the more mature method for making the motion blur removed data set is as follows: the method comprises the steps of shooting a video of a moving picture through a high frame rate (240fps) mode of a camera, accumulating and averaging n continuous frames in the video, correcting gamma to obtain a motion blurred image, and selecting a middle frame in the n frames as a clear frame corresponding to the middle frame to form an image sample pair. The current virtual focus blurred image sample pair is prepared in the following way: the method comprises the steps that a shot object is a static scene, the focusing state of a camera is adjusted manually, one shot is taken when the scene is in focus, and the other shot is taken when the scene is in virtual focus, so that a virtual focus fuzzy image sample pair is obtained.
In the method for generating an image sample set according to the embodiment of the present application, a single first image P1 (sharp image) is acquired, and a convolution operation is performed on the first image P1 by using a first blurring kernel H1 or a second blurring kernel H2 to generate a second image P2 (blurred image) corresponding to the first image P1, wherein the first image P1 and the second image P2 form an image sample pair, and a cyclic convolution operation is performed on the first image P1 to acquire a plurality of image sample pairs based on the first image P1, so as to increase the number of the image sample sets and reduce the acquisition cost of the image sample pairs.
Referring to fig. 3 and 4, in the embodiment of the present application, the predetermined first blur kernel H1 may be designed according to an actual application scenario. The blur kernel is actually a matrix, and the blur kernel is converted into an image by visualizing the matrix, so that the acquisition of the first blur kernel H1 can be obtained by designing the matrix and visualizing the matrix. In particular, different kinds of first blur kernel H1 may be generated by the blur kernel generator, e.g., fig. 3 including 1-15 different kinds of first blur kernels H1. When the first image P1 is convolved with different kinds of first blur kernels H1, different kinds of second images P2 can be obtained, for example, the first blur kernel H1 numbered 1 in fig. 3 convolves the first image P1 to obtain the first kind of second image P2 (numbered 1), the first blur kernel H1 numbered 2 in fig. 3 convolves the first image P1 to obtain the second kind of second image P2 (numbered 2), the first blur kernel H1 numbered 3 in fig. 3 convolves the first image P1 to obtain the third kind of second image P2 (numbered 3), and so on, fifteen kinds of second images P2 can be obtained, numbered from 1 to 15. The first image P1 respectively forms image sample pairs with a plurality of different kinds of second images P2, for example, the first image P1 forms a first image sample pair with the second image P2 numbered 1, the first image P1 forms a second image sample pair with the second image P2 numbered 2, the first image P1 forms a third image sample pair with the second image P2 numbered 3, and so on, fifteen image sample pairs can be obtained. Thus, a greater number of pairs of image samples can be acquired by a single first image P1 to increase the number of image sample sets and reduce the acquisition cost of the image sample pairs. The number of the first blur kernel H1 is not limited to 15 shown in fig. 3, and may be one or more, and may be considered as a set number.
In some embodiments, when the convolution operation is performed on the first image P1 by using the first blur kernel H1 or the second blur kernel H2, the pixel values of each point in the first image P1 form a matrix, and since the blur kernel is actually a matrix, the matrix formed by the second image P2 is obtained by performing a mathematical operation on the two matrices. For example, the first image P1 forms a matrix of
Figure BDA0002939110990000041
The matrix formed by the first fuzzy core H1 or the second fuzzy core H2 is
Figure BDA0002939110990000051
The matrix of the second image P2 obtained after the convolution operation is
Figure BDA0002939110990000052
Wherein, C (1, 1) — 10 × (-1) +10 × 0+10 × 1 ═ 10, that is, matrix a is sequentially moved from left to right, from top to bottom row by row, and divided into matrices of the same size as matrix B, and then matrix-multiplied by matrix B and added to obtain corresponding element values in matrix C.
Referring to fig. 5, in some embodiments, method 03: processing the preset first fuzzy core H1 to obtain a second fuzzy core H2, including:
031: performing one or more of translation, scaling, rotation, or affine transformation on the first blur kernel H1 to obtain an intermediate blur kernel; and
033: and calculating the point values of the second fuzzy core H2 according to the point values of the intermediate fuzzy core and the point values of the first fuzzy core H1 to obtain a second fuzzy core H2.
Referring to fig. 2, one or more processors 10 are also configured to execute 031, 033, i.e., one or more processors 10 are configured to: performing one or more of translation, scaling, rotation, or affine transformation on the first blur kernel H1 to obtain an intermediate blur kernel; and calculating the point values of the second fuzzy core H2 according to the point values of the intermediate fuzzy core and the point values of the first fuzzy core H1 to obtain a second fuzzy core H2.
In particular, the one or more processors 10 may implement any one or more of translation, scaling, rotation, or affine transformation on a preset first blur kernel H1 to obtain more kinds of intermediate blur kernels with different degrees of blur, and computing different kinds of intermediate blur kernels with different degrees of blur may obtain a second blur kernel H2 with different kinds of blur or different degrees of blur. Therefore, the types of the blur kernels are increased, the data volume of the image sample set is increased more easily, and various types of blur (such as motion blur, virtual focus blur and lens blur) can be simulated.
In some embodiments, the one or more processors 10 perform any one of translation, scaling, rotation, or emulation on a preset first blur kernel H1 to obtain an intermediate blur kernel, and obtain point values of the second blur kernel H2 according to the point values of the intermediate blur kernel and the point values of the first blur kernel H1, so as to obtain a second blur kernel H2 corresponding to the first blur kernel H1.
In one embodiment, the one or more processors 10 perform a panning transformation on a preset first blur kernel H1 to obtain an intermediate blur kernel corresponding to the first blur kernel H1. Specifically, the first blurring kernel H1 includes a plurality of pixels, and the first blurring kernel H1 performs a translation transformation such that each pixel of the first blurring kernel H1 is translated from an initial position to an end position by a given offset, and finally, a gray value of each pixel of the intermediate blurring kernel is obtained through an interpolation algorithm. For example, the coordinate of a certain pixel in the first blurring kernel H1 in space is (x)0，y0) The first blur kernel H1 is translated in the horizontal direction by xtIs translated in the vertical directiontThe pixel coordinate of the pixel to be checked by the intermediate blur kernel obtained after the translation is (x)0+xt，y0+yt). The pixel points in the first blur kernel H1 that are moved out of the image display area can be increased by increasing the size of the intermediate blur kernel (e.g., the image width of the intermediate blur kernel is increased by T)xIncrease in height Ty) To ensure that the intermediate blur kernel can contain all the pixel points in the first blur kernel H1. And finally, carrying out gray level interpolation processing on the intermediate fuzzy core to ensure that pixel points of the intermediate fuzzy core obtained after translation can correspond to pixel points in the first fuzzy core H1. The interpolation algorithm comprises a nearest neighbor interpolation method, a bilinear interpolation method and a bicubic interpolation method. After the intermediate blur kernel is obtained, the numerical values of the points of the intermediate blur kernel are divided by the accumulated value of the numerical values of the points of the first blur kernel H1 to obtain a second blur kernel H2 corresponding to the intermediate blur kernel, so that the numerical accumulation of the points of the obtained second blur kernel H2 is equal to 1, and the brightness of the second blur kernel H2 obtained by transforming the first blur kernel H1 is consistent with the brightness of the first blur kernel H1.
Referring to fig. 6, in one example, one or more processors 10 perform a scaling transform on a preset first blur core H1 to obtain an intermediate blur core corresponding to the first blur core H1. Specifically, the first blurring kernel H1 includes a plurality of pixels, and the first blurring kernel H1 performs a scaling transform including a magnification transform or a reduction transformThat is, the first blur kernel H1 is scaled up or down by a certain scale, and finally, the gray value of each pixel of the intermediate blur kernel is obtained through an interpolation algorithm. For example, the coordinate of a certain pixel in the first blurring kernel H1 in space is (x)0，y0) Scaling s in the horizontal direction by scaling the first blur kernel H1 centered at an arbitrary point (x, y)xMultiple, scaling s in the vertical directionyMultiplying, then the scaled coordinate of the point pixel is (x)0+sx(x-x0)，y0+sy(y-y0). Similarly, for pixels in the first blur kernel H1 that are moved out of the image display area, the size of the intermediate blur kernel may be increased (e.g., the image width of the intermediate blur kernel is increased by T)xIncrease in height Ty) To ensure that the intermediate blur kernel can contain all the pixel points in the first blur kernel H1. And finally, carrying out gray level interpolation processing on the intermediate fuzzy core to ensure that all pixel points of the intermediate fuzzy core obtained after zooming can correspond to the pixel points in the first fuzzy core H1. The interpolation algorithm comprises a nearest neighbor interpolation method, a bilinear interpolation method and a bicubic interpolation method. After the intermediate blur kernel is obtained, the numerical values of each point of the intermediate blur kernel are divided by the accumulated value of the numerical values of each point of the first blur kernel H1 to obtain a second blur kernel H2 corresponding to the intermediate blur kernel, so that the numerical accumulation of each point of the obtained second blur kernel H2 is equal to 1, and the brightness of the second blur kernel H2 obtained by transforming the first blur kernel H1 is consistent with the brightness of the first blur kernel H1.
Referring to fig. 7, in one embodiment, the one or more processors 10 perform a rotation transformation on a preset first blur kernel H1 to obtain an intermediate blur kernel corresponding to the first blur kernel H1. Specifically, the first blur kernel H1 includes a plurality of pixels, and the rotation transformation performed by the first blur kernel H1 is that each pixel point in the first blur kernel H1 rotates around a certain point as a center by a certain angle, and finally, the gray value of each pixel of the intermediate blur kernel is obtained through an interpolation algorithm. For example, the coordinate of a certain pixel in the first blurring kernel H1 in space is (x)0，y0) The corresponding point of the first blur kernel H1 after being rotated counterclockwise by an angle θ about a point is (x, y), where(x, y) can be calculated by the following matrix expression.
Figure BDA0002939110990000061
Similarly, for pixels in the first blur kernel H1 that are moved out of the image display area, the size of the intermediate blur kernel may be increased (e.g., the image width of the intermediate blur kernel is increased by T)xIncrease in height Ty) The interpolation method includes a nearest neighbor interpolation method, a bilinear interpolation method and a bicubic interpolation method. The certified intermediate blur kernel can contain all the pixel points in the first blur kernel H1. And finally, carrying out gray level interpolation processing on the intermediate fuzzy core to ensure that all pixel points of the intermediate fuzzy core obtained after zooming can correspond to the pixel points in the first fuzzy core H1. After the intermediate blur kernel is reached, dividing the numerical values of each point of the intermediate blur kernel by the accumulated value of the numerical values of each point of the first blur kernel H1 to obtain a second blur kernel H2 corresponding to the intermediate blur kernel, so as to ensure that the numerical accumulation of each point of the obtained second blur kernel H2 is equal to 1, and further ensure that the brightness of the second blur kernel H2 obtained by transforming the first blur kernel H1 is consistent with the brightness of the first blur kernel H1.
Referring to fig. 2, in one embodiment, the one or more processors 10 perform affine transformation on a preset first blur kernel H1 to obtain an intermediate blur kernel corresponding to the first blur kernel H1. Specifically, the first blur kernel H1 includes a plurality of pixels, and the affine transformation performed by the first blur kernel H1 is that each pixel point in the first blur kernel H1 is subjected to linear (e.g., translation, scaling, rotation, miscut, etc.) transformation once and added with a translation, and finally, the gray value of each pixel of the intermediate blur kernel is obtained through an interpolation algorithm. For example, the coordinate of a certain pixel in the first blurring kernel H1 in space is (x)0，y0) Affine matrices are 2-by-3 matrices, e.g. matrices
Figure BDA0002939110990000071
Wherein the elements of the third column play a translational role, the diagonal of the elements of the first two columns play a zooming role, and the anti-diagonal plays a roleWith a rotating or shearing action. A certain pixel (x) in the first blur kernel H10，y0) And (3) obtaining an intermediate fuzzy core corresponding point after affine matrix A transformation, wherein the (x, y) can be calculated by the following matrix expression.
Figure BDA0002939110990000072
Similarly, for pixels in the first blur kernel H1 that are moved out of the image display area, the size of the intermediate blur kernel may be increased (e.g., the image width of the intermediate blur kernel is increased by T)xIncrease in height Ty) To ensure that the intermediate blur kernel can contain all the pixel points in the first blur kernel H1. And finally, carrying out gray level interpolation processing on the intermediate fuzzy core to ensure that all pixel points of the intermediate fuzzy core obtained after zooming can correspond to the pixel points in the first fuzzy core H1. The interpolation algorithm comprises a nearest neighbor interpolation method, a bilinear interpolation method and a bicubic interpolation method. After the intermediate blur kernel is obtained, the numerical values of each point of the intermediate blur kernel are divided by the accumulated value of the numerical values of each point of the first blur kernel H1 to obtain a second blur kernel H2 corresponding to the intermediate blur kernel, so that the numerical accumulation of each point of the obtained second blur kernel H2 is equal to 1, and the brightness of the second blur kernel H2 obtained by transforming the first blur kernel H1 is consistent with the brightness of the first blur kernel H1.
Referring to fig. 2, in some embodiments, the one or more processors 10 perform a plurality of operations of translating, scaling, rotating or emulating the preset first blur kernel H1 to obtain the intermediate blur kernel. For example, the one or more processors 10 sequentially perform panning and scaling transformations on a preset first blur kernel H1 to obtain intermediate blur kernels; for another example, the one or more processors 10 sequentially perform panning and retranslating transforms on the preset first blur kernel H1 to obtain intermediate blur kernels. That is, when the one or more processors 10 may perform any combination of translation, scaling, rotation, and affine transformation on the preset first blur kernel H1 to obtain intermediate blur kernels with more blur types or blur degrees, so as to obtain more second blur kernels H2 corresponding to the first blur kernel H1. Various transformation principles are consistent with the above description, and are not described in detail herein.
Referring to fig. 2 and 4, in the embodiment of the present application, the types of the first blur kernel H1 include multiple types, and the one or more processors 10 may randomly select a predetermined first blur kernel H1 to perform one or more transformations of translation, scaling, rotation, or affine transformation, so as to obtain a second blur kernel H2 with a greater variety or blur degree different from that of the first blur kernel H1. When performing a convolution operation on the first image P1, the one or more processors 10 randomly select the first blurring kernel H1 or the second blurring kernel H2 to perform blurring processing on the first image P1, so that the second image P2 with different blurring types or blurring degrees can be obtained only by performing a single first image P1, thereby obtaining multiple pairs of image sample pairs, and performing a circular convolution operation on the first image P1 can further increase the number of the image sample pairs, so that the number of the image sample sets is richer. Meanwhile, one or more processors 10 perform convolution operations by acquiring different first images P1 and then using different first blurring kernels H1 or second blurring kernels H2 for different first images P1, so that scenes of the image sample set are richer.
Referring to FIG. 2, in some embodiments, one or more processors 10 perform multiple transformations on each first obfuscated core H1, in the same or different manners.
When the one or more processors 10 perform multiple transformations on each first blur core H1, the manner of the multiple transformations may be the same. For example, the one or more processors 10 may perform two, three, or more times of translation or scaling or rotation or affine transformation on each first blur kernel H1, and may obtain the second blur kernels H2 having different blur types or blur degrees by transforming the first blur kernel H1 a plurality of times to obtain intermediate blur kernels, thereby increasing the types of blur kernels for the convolution operation of the first image P1 to increase the types of image sample pairs.
When the one or more processors 10 perform multiple transformations on each first blur core H1, the manner of the multiple transformations may be different. For example, when the number of transformations is two, the one or more processors 10 perform a translation on the first blur kernel H1, then may perform one or more of scaling, rotation, and emulation on the transformed first blur kernel H1 to obtain an intermediate blur kernel, and finally perform a calculation on the intermediate blur kernel to obtain a second blur kernel H2 with an image brightness consistent with the first blur kernel H1. For another example, when the number of times of transformation is three or more, the one or more processors 10 may perform translation on the first blur kernel H1, perform any one of scaling, rotation, and affine transformation on the transformed first blur kernel H1, and finally perform one or more of translation, scaling, rotation, and reflection modeling on the twice transformed first blur kernel H1 to obtain an intermediate blur kernel, and finally perform calculation on the intermediate blur kernel to obtain the second blur kernel H2 whose image brightness is consistent with the first blur kernel H1. Of course, when one or more processors 10 perform multiple transformations on each first obfuscation kernel H1, the way of the multiple transformations may be partially different and partially the same.
Referring to fig. 8, in some embodiments, the generating method may further include:
02: processing the first image P1 to obtain at least one first sub-image, the first sub-image including a subject to be blurred;
05: performing a convolution operation on the first image P1 by using the first blurring kernel H1 or the second blurring kernel H2 to obtain a second image P2 corresponding to the first image P1, including:
051: performing convolution operation on the first sub-image by using the first blurring kernel H1 or the second blurring kernel H2 to obtain a second sub-image; and
053: the second sub-image and the first image P1 are fused to obtain a second image P2.
Referring to fig. 2, one or more processors 10 are also configured to perform the methods of 02, 051, 053, i.e., one or more processors 10 are further configured to: processing the first image P1 to obtain at least one first sub-image, the first sub-image including a subject to be blurred; performing convolution operation on the first sub-image by using the first blurring kernel H1 or the second blurring kernel H2 to obtain a second sub-image; and fusing the second sub-image and the first image P1 to obtain a second image P2.
Referring to FIG. 4, in one embodiment, to simulate a non-uniform blur of the full map of the first image P1, i.e., the full map corresponds to a plurality of different types of blur kernels. In one example, the first image P1 is subjected to image segmentation to obtain at least one first sub-image, where the number of the first sub-images may include one, two, three or more, and the specific number of the first sub-images is related to the number of the subject to be blurred contained in the first image P1. For example, the first image P1 includes three subjects to be blurred, that is, when the first image P1 is blurred, three different types of blur kernels need to be used for performing convolution operation, so that the second image P2 is a blurred image including three different types of blur kernels, and the image generation apparatus 100 (shown in fig. 2) can generate various types of blur generated by various types of targets in more real scenes, so that the blur model trained through deep learning does not have a condition of overfitting, and the trained blur types have high practicability.
In particular, segmenting the first image P1 may obtain at least one first sub-image by a threshold segmentation method. The threshold segmentation method is obtained by the following transformation formula:
Figure BDA0002939110990000091
where T is the threshold, g is the first sub-image, f is the first image P1, g (i, j) is 1 for the image of the object and g (i, j) is 0 for the image element of the background. The determination of the threshold T may be segmented by a global threshold, an adaptive threshold, an optimal threshold. For example, when the background and foreground portions of the first image P1 are significantly contrasted, a global threshold may be used for segmentation. For another example, when the contrast of the object and the background of the first image P1 is different everywhere in the first image P1, segmentation may be performed with different thresholds according to the local features of the first image P1. Preferably, the first image P1 may be segmented using adaptive thresholds, i.e. different thresholds are selected for different regions of the first image P1 or the threshold at each point is dynamically selected according to a certain neighborhood range for image segmentation.
After the first image P1 is divided into a plurality of first sub-images, the first sub-images are convolved by the one or more processors 10 (shown in fig. 2) by using different types of first blur kernel H1 or second blur kernel H2 to obtain second sub-images corresponding to the first sub-images, and finally the second sub-images and the divided first image P1 are subjected to a fitting process to obtain a second image P2 corresponding to the first image P1. The specific convolution operation is consistent with the above-mentioned convolution operation principle, and is not described herein again.
In some embodiments, the first sub-image comprises a plurality of different first sub-images, different first sub-images being convolved with different first blurring kernel H1 or second blurring kernel H2; the subject to be blurred in the first sub-images includes a plurality of categories, and the first sub-images of the subject to be blurred with the same category are convolved by using the same first blurring kernel H1 or the second blurring kernel H2.
For example, the first image P1 is divided into a plurality of different first sub-images, which may be convolved with different first blurring kernels H1 or second blurring kernels H2, so that the resulting second image P2 (blurred image) more closely resembles the type of blur in a real scene, e.g. moving classes of objects in the street, from which scene the first image P1 (sharp image) was taken, due to the different motion trajectories of different objects in the scene, the first image P1 (sharp image) needs to be segmented into a plurality of different first sub-images, and at the same time, convolving the first sub-image with a different first blur kernel H1 or second blur kernel H2 to obtain a second image P2 (blurred image) that corresponds more closely to the first image P1 (sharp image) in the scene, such that pairs of image samples in the set of image samples have pairs of motion blurred samples in the real scene.
For another example, the first image P1 is divided into a plurality of different first sub-images, where the subject to be blurred in the plurality of first sub-images includes a plurality of categories (e.g., categories including human, animal, plant, etc.), where both the subject to be blurred in the two divided first sub-images are stationary plants, the two first sub-images may be convolved by using the same kind of first blur kernel H1 or second blur kernel H2, and the person or other animal may be convolved by using another kind of first blur kernel H1 or second blur kernel H2, so that the second image P2 (blurred image) is closer to a blurred image of a scene image (including human, animal, plant) actually captured, so that the image samples in the image sample set have motion blurred sample pairs in a real scene.
Referring to fig. 9, the present application provides a non-volatile computer-readable storage medium 200 comprising 201. The computer program 201, when executed by the one or more processors 10, causes the one or more processors 10 to perform the generation methods in 01, 02, 03, 031, 033, 05, 051, 053.
For example, the computer program 201, when executed by the one or more processors 10, causes the processors 10 to perform the generation method of:
01: acquiring a first image P1, the first image P1 having a first resolution;
03: processing a preset first fuzzy core H1 to obtain a second fuzzy core H2, wherein the second fuzzy core H2 is different from the first fuzzy core H1 in type or fuzzy degree; and
05: convolving the first image P1 with the first blur kernel H1 or the second blur kernel H2 to obtain a second image P2 corresponding to the first image P1, the second image P2 having a second resolution, the first resolution being greater than the second resolution, the first image P1 and the second image P2 forming pairs of image samples, circularly performing the convolution operation to obtain a plurality of pairs of image samples, and forming the plurality of sets of pairs of image samples into a set of image samples.
As another example, the computer program 201, when executed by the one or more processors 10, causes the processors 10 to perform the generation method as follows:
01: acquiring a first image P1, the first image P1 having a first resolution;
031: performing one or more of translation, scaling, rotation, or affine transformation on the first blur kernel H1 to obtain an intermediate blur kernel;
033: calculating the numerical values of all points of a second fuzzy core H2 according to the numerical values of all points of the intermediate fuzzy core and the numerical values of all points of the first fuzzy core H1 to obtain a second fuzzy core H2;
05: convolving the first image P1 with the first blur kernel H1 or the second blur kernel H2 to obtain a second image P2 corresponding to the first image P1, the second image P2 having a second resolution, the first resolution being greater than the second resolution, the first image P1 and the second image P2 forming pairs of image samples, circularly performing the convolution operation to obtain a plurality of pairs of image samples, and forming the plurality of sets of pairs of image samples into a set of image samples.
As another example, the computer program 201, when executed by the one or more processors 10, causes the processors 10 to perform the generation method as follows:
01: acquiring a first image P1, the first image P1 having a first resolution;
02: processing the first image P1 to obtain at least one first sub-image, the first sub-image including a subject to be blurred;
03: processing a preset first fuzzy core H1 to obtain a second fuzzy core H2, wherein the second fuzzy core H2 is different from the first fuzzy core H1 in type or fuzzy degree;
051: performing convolution operation on the first sub-image by using the first blurring kernel H1 or the second blurring kernel H2 to obtain a second sub-image; and
053: the second sub-image and the first image P1 are fused to obtain a second image P2.
Also for example, the computer program 201, when executed by the one or more processors 10, causes the processors 10 to perform the generation method of:
01: acquiring a first image P1, the first image P1 having a first resolution;
02: processing the first image P1 to obtain at least one first sub-image, the first sub-image including a subject to be blurred;
031: performing one or more of translation, scaling, rotation, or affine transformation on the first blur kernel H1 to obtain an intermediate blur kernel;
033: calculating the numerical values of all points of a second fuzzy core H2 according to the numerical values of all points of the intermediate fuzzy core and the numerical values of all points of the first fuzzy core H1 to obtain a second fuzzy core H2;
051: performing convolution operation on the first sub-image by using the first blurring kernel H1 or the second blurring kernel H2 to obtain a second sub-image; and
053: the second sub-image and the first image P1 are fused to obtain a second image P2.
In the description herein, references to the description of the terms "certain embodiments," "one example," "exemplary," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
Although embodiments of the present application have been shown and described above, it is to be understood that the above embodiments are exemplary and not to be construed as limiting the present application, and that changes, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.

Claims (11)

1.一种图像样本集的生成方法，其特征在于，包括：1. a generation method of image sample set, is characterized in that, comprises: 获取第一图像，所述第一图像具有第一分辨率；acquiring a first image, the first image having a first resolution; 对预设的第一模糊核进行处理以获取第二模糊核，所述第二模糊核与第一模糊核的类型或模糊程度不同；及processing the preset first blur kernel to obtain a second blur kernel, the second blur kernel is different in type or blur degree from the first blur kernel; and 利用所述第一模糊核或所述第二模糊核对所述第一图像进行卷积操作，以获取与所述第一图像对应的第二图像，所述第二图像具有第二分辨率，所述第一分辨率大于所述第二分辨率，所述第一图像与所述第二图像构成图像样本对，循环执行卷积操作以获取多个所述图像样本对，并将多个所述图像样本对集合形成图像样本集。A convolution operation is performed on the first image by using the first blur kernel or the second blur kernel to obtain a second image corresponding to the first image, the second image has a second resolution, so The first resolution is greater than the second resolution, the first image and the second image constitute an image sample pair, and a convolution operation is performed cyclically to obtain a plurality of the image sample pairs, and a plurality of the image sample pairs are obtained. The collection of image sample pairs forms an image sample set. 2.根据权利要求1所述的生成方法，其特征在于，所述对预设的第一模糊核进行处理以获取第二模糊核，包括：2 . The generating method according to claim 1 , wherein the processing of the preset first blur kernel to obtain the second blur kernel comprises: 2 . 对所述第一模糊核执行平移、缩放、旋转或仿射变换中的一种或多种以获取中间模糊核；及performing one or more of translation, scaling, rotation, or affine transformation on the first blur kernel to obtain an intermediate blur kernel; and 根据所述中间模糊核的各点数值及所述第一模糊核各点数值计算所述第二模糊核的各点数值，以获取所述第二模糊核。Calculate the value of each point of the second blur kernel according to the value of each point of the intermediate blur kernel and the value of each point of the first blur kernel, so as to obtain the second blur kernel. 3.根据权利要求2所述的生成方法，其特征在于，对每个所述第一模糊核执行多次变换，所述多次变换的方式相同或不同。3 . The generating method according to claim 2 , wherein multiple transformations are performed on each of the first blur kernels, and the multiple transformations are performed in the same or different manners. 4 . 4.根据权利要求1所述的生成方法，其特征在于，所述生成方法还包括：4. The generation method according to claim 1, wherein the generation method further comprises: 处理所述第一图像以获取至少一个第一子图像，所述第一子图像中包括待模糊主体；processing the first image to obtain at least one first sub-image, the first sub-image includes the subject to be blurred; 所述利用所述第一模糊核或所述第二模糊核对所述第一图像进行卷积操作，以获取与所述第一图像对应的第二图像，包括：The performing a convolution operation on the first image by using the first blur kernel or the second blur kernel to obtain a second image corresponding to the first image includes: 利用所述第一模糊核或所述第二模糊核对所述第一子图像进行卷积操作，以获取第二子图像；及Convolving the first sub-image with the first blur kernel or the second blur kernel to obtain a second sub-image; and 融合所述第二子图像及所述第一图像以获取第二图像。The second sub-image and the first image are fused to obtain a second image. 5.根据权利要求4所述的生成方法，其特征在于，所述第一子图像包括多个，不同的所述第一子图像采用不同的所述第一模糊核或所述第二模糊核进行卷积操作；或5 . The generating method according to claim 4 , wherein the first sub-image includes multiple ones, and different first sub-images adopt different first blur kernels or second blur kernels. 6 . perform a convolution operation; or 多个所述第一子图像中的待模糊主体包括多个类别，具有相同类别的待模糊主体的所述第一子图像采用相同的所述第一模糊核或所述第二模糊核进行卷积操作。The subjects to be blurred in the plurality of first sub-images include a plurality of categories, and the first sub-images of the subjects to be blurred with the same category use the same first blur kernel or the same second blur kernel for rolling. accumulation operation. 6.一种用于图像样本集的生成装置，其特征在于，包括一个或多个处理器，一个或多个所述处理器用于：6. An apparatus for generating an image sample set, characterized in that it comprises one or more processors, wherein the one or more processors are used for: 获取第一图像，所述第一图像具有第一分辨率；acquiring a first image, the first image having a first resolution; 对预设的第一模糊核进行处理以获取第二模糊核，所述第二模糊核与第一模糊核的类型或模糊程度不同；及processing the preset first blur kernel to obtain a second blur kernel, the second blur kernel is different in type or blur degree from the first blur kernel; and 利用所述第一模糊核或所述第二模糊核对所述第一图像进行卷积操作，以获取与所述第一图像对应的第二图像，所述第二图像具有第二分辨率，所述第一分辨率大于所述第二分辨率，所述第一图像与所述第二图像构成图像样本对，循环执行卷积操作以获取多个所述图像样本对，并将多个所述图像样本对集合形成图像样本集。A convolution operation is performed on the first image by using the first blur kernel or the second blur kernel to obtain a second image corresponding to the first image, the second image has a second resolution, so The first resolution is greater than the second resolution, the first image and the second image constitute an image sample pair, and a convolution operation is performed cyclically to obtain a plurality of the image sample pairs, and a plurality of the image sample pairs are obtained. The collection of image sample pairs forms an image sample set. 7.根据权利要求6所述的生成装置，其特征在于，一个或多个所述处理器还用于：7. The generating apparatus of claim 6, wherein one or more of the processors are further configured to: 对所述第一模糊核执行平移、缩放、旋转或仿射变换中的一种或多种以获取中间模糊核；及performing one or more of translation, scaling, rotation, or affine transformation on the first blur kernel to obtain an intermediate blur kernel; and 根据所述中间模糊核的各点数值及所述第一模糊核各点数值计算所述第二模糊核的各点数值，以获取所述第二模糊核。Calculate the value of each point of the second blur kernel according to the value of each point of the intermediate blur kernel and the value of each point of the first blur kernel, so as to obtain the second blur kernel. 8.根据权利要求7所述的生成装置，其特征在于，一个或多个所述处理器还用于：对每个所述第一模糊核执行多次变换，所述多次变换的方式相同或不同。8 . The generating apparatus according to claim 7 , wherein one or more of the processors are further configured to: perform multiple transformations on each of the first fuzzy kernels, and the multiple transformations are performed in the same manner. 9 . or different. 9.根据权利要求6所述的生成装置，其特征在于，一个或多个所述处理器还用于：9. The generating apparatus of claim 6, wherein one or more of the processors are further configured to: 处理所述第一图像以获取至少一个第一子图像，所述第一子图像中包括待模糊主体；processing the first image to obtain at least one first sub-image, the first sub-image includes the subject to be blurred; 利用所述第一模糊核或所述第二模糊核对所述第一子图像进行卷积操作，以获取第二子图像；及Convolving the first sub-image with the first blur kernel or the second blur kernel to obtain a second sub-image; and 融合所述第二子图像及所述第一图像以获取第二图像。The second sub-image and the first image are fused to obtain a second image. 10.根据权利要求9所述的生成装置，其特征在于，所述第一子图像包括多个，10. The generating apparatus according to claim 9, wherein the first sub-image comprises a plurality of, 不同的所述第一子图像采用不同的所述第一模糊核或所述第二模糊核进行卷积操作；或Different first sub-images use different first blur kernels or different second blur kernels to perform convolution operations; or 具有相同类别的待模糊主体的所述第一子图像采用相同的所述第一模糊核或所述第二模糊核进行卷积操作。The first sub-images with the same category of subjects to be blurred use the same first blur kernel or the same second blur kernel to perform convolution operations. 11.一种包含计算机程序的非易失性计算机可读存储介质，其特征在于，所述计算机程序被处理器执行时，使得所述处理器执行权利要求1至5任意一项所述的生成方法。11. A non-volatile computer-readable storage medium containing a computer program, wherein, when the computer program is executed by a processor, the processor causes the processor to perform the generation process described in any one of claims 1 to 5. method.
CN202110172289.5A 2021-02-08 2021-02-08 Image sample set generation method, image sample set generation device and non-volatile computer-readable storage medium Pending CN112950496A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110172289.5A CN112950496A (en) 2021-02-08 2021-02-08 Image sample set generation method, image sample set generation device and non-volatile computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110172289.5A CN112950496A (en) 2021-02-08 2021-02-08 Image sample set generation method, image sample set generation device and non-volatile computer-readable storage medium

Publications (1)

Publication Number Publication Date
CN112950496A true CN112950496A (en) 2021-06-11

Family

ID=76244173

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110172289.5A Pending CN112950496A (en) 2021-02-08 2021-02-08 Image sample set generation method, image sample set generation device and non-volatile computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN112950496A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170013248A1 (en) * 2015-07-07 2017-01-12 Canon Kabushiki Kaisha Image generating apparatus, image generating method, and image generating program
CN109727201A (en) * 2017-10-30 2019-05-07 富士通株式会社 Information processing apparatus, image processing method, and storage medium
CN110189251A (en) * 2019-06-04 2019-08-30 厦门美图之家科技有限公司 A kind of blurred picture generation method and device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170013248A1 (en) * 2015-07-07 2017-01-12 Canon Kabushiki Kaisha Image generating apparatus, image generating method, and image generating program
CN109727201A (en) * 2017-10-30 2019-05-07 富士通株式会社 Information processing apparatus, image processing method, and storage medium
CN110189251A (en) * 2019-06-04 2019-08-30 厦门美图之家科技有限公司 A kind of blurred picture generation method and device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
毛勇: ""模糊车牌图像的盲去模糊算法研究"", 《中国优秀硕士学位论文全文数据库（信息科技辑）》, pages 31 *
胡绍海 等: ""基于区域检测分割的运动模糊图像复原"", 《北京交通大学学报》, vol. 43, no. 05, pages 20 - 26 *
马少勇: ""基于生成式对抗网络的运动模糊图像复原研究"", 《中国优秀硕士学位论文全文数据库（信息科技辑）》, pages 6 - 9 *

Similar Documents

Publication Publication Date Title
US12148123B2 (en) Multi-stage multi-reference bootstrapping for video super-resolution
Zhang et al. Deep image deblurring: A survey
Bao et al. Memc-net: Motion estimation and motion compensation driven neural network for video interpolation and enhancement
US12008797B2 (en) Image segmentation method and image processing apparatus
Li et al. Efficient and interpretable deep blind image deblurring via algorithm unrolling
Wang et al. Real-esrgan: Training real-world blind super-resolution with pure synthetic data
Cai et al. Toward real-world single image super-resolution: A new benchmark and a new model
Wang et al. Deeplens: Shallow depth of field from a single image
US12026857B2 (en) Automatically removing moving objects from video streams
CN106127689B (en) Image and video super-resolution method and device
CN112837245B (en) Dynamic scene deblurring method based on multi-mode fusion
Zhang et al. MC-Blur: A comprehensive benchmark for image deblurring
WO2020215644A1 (en) Video image processing method and apparatus
Dai et al. Sparse representation-based multiple frame video super-resolution
Dutta Depth-aware blending of smoothed images for bokeh effect generation
Cheng et al. A dual camera system for high spatiotemporal resolution video acquisition
Guan et al. Srdgan: learning the noise prior for super resolution with dual generative adversarial networks
Su et al. Super-resolution without dense flow
Li et al. A maximum a posteriori estimation framework for robust high dynamic range video synthesis
Conde et al. Lens-to-lens bokeh effect transformation. NTIRE 2023 challenge report
An et al. Single-shot high dynamic range imaging via deep convolutional neural network
Sharif et al. DarkDeblur: Learning single-shot image deblurring in low-light condition
Yang et al. SRDN: A unified super-resolution and motion deblurring network for space image restoration
Sun et al. Dyblurf: Dynamic neural radiance fields from blurry monocular video
CN106846250A (en) A kind of super resolution ratio reconstruction method based on multi-scale filtering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210611