WO2023155305A1 - 图像重建方法、装置、电子设备及存储介质 - Google Patents

图像重建方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023155305A1
WO2023155305A1 PCT/CN2022/090747 CN2022090747W WO2023155305A1 WO 2023155305 A1 WO2023155305 A1 WO 2023155305A1 CN 2022090747 W CN2022090747 W CN 2022090747W WO 2023155305 A1 WO2023155305 A1 WO 2023155305A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
blur kernel
processing
degraded
original image
Prior art date
Application number
PCT/CN2022/090747
Other languages
English (en)
French (fr)
Inventor
张楠
王健宗
瞿晓阳
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2023155305A1 publication Critical patent/WO2023155305A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the present application relates to the technical field of image processing, and in particular to an image reconstruction method, device, electronic equipment and storage medium.
  • the single image super resolution (Single Image Super Resolution, SISR) reconstruction technology is mainly based on the design of the software algorithm level, which can realize the transformation from low resolution (Low Resolution, LR) observation image to high resolution (High Resolution, HR) observation image improvement.
  • Low Resolution, LR Low Resolution
  • HR High Resolution, HR
  • convolutional neural networks are often used for single image super-resolution reconstruction, but image reconstruction in related technologies mostly focuses on learning a wider or deeper model, which often affects the image quality of the reconstructed image. Therefore, how to improve the image quality of the reconstructed image has become an urgent technical problem to be solved.
  • an image reconstruction method including:
  • the resolution of the reconstructed image is greater than that of the original image.
  • the embodiment of the present application proposes an image reconstruction device, the device includes:
  • An image acquisition module used to acquire the original image
  • An image processing module the image processing module is specifically used to: use a preset image degradation model to perform degradation processing on the original image, and determine a target blur kernel that matches the original image;
  • the embodiment of the present application provides an electronic device, the electronic device includes a memory and a processor, wherein the memory stores a program, and when the program is executed by the processor, the processor is used to perform an image reconstruction method, wherein the Described image reconstruction method comprises:
  • the resolution of the reconstructed image is greater than that of the original image.
  • the embodiment of the present application provides a computer-readable storage medium, which stores a computer program readable by the computer, and when the computer program is executed by a computer, the computer is used to perform an image reconstruction method, wherein,
  • the image reconstruction method includes:
  • the resolution of the reconstructed image is greater than that of the original image.
  • the image reconstruction method, device, electronic device, and storage medium proposed in the embodiments of the present application obtain an original image, perform degradation processing on the original image using a preset image degradation model, and determine a target blur kernel that matches the original image. Furthermore, the variable dimension processing is performed on the target blur kernel to generate a degraded image, and the original image and the degraded image are input into the pre-trained image restoration model for processing to obtain a reconstructed image.
  • this application determines the target blur kernel that matches the original image, so that the target blur kernel is closer to the real blur kernel, which can reduce the influence of the unknown blur kernel on image reconstruction, and realize the image reconstruction with any blur kernel. The image is reconstructed.
  • this application uses the degraded image generated according to the target blur kernel together with the original image as the input of the trained image restoration model, so that in the process of the image restoration model processing the original image and the degraded image, the modules in the image restoration model can
  • the image information of the degraded image is fused to improve the image quality of the reconstructed image.
  • Fig. 1 is a flow chart of the image reconstruction method provided by the embodiment of the present application.
  • Fig. 2 is the flowchart of step S102 in Fig. 1;
  • Fig. 3 is the flowchart of step S103 in Fig. 1;
  • FIG. 4 is a schematic diagram of an image reconstruction method provided by an embodiment of the present application applied to a specific application scenario
  • Fig. 5 is a schematic diagram of residual dense blocks in the image restoration model of the image reconstruction method provided by the embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of an image reconstruction device provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.
  • Artificial Intelligence It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science. Intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a manner similar to human intelligence. Research in this field includes robotics, language recognition, image recognition, natural language processing, and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
  • Natural language processing uses computers to process, understand and use human languages (such as Chinese, English, etc.). NLP belongs to a branch of artificial intelligence and is an interdisciplinary subject between computer science and linguistics. Known as computational linguistics. Natural language processing includes syntax analysis, semantic analysis, text understanding, etc. Natural language processing is often used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information retrieval, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves language processing Related data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research and linguistics research related to language computing, etc.
  • Image restoration is one of the most important tasks in image processing, including image denoising, deblurring, image restoration, super-resolution, etc., which are widely studied issues in the underlying vision. In practice, the images we get are often degraded images (such as noisy images, blurred images, sampled images, etc.
  • Natural image prior With the help of different natural image prior information, different original images can be estimated.
  • the commonly used prior information of natural images includes local smoothness, non-local self-similarity, sparsity and other characteristics of natural images;
  • Non-local similarity There are similar textures in different positions of natural images, and the textures of many natural images themselves have regularity. This shows that the information of the natural image itself is redundant, and we can use the redundant information of the image to repair the missing or polluted part of the image.
  • To use the non-local similarity of the image we must first find the similar texture in the image.
  • the most commonly used method is block matching, that is, the image is decomposed into small blocks one by one, and each small block is regarded as a unit, and similar textures are found in the image. One or more small blocks of .
  • Sparsity itself means that the number of non-zero elements in a matrix or vector is very small. For natural images, it can be represented by a small number of independent components. That is, images can become sparse signals through some linear changes. Image sparsity is a prerequisite for images to be recoverable with compressive sensing methods.
  • the process of compressive sensing for image restoration is as follows: the image can be transformed into a sparse vector S after linear basis transformation ⁇ . Random sampling of the original image yields the observation vector y. The original image can be recovered by using the observation vector y and the restoration matrix ⁇ (often a redundant dictionary).
  • Statistical characteristics are statistical laws obtained by learning a large number of images. This kind of characteristic is relatively abstract. Generally, the probability distribution modeling is carried out on the image, and the statistical characteristics are integrated into the parameters of the solution of the probability model.
  • EPLL prior Exected Patch Log LIkelihood
  • Deep learning methods based on supervised models also use neural networks to self-learn statistical properties in natural images.
  • Image Super resolution refers to recovering a high resolution image from a low resolution image or image sequence.
  • Image super-resolution technology is divided into super-resolution restoration and super-resolution reconstruction.
  • Super-resolution is to improve the resolution of the original image through hardware or software methods, and the process of obtaining a high-resolution image through a series of low-resolution images is super-resolution reconstruction.
  • the core idea of super-resolution reconstruction is to trade temporal bandwidth (acquiring multi-frame image sequences of the same scene) for spatial resolution, and realize the conversion from temporal resolution to spatial resolution.
  • Image degradation In the process of image formation, recording, processing and transmission, due to the imperfection of imaging system, recording equipment, transmission medium and processing method, the image quality will decline. This phenomenon is called image degradation.
  • Image degradation model After the input image f(x,y) passes through a degraded system, the output is a degraded image.
  • the original image f(x, y) is subjected to a degraded operator or a degraded system H(x, y), and then superimposed with noise n(x, y) to form a degraded image g(x, y).
  • Figure 1 shows the relationship between the input and output of the degradation process, where H(x, y) summarizes the physical process of the degradation system, which is the required mathematical model of degradation.
  • the degradation models mainly include: nonlinear degradation, fuzzy degradation, motion degradation and random noise degradation.
  • Blur kernel refers to the use of kernels and masks when blurring images (or smooth images).
  • the blur kernel is actually a matrix. After the clear image is convolved with the blur kernel, the image becomes blurred, so it is called the blur kernel.
  • the blur kernel is a type of convolution kernel. The essence of image convolution operation is matrix convolution.
  • Gaussian white noise If a noise whose instantaneous value obeys a Gaussian distribution and its power spectral density is evenly distributed, it is called Gaussian white noise.
  • the spectrum of thermal noise is uniformly distributed within the operating frequency range of general communication systems, just like the spectrum of white light is uniformly distributed within the spectrum of visible light, so thermal noise is often called white noise. Because thermal noise is generated by the movement of a large number of free electrons, and its statistical characteristics obey the Gaussian distribution, thermal noise is called Gaussian white noise.
  • the power spectral density of Gaussian white noise obeys a uniform distribution, and the amplitude distribution obeys a Gaussian distribution. Gaussian white noise is not only uncorrelated but also statistically independent between any two random variables at different times.
  • Image upsampling refers to enlarging the image, also known as image interpolation (interpolating), its main purpose is to enlarge the original image, so that the image can be displayed on a higher resolution display device.
  • Upsampling principle Image enlargement almost always adopts the interpolation method, that is, inserts new elements between pixels using a suitable interpolation algorithm on the basis of the original image pixels.
  • Interpolation algorithms mainly include edge-based image interpolation algorithms and region-based image interpolation algorithms.
  • Image downsampling refers to shrinking an image, also known as downsampling (downsampled), its main purpose is to make the image conform to the size of the display area and generate a thumbnail of the corresponding image.
  • Downsampling principle For an image whose I size is M*N, it is downsampled by s times to obtain a resolution image of (M/s)*(N/s) size, of course s should be M and N If the image in matrix form is considered, the image in the original image s*s window is converted into a pixel, and the value of this pixel is the average value of all pixels in the window.
  • HQS Half Quadratic Splitting
  • FFT Fast Fourier transform
  • DFT discrete Fourier transform
  • the basic idea of FFT is to decompose the original N-point sequence into a series of short sequences in turn. By making full use of the symmetric and periodic properties of the exponential factor in the DFT calculation formula, the corresponding DFTs of these short sequences are obtained and combined appropriately, so as to achieve the purpose of deleting repeated calculations, reducing multiplication operations and simplifying the structure.
  • the fast method for computing the discrete Fourier transform includes the FFT algorithm decimated by time and the FFT algorithm decimated by frequency.
  • the former sorts the time-domain signal sequences according to even-odd
  • the latter sorts the frequency-domain signal sequences according to even-odd. They all rely on two characteristics: one is periodicity; the other is symmetry, where the symbol * represents its conjugate. In this way, the calculation of the discrete Fourier transform can be divided into several steps, and the calculation efficiency is greatly improved.
  • MAP estimation can be calculated in the following ways: 1. Analytical method, which is used when the modulus of the posterior distribution can be expressed in an analytical solution; 2. By numerical optimization, such as the conjugate gradient method or Newton's method. This usually requires a first or second derivative, which must be evaluated analytically or numerically; 3. by a modification of the expectation-maximization algorithm, which does not require the derivative of the posterior density; 4. by Monte Carlo using simulated annealing Luo method.
  • PCA Principal Component Analysis
  • PCA is essentially a base transformation that maximizes the variance of the transformed data, that is, through the rotation of the coordinate axes and the translation of the origin of the coordinates, the variance between one of the axes (principal axes) and the data points is minimized , the orthogonal axis with high variance is removed after coordinate transformation, and the dimensionality reduction data set is obtained.
  • AI artificial intelligence
  • the embodiments of the present application may acquire and process relevant data based on artificial intelligence technology.
  • artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. .
  • Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics.
  • Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning/deep learning.
  • the embodiments of the present application provide an image reconstruction method, device, electronic equipment, and storage medium, so as to reconstruct an image with an arbitrary blur kernel and improve the image quality of the reconstructed image.
  • the image reconstruction method, device, electronic device, and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the image reconstruction method in the embodiments of the present application is described.
  • the image reconstruction method provided in the embodiment of the present application relates to the technical field of image processing.
  • the image reconstruction method provided in the embodiment of the present application may be applied to a terminal, may also be applied to a server, and may also be software running on the terminal or the server.
  • the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.
  • the server end can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or It can be configured as a cloud that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
  • the server; the software can be an application for realizing the medical image comparison method, etc., but is not limited to the above forms.
  • Fig. 1 is an optional flow chart of the image reconstruction method provided by the embodiment of the present application.
  • the method in Fig. 1 may include but not limited to steps S101 to S104:
  • Step S101 acquiring an original image
  • Step S102 using a preset image degradation model to degrade the original image, and determine a target blur kernel that matches the original image;
  • Step S103 performing variable dimension processing on the target blur kernel to generate a degraded image
  • Step S104 inputting the original image and the degraded image into a pre-trained image restoration model for processing to obtain a reconstructed image, wherein the resolution of the reconstructed image is greater than that of the original image.
  • the above-mentioned original image may be a low-resolution image, and image reconstruction can be performed on this low-resolution image through the above-mentioned steps S101 to S104, so as to improve the image resolution of the original image.
  • the degraded image generated according to the target blur kernel is also used as a part of the input, and the original image and the degraded image are processed by the trained image restoration model.
  • the images are processed together, so that the image information of the degraded image is also fused in the trained image restoration model.
  • the image reconstruction method before performing step 101 of the embodiment of the present application, the image reconstruction method further includes:
  • the image restoration model is trained to obtain a trained image restoration model.
  • the image restoration model can be a deep residual convolution model using a residual dense block (RDB) as a basic building block.
  • the image restoration model is trained according to the training image set in the training set, wherein the training set includes the input image, the standard image corresponding to the input image, and the degraded reference image determined according to the input image, wherein the input image is degraded to the standard image image obtained after.
  • RDB residual dense block
  • the aforementioned input image may include multiple images, and each image is an image obtained by degrading a corresponding standard image.
  • This input image may be referred to as a low resolution image, and the standard image may be referred to as a high resolution image.
  • the degraded reference image is determined from the blur kernel matching the input image.
  • the dimension of the degraded reference image is the same as that of the input image. Specifically, it may mean that the width of the degraded image is the same as that of the original image, and the height of the degraded image is the same as that of the original image.
  • this SISR degradation model can be used as a preset image degradation model, and the original image can be degraded by using the SISR degradation model processing to determine the target blur kernel that matches the original image.
  • the representation of the SISR degradation model is shown in formula (1):
  • LR means low-resolution image
  • HR means high-resolution image
  • k means blur kernel
  • n means Gaussian white noise
  • * means convolution operation
  • ⁇ means down-sampling operation
  • s means down-sampling multiple.
  • step S102 may include, but is not limited to, step S201 to step S203:
  • Step S201 the original image is multiplied by the intermediate blur kernel and subjected to down-sampling processing to obtain the first image;
  • Step S202 adding noise to the first image to obtain a second image
  • Step S203 determining the target blur kernel according to the pixel value difference between the first image and the second image
  • the intermediate blur kernel is the blur kernel set in the process of determining the target blur kernel.
  • the intermediate blur kernel corresponding to the case where the pixel value difference between the first image and the second image satisfies a preset condition may be determined as the target blur kernel.
  • multiple intermediate blur kernels may be set until the difference between the pixel values of the first image and the second image meets the preset condition, and the corresponding intermediate blur kernel at this time can be determined as the target blur kernel.
  • the pixel value difference between the first image and the second image satisfies the preset condition may specifically mean that the difference between the pixel values of the first image and the second image is less than or equal to a first preset threshold, and the first preset threshold may be preset value.
  • the first image can be expressed as (x*k) ⁇ s
  • the target blur kernel is determined according to the pixel value difference between the first image and the second image.
  • the intermediate blur kernel corresponding to the case where the pixel value difference between the first image and the second image is less than or equal to the first preset threshold may be determined as the target blur kernel.
  • the maximum a posteriori probability estimation of the fusion data item and the prior item is calculated, and the difference between the pixel values of the first image and the second image satisfies the formula (2).
  • the blur kernel is determined as the target blur kernel k.
  • the argmin function refers to the combined formula k and x when the minimum value is reached, the data item is The a priori term is and is the regular term, and ⁇ and ⁇ are the regular term parameters. It should be noted, Represents the gradient map of x.
  • formula (2) is solved by half quadratic splitting method (HQS), and the k and The iterative solution of k and The iterative solutions of are represented by formula (3) and formula (4):
  • F and F -1 represent FFT and inverse FFT, and Represents the gradient transformation in the horizontal and vertical directions, respectively.
  • the target blur kernel matching the original image can be determined more conveniently, so that the target blur kernel is closer to the real blur kernel, and the influence of the unknown blur kernel on image reconstruction can be reduced, and the image with arbitrary blur kernel can be realized. to rebuild.
  • the intermediate blur kernel corresponding to the case where the pixel value difference between the first image and the second image satisfies other preset conditions can also be determined as the target blur kernel;
  • the fuzzy kernel k is calculated and solved, but is not limited thereto.
  • step S103 may include, but is not limited to, performing dimensionality reduction and stretching on the target blur kernel, so that the degraded image has the same dimension as the original image.
  • the aforementioned dimensionality of the degraded image being the same as that of the original image may specifically mean that the width of the degraded image is the same as that of the original image, and the height of the degraded image is the same as that of the original image. That is to say, the same dimensions of the degraded image and the original image may mean that the width and height of each other are the same, and when the dimensions of the degraded image and the original image are the same, the number of channels of the degraded image and the original image may be the same or different.
  • performing dimensionality reduction processing and stretching processing on the target blur kernel may include but not limited to steps S301 to S303:
  • Step S301 converting the target blur kernel into a column vector
  • Step S302 performing dimensionality reduction processing on the column vector to obtain a dimensionality-reduced linear vector
  • Step S303 stretching the reduced linear vector to obtain a degraded image.
  • the size of the target blur kernel is W ⁇ H ⁇ C, where C is the number of channels.
  • the target blur kernel k is converted into a column vector, where the representation form of the column vector can be w 2 +1.
  • the dimensionality reduction process is performed on the column vector by the principal component analysis (PCA), for example, the column vector is reduced to the t dimension by the principal component analysis method, and the linear vector after dimension reduction is obtained. Furthermore, the linear vector after dimension reduction is stretched, and the linear vector is stretched into a degenerated image with a size of w ⁇ H ⁇ t.
  • PCA principal component analysis
  • the degraded image is in the same dimension as the original image, and in the same dimension, the width and height of the degraded image and the original image are the same, and the number of channels of the degraded image and the original image can be the same or different, without limitation.
  • the degraded image is in the same dimension as the original image, so that the degraded image and the original image can be connected, and the degraded image and the original image are input together into the trained image restoration in the subsequent step
  • the processing is carried out in the model, so that the trained image restoration model can fuse the image prior information of the degraded image in the process of image reconstruction, and improve the image quality of the reconstructed image.
  • the image restoration model includes at least one residual dense block (RDB) and at least one convolutional layer.
  • the degraded image generated according to the target blur kernel is used as the input of the trained image restoration model together with the original image.
  • the modules in the image restoration model can fuse the degraded image
  • the image information of the image is processed by the trained image restoration model on the original image LR and the degraded image P to obtain the reconstructed image SR, so that the image resolution of the reconstructed image SR is greater than that of the original image LR.
  • the image restoration model includes M residual dense blocks, and in the process of processing the original image and the degraded image by the image restoration model, the input information of any RDB module in the M residual dense blocks includes the degraded image, M is an integer greater than or equal to 1.
  • the input of some residual dense blocks among the M residual dense blocks may also include degradation information, which is not limited thereto.
  • the image restoration model includes N residual dense blocks.
  • N is an integer greater than or equal to 1.
  • the residual dense block contains densely connected layers and local feature fusion (LFF) with local residual learning (LRL).
  • LFF local feature fusion
  • LTL local residual learning
  • residual dense blocks also support continuous memory between residual dense blocks.
  • the output of a residual dense block can directly access any layer of the next residual dense block, so that the state can be continuously passed between any two layers in any residual dense block, and can fully extract the information of all layers.
  • Hierarchical features also facilitate the fusion of feature information.
  • Each convolutional layer of a residual dense block has access to all subsequent layers, passing on information that needs to be preserved.
  • GFF global feature fusion
  • the original image is obtained, and a preset image degradation model is used to degrade the original image, so as to determine a target blur kernel matching the original image. Furthermore, the variable dimension processing is performed on the target blur kernel to generate a degraded image, and the original image and the degraded image are input into the pre-trained image restoration model for processing to obtain a reconstructed image.
  • this application determines the target blur kernel that matches the original image, so that the target blur kernel is closer to the real blur kernel, which can reduce the influence of the unknown blur kernel on image reconstruction, and realize the image reconstruction with any blur kernel. image for image reconstruction.
  • this application uses the degraded image generated according to the target blur kernel together with the original image as the input of the trained image restoration model, so that in the process of the image restoration model processing the original image and the degraded image, the modules in the image restoration model can
  • the image information of the degraded image is fused to improve the image quality of the reconstructed image.
  • the embodiment of the present application also provides an image reconstruction device, which can realize the above image reconstruction method, and the device includes:
  • Image processing module 602 the image processing module is specifically used for:
  • the original image and the degraded image are input into a pre-trained image restoration model for processing to obtain a reconstructed image, wherein the resolution of the reconstructed image is greater than that of the original image.
  • the embodiment of the present application also provides an electronic device, the electronic device includes a memory, a processor, wherein a program is stored in the memory, and when the program is executed by the processor, an image reconstruction method is implemented, wherein the image reconstruction method includes: Acquire an original image; use a preset image degradation model to degrade the original image, determine a target blur kernel that matches the original image; perform variable dimension processing on the target blur kernel to generate a degraded image; convert the original image and inputting the degraded image into a pre-trained image restoration model for processing to obtain a reconstructed image, wherein the resolution of the reconstructed image is greater than that of the original image.
  • the electronic device may be any intelligent terminal including a tablet computer, a vehicle-mounted computer, and the like.
  • the electronic equipment includes:
  • the processor 701 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize The technical solutions provided by the embodiments of the present application;
  • a general-purpose CPU Central Processing Unit, central processing unit
  • a microprocessor an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize The technical solutions provided by the embodiments of the present application;
  • ASIC Application Specific Integrated Circuit
  • the memory 702 may be implemented in the form of a ROM (ReadOnly Memory, read only memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory, random access memory).
  • the memory 702 can store operating systems and other application programs.
  • the relevant program codes are stored in the memory 702 and called by the processor 701 to execute the implementation of the present application.
  • An example image reconstruction method is described in detail below.
  • the electronic equipment may also include:
  • the input/output interface 703 is used to realize information input and output
  • the communication interface 704 is used to realize the communication interaction between the device and other devices, and the communication can be realized through a wired method (such as USB, network cable, etc.), or can be realized through a wireless method (such as a mobile network, WIFI, Bluetooth, etc.); and
  • the processor 701 , the memory 702 , the input/output interface 703 and the communication interface 704 are connected to each other within the device through the bus 705 .
  • the embodiment of the present application also provides a computer-readable storage medium, which stores a computer program readable by the computer.
  • the computer program When the computer program is executed by the computer, the computer is used to perform an image reconstruction method, wherein the image reconstruction method includes: obtaining the original image; using a preset image degradation model to degrade the original image, and determine a target blur kernel that matches the original image; perform variable dimension processing on the target blur kernel to generate a degraded image; combine the original image and the The degraded image is input to the pre-trained image restoration model for processing to obtain a reconstructed image, wherein the resolution of the reconstructed image is greater than the resolution of the original image.
  • the computer-readable storage medium may be non-volatile or volatile.
  • memory can be used to store non-transitory software programs and non-transitory computer-executable programs.
  • the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage devices.
  • the memory optionally includes memory located remotely from the processor, and these remote memories may be connected to the processor via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • At least one (item) means one or more, and “multiple” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, “A and/or B” can mean: only A exists, only B exists, and A and B exist at the same time , where A and B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c ", where a, b, c can be single or multiple.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the above units is only a logical function division.
  • multiple units or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • an integrated unit is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including multiple instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method in each embodiment of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store programs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例提供了一种图像重建方法、装置、电子设备及存储介质，属于图像处理技术领域。该方法包括：获取原始图像；利用预设的图像退化模型对原始图像进行退化处理，确定与原始图像匹配的目标模糊核；对目标模糊核进行变维处理，生成退化图像；将原始图像和退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，重建图像的分辨率大于原始图像的分辨率。本申请实施例能够减小模糊核的未知对图像重建造成的影响，实现对具有任意模糊核的图像进行重建，提高重建图像的图像质量。

Description

图像重建方法、装置、电子设备及存储介质
本申请要求于2022年2月16日提交中国专利局、申请号为202210143565.X，发明名称为“图像重建方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种图像重建方法、装置、电子设备及存储介质。
背景技术
随着信息化技术的发展，人们对数字图像质量的要求也越来越高，特别是在医学、自动驾驶、天文以及监控等计算机视觉领域中，都需要获得高分辨率的、细节丰富的高清图像。然而，在实际图像采集时，往往受到成像系统自身的空间分辨率、光线或射线强度、空间距离以及系统噪声等因素的影响，而导致获得的图像的空间分辨率不高。
单图像超分辨率(Single Image Super Resolution，SISR)重建技术主要基于软件算法层面的设计，其可以实现从低分辨率(Low Resolution，LR)观测图像到高分辨率(High Resolution，HR)观测图像的提升。随着深度学习的发展和普及应用，在图像处理领域里也常常采用深度学习技术对超分辨率图像进行处理。
技术问题
以下是发明人意识到的现有技术的技术问题：
相关技术中，常常采用卷积神经网络进行单图像超分辨率重建，但相关技术中的图像重建大多侧重于学习更广泛或更深的模型，往往会影响重建图像的图像质量。因此，如何提高重建图像的图像质量，成为了亟待解决的技术问题。
技术解决方案
第一方面，本申请实施例提出了一种图像重建方法，包括：
获取原始图像；
利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
对所述目标模糊核进行变维处理，根据目标模糊核生成退化图像；
将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像。
其中，所述重建图像的分辨率大于所述原始图像的分辨率。
第二方面，本申请实施例提出了一种图像重建装置，所述装置包括：
图像获取模块，用于获取原始图像；
图像处理模块，图像处理模块具体用于：利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
对所述目标模糊核进行变维处理，生成退化图像；
将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。
第三方面，本申请实施例提出了一种电子设备，电子设备包括存储器和处理器，其中，存储器中存储有程序，程序被处理器执行时处理器用于执行一种图像重建方法，其中，所述 图像重建方法包括：
获取原始图像；
利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
对所述目标模糊核进行变维处理，根据目标模糊核生成退化图像；
将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像；
其中，所述重建图像的分辨率大于所述原始图像的分辨率。
第四方面，本申请实施例提出了一种计算机可读存储介质，计算机可读存储有计算机程序，在所述计算机程序被计算机执行时，所述计算机用于执行一种图像重建方法，其中，所述图像重建方法包括：
获取原始图像；
利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
对所述目标模糊核进行变维处理，根据目标模糊核生成退化图像；
将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像。
其中，所述重建图像的分辨率大于所述原始图像的分辨率。
有益效果
本申请实施例提出的图像重建方法、装置、电子设备及存储介质，其通过获取原始图像，利用预设的图像退化模型对原始图像进行退化处理，确定与原始图像匹配的目标模糊核。进而，对目标模糊核进行变维处理，生成退化图像，将原始图像和退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像。相较于相关技术，本申请通过确定与原始图像匹配的目标模糊核，使得目标模糊核更接近于真实模糊核，能够减小模糊核的未知对图像重建造成的影响，实现对具有任意模糊核的图像进行重建。另外，本申请将根据目标模糊核生成的退化图像与原始图像一起作为已训练的图像恢复模型的输入，使得在图像恢复模型对原始图像和退化图像处理的过程中，图像恢复模型内的模块能够融合退化图像的图像信息，提高重建图像的图像质量。
附图说明
附图用来提供对本申请技术方案的进一步理解，并且构成说明书的一部分，与本申请的实施例一起用于解释本申请的技术方案，并不构成对本申请技术方案的限制。
图1是本申请实施例提供的图像重建方法的流程图；
图2是图1中的步骤S102的流程图；
图3是图1中的步骤S103的流程图；
图4是本申请实施例提供的图像重建方法应用于一具体应用场景的示意图；
图5是本申请实施例提供的图像重建方法的图像恢复模型中的残差密集块的示意图；
图6是本申请实施例提供的图像重建装置的结构示意图；
图7是本申请实施例提供的电子设备的硬件结构示意图。
本发明的实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。
需要说明的是，虽然在装置示意图中进行了功能模块划分，在流程图中示出了逻辑顺序， 但是在某些情况下，可以以不同于装置中的模块划分，或流程图中的顺序执行所示出或描述的步骤。说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
除非另有定义，本申请所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本申请中所使用的术语只是为了描述本申请实施例的目的，不是旨在限制本申请。
首先，对本申请中涉及的若干名词进行解析：
人工智能(artificial intelligence，AI)：是研究、开发用于模拟、延伸和扩展人的智能的理论、方法、技术及应用系统的一门新的技术科学；人工智能是计算机科学的一个分支，人工智能企图了解智能的实质，并生产出一种新的能以人类智能相似的方式做出反应的智能机器，该领域的研究包括机器人、语言识别、图像识别、自然语言处理和专家系统等。人工智能可以对人的意识、思维的信息过程的模拟。人工智能还是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。
自然语言处理(natural language processing，NLP)：NLP用计算机来处理、理解以及运用人类语言(如中文、英文等)，NLP属于人工智能的一个分支，是计算机科学与语言学的交叉学科，又常被称为计算语言学。自然语言处理包括语法分析、语义分析、篇章理解等。自然语言处理常用于机器翻译、手写体和印刷体字符识别、语音识别及文语转换、信息检索、信息抽取与过滤、文本分类与聚类、舆情分析和观点挖掘等技术领域，它涉及与语言处理相关的数据挖掘、机器学习、知识获取、知识工程、人工智能研究和与语言计算相关的语言学研究等。
图像复原：图像复原是图像处理中最重要的任务之一，其包括图像去噪、去模糊、图像修复、超分辨等，都是底层视觉中被广泛研究的问题。实际中我们得到的图像往往是退化后的图像(如带噪声图像、模糊图像、被采样的图像等。
自然图像先验：借助于不同的自然图像先验信息，可以估计出不同的原始图像。常用的自然图像的先验信息有自然图像的局部平滑性、非局部自相似性、稀疏性等特征；
(1)局部平滑性：自然图像相邻像素点之间的像素值在一定程度上是连续变化的。从频谱上观察，自然图像以低频分量为主；从梯度直方图上观察，自然图像梯度统计趋近于0。
(2)非局部相似性：在自然图像的不同位置，存在相似的纹理，且许多自然图像自身的纹理存在规律性。这说明自然图像本身信息是冗余的，我们可以利用图像的冗余信息对图像缺失或被污染的部分进行修复。利用图像非局部相似性首先要找到图像中相似的纹理，最常用的方法是块匹配，即把图像分解成一个一个的小块，每个小块看作是一个单元，在图像中寻找与其相似的一个或多个小块。
(3)稀疏性：稀疏性本身是指矩阵或向量中非零元素个数很少。对于自然图像来说，就是其可以用少量的几个独立成分来表示。即图像可以通过某些线性变化变成稀疏信号。图像的稀疏性是图像可以用压缩感知方法进行恢复的先决条件。压缩感知进行图像恢复的过程为：图像经过线性基变换Ψ可以变成稀疏向量S。对原始图像进行随机采样可以得到观测向量y。利用观测向量y和恢复矩阵Θ(常常是冗余字典)可以恢复出原始图像。
(4)统计特性：统计特性是通过对大量图像进行学习得到的统计规律。这种特性比较抽象，一般对图像进行概率分布建模，将统计特性融合在概率模型的求解的参数里。一个比较常见的例子是EPLL先验(Expected Patch Log LIkelihood)，其使用混合高斯模型从大量自然图像块中学习到先验知识。基于监督模型的深度学习方法也是利用神经网络去自学习自然图像中的统计特性。
图像超分辨率(Image Super Resolution)：图像超分辨率是指由一幅低分辨率图像或图像序列恢复出高分辨率图像。图像超分辨率技术分为超分辨率复原和超分辨率重建。超分辨率(Super-Resolution)即通过硬件或软件的方法提高原有图像的分辨率，通过一系列低分 辨率的图像来得到一幅高分辨率的图像过程就是超分辨率重建。超分辨率重建的核心思想就是用时间带宽(获取同一场景的多帧图像序列)换取空间分辨率,实现时间分辨率向空间分辨率的转换。
图像退化：图像在形成、记录、处理和传输过程中，由于成像系统、记录设备、传输介质和处理方法的不完善，导致图像质量的下降，这种现象叫做图像退化。
图像退化模型：输入图像f(x,y)经过某个退化系统后输出的是一幅退化的图像。原始图像f(x,y)经过一个退化算子或退化系统H(x,y)的作用，再和噪声n(x,y)进行叠加，形成退化后的图像g(x,y)。图1表示退化过程的输入和输出的关系，其中H(x,y)概括了退化系统的物理过程，就是所需的退化数学模型。退化模型主要有：非线性退化、模糊退化、运动退化和随机噪声退化。
模糊核：指模糊图像(或者叫平滑图像)时使用内核、掩码。模糊核实际上就是一个矩阵，清晰图像与模糊核卷积后导致图像变得模糊，因此叫模糊核。模糊核是卷积核的一种。图像卷积操作的本质是矩阵卷积。
高斯白噪声(White Gaussian Noise)：如果一个噪声，它的瞬时值服从高斯分布，而它的功率谱密度又是均匀分布的，则称它为高斯白噪声。在一般的通信系统的工作频率范围内热噪声的频谱是均匀分布的，好像白光的频谱在可见光的频谱范围内均匀分布那样，所以热噪声又常称为白噪声。由于热噪声是由大量自由电子的运动产生的，其统计特性服从高斯分布，因而将热噪声称为高斯白噪声。高斯白噪声的功率谱密度服从均匀分布，幅度分布服从高斯分布。高斯白噪声在任意两个不同时刻上的随机变量之间，不仅是互不相关的，而且还是统计独立的。
图像上采样(upsampling)：图像上采样是指放大图像，也称图像插值(interpolating)，其主要目的在于放大原图像,从而使图像可以显示在更高分辨率的显示设备上。上采样原理：图像放大几乎都是采用内插值方法，即在原有图像像素的基础上在像素点之间采用合适的插值算法插入新的元素。插值算法主要包括基于边缘的图像插值算法和基于区域的图像插值算法。
图像下采样(subsampled)：图像下采样是指缩小图像，也称降采样(downsampled)，其主要目的在于使得图像符合显示区域的大小，以及生成对应图像的缩略图。下采样原理：对于一幅图像I尺寸为M*N，对其进行s倍下采样，即得到(M/s)*(N/s)尺寸的得分辨率图像，当然s应该是M和N的公约数才行，如果考虑的是矩阵形式的图像，就是把原始图像s*s窗口内的图像变成一个像素，这个像素点的值就是窗口内所有像素的均值。
半二次方分裂法(Half Quadratic Splitting；HQS)：HQS一般是将正则项中的原始变量进行变量替换，然后增加拉格朗日乘子项和二次惩罚项，使得在去耦合的同时，能够简化计算。
快速傅里叶变换(Fast Fourier transform；FFT)：FFT是利用计算机计算离散傅里叶变换(DFT)的高效、快速计算方法的统称。FFT的基本思想是把原始的N点序列，依次分解成一系列的短序列。充分利用DFT计算式中指数因子所具有的对称性质和周期性质，进而求出这些短序列相应的DFT并进行适当组合，达到删除重复计算，减少乘法运算和简化结构的目的。计算离散傅里叶变换的快速方法，有按时间抽取的FFT算法和按频率抽取的FFT算法。前者是将时域信号序列按偶奇分排，后者是将频域信号序列按偶奇分排。它们都借助于的两个特点：一是周期性；二是对称性，这里符号*代表其共轭。这样，便可以把离散傅里叶变换的计算分成若干步进行，计算效率大为提高。
最大后验概率估计(Maximum a posteriori estimation，MAP)：在贝叶斯统计学中，“最大后验概率估计”是后验概率分布的众数。利用最大后验概率估计可以获得对实验数据中无法直接观察到的量的点估计。它与最大似然估计中的经典方法有密切关系，但是它使用了一个增广的优化目标，进一步考虑了被估计量的先验概率分布。所以最大后验概率估计可以看作是规则化(regularization)的最大似然估计。MAP估计可以通过以下几种方式计算： 1、解析方法，当后验分布的模能够用解析解方式表示的时候用这种方法；2、通过数值优化，如共轭梯度法或牛顿法。这通常需要一阶或二阶导数，必须通过分析或数值方法进行评估；3、通过期望最大化算法的修改实现，这种方法不需要后验密度的导数；4、通过使用模拟退火的蒙特卡罗方法。
主成分分析法(Principal Component Analysis，PCA)：PCA是一种多变量统计方法，它是最常用的降维方法之一，通过正交变换将一组可能存在相关性的变量数据转换为一组线性不相关的变量，转换后的变量被称为主成分。可以使用特征分解或奇异值分解(SVD)这两种方法进行PCA。PCA将n维输入数据缩减为r维，其中r<n。简单地说，PCA实质上是一个基变换，使得变换后的数据有最大的方差，也就是通过对坐标轴的旋转和坐标原点的平移使得其中一个轴(主轴)与数据点之间的方差最小，坐标转换后去掉高方差的正交轴，得到降维数据集。
本申请实施例可以基于人工智能技术对相关的数据进行获取和处理。其中，人工智能(Artificial Intelligence，AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。
人工智能基础技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理技术、操作/交互系统、机电一体化等技术。人工智能软件技术主要包括计算机视觉技术、机器人技术、生物识别技术、语音处理技术、自然语言处理技术以及机器学习/深度学习等几大方向。
基于此，本申请实施例提供一种图像重建方法、装置、电子设备及存储介质，实现对具有任意模糊核的图像进行重建，提高重建图像的图像质量。
本申请实施例提供的图像重建方法、装置、电子设备及存储介质，具体通过如下实施例进行说明，首先描述本申请实施例中的图像重建方法。
本申请实施例提供的图像重建方法，涉及图像处理技术领域。本申请实施例提供的图像重建方法可应用于终端中，也可应用于服务器端中，还可以是运行于终端或服务器端中的软件。在一些实施例中，终端可以是智能手机、平板电脑、笔记本电脑、台式计算机等；服务器端可以配置成独立的物理服务器，也可以配置成多个物理服务器构成的服务器集群或者分布式系统，还可以配置成提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN以及大数据和人工智能平台等基础云计算服务的云服务器；软件可以是实现医疗图像对比方法的应用等，但并不局限于以上形式。
图1是本申请实施例提供的图像重建方法的一个可选的流程图，图1中的方法可以包括但不限于包括步骤S101至步骤S104：
步骤S101，获取原始图像；
步骤S102，利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
步骤S103，对所述目标模糊核进行变维处理，生成退化图像；
步骤S104，将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。
上述原始图像可以是低分辨率图像，通过上述步骤S101至步骤S104能够对这一低分辨率图像进行图像重建，以提高原始图像的图像分辨率。
具体地，由于在进行图像重建过程中，考虑到了模糊核对重建图像的图像质量的影响，将根据目标模糊核生成的退化图像也作为输入的一部分，通过已训练的图像恢复模型对原始图像和退化图像一起进行处理，使得已训练的图像恢复模型中也融合了退化图像的图像信息，经过以上步骤S101至步骤S104得到的重建图像，相较于原始图像以及通过相关技术进行图像重建得到的新图像，图像分辨率更高。
在一些实施例，在执行本申请实施例的步骤101之前，该图像重建方法还包括：
获取图像恢复模型；
对图像恢复模型进行训练得到已训练的图像恢复模型。
具体地，图像恢复模型可以是采用残差密集块(residual dense block，RDB)作为基本组成块的深度残差卷积模型。图像恢复模型根据训练集合中的训练图像集训练得到，其中，训练集合包括输入图像，输入图像对应的标准图像，以及根据输入图像确定的退化参考图像，其中，输入图像是对标准图像进行退化处理后得到的图像。
上述输入图像可以包括多个图像，每个图像是对对应的标准图像进行退化处理后得到的图像。该输入图像可以称为低分辨率图像，标准图像可以称为高分辨率图像。退化参考图像是根据与输入图像匹配的模糊核确定的。退化参考图像与输入图像的维度相同，具体可以是指退化图像的宽与原始图像的宽相同，退化图像的高与原始图像的高相同。
由于，利用常用的SISR退化模型可以方便地将高分辨率图像HR转化为低分辨图像LR，因此，可以将这一SISR退化模型作为预设的图像退化模型，利用SISR退化模型对原始图像进行退化处理，从而确定与原始图像匹配的目标模糊核。SISR退化模型的表示形式如公式(1)所示：
LR＝(HR*k)↓ s+n    公式(1)
其中，LR表示低分辨率图像，HR表示高分辨率图像，k表示模糊核，n表示高斯白噪声，*表示卷积运算，↓表示下采样操作，s表示下采样倍数。
具体地，请参阅图2，,步骤S102可以包括但不限于包括步骤S201至步骤S203：
步骤S201，将原始图像与中间模糊核相乘并经过下采样处理，得到第一图像；
步骤S202，将第一图像加上噪声，得到第二图像；
步骤S203，根据第一图像与第二图像的像素值差异确定目标模糊核；
其中，中间模糊核是确定目标模糊核过程中设置的模糊核。可以将第一图像与第二图像的像素值差异满足预设条件的情况下对应的中间模糊核确定为目标模糊核。
在确定目标模糊核的过程中可能会设置多个中间模糊核，直到第一图像和第二图像的像素值的差异满足预设条件，将此时对应的中间模糊核确定为目标模糊核即可。
而第一图像与第二图像的像素值差异满足预设条件具体可以是指，第一图像和第二图像的像素值的差异小于或者等于第一预设阈值，该第一预设阈值可以是预先设置的数值。
具体地，根据SISR退化模型，在本申请实施例中可以将第一图像表示为(x*k)↓ s,第二图像可以表示为y，y＝(x*k)↓ s+n,其中，x为原始图像，k为中间模糊核，s为下采样倍数，n为加上的噪声。进而根据第一图像与第二图像的像素值差异来确定目标模糊核。在一些具体实施例中，可以将第一图像与第二图像的像素值差异小于或者等于第一预设阈值的情况下对应的中间模糊核确定为目标模糊核。例如，根据图像的先验信息，在确定预设条件时，融合数据项与先验项的最大后验概率估计，将第一图像与第二图像的像素值差异满足公式(2)时的中间模糊核确定为目标模糊核k。
Figure PCTCN2022090747-appb-000001
其中，argmin函数是指使组合式
Figure PCTCN2022090747-appb-000002
达到最小值时的k与x，数据项为
Figure PCTCN2022090747-appb-000003
先验项为
Figure PCTCN2022090747-appb-000004
Figure PCTCN2022090747-appb-000005
和
Figure PCTCN2022090747-appb-000006
为正则项，γ和η为正则项参数。需要说明的是，
Figure PCTCN2022090747-appb-000007
表示x的梯度图。
进一步地，利用半二次方分裂法(HQS)对公式(2)进行求解，得到关于k和
Figure PCTCN2022090747-appb-000008
的迭代解，k和
Figure PCTCN2022090747-appb-000009
的迭代解分别以公式(3)和公式(4)表示：
Figure PCTCN2022090747-appb-000010
Figure PCTCN2022090747-appb-000011
其中，
Figure PCTCN2022090747-appb-000012
是y的梯度图，
Figure PCTCN2022090747-appb-000013
是通过
Figure PCTCN2022090747-appb-000014
估计得到的边缘图，
Figure PCTCN2022090747-appb-000015
是x的潜影。
进一步地，由于上述公式(3)和公式(4)可以看作是关于k和
Figure PCTCN2022090747-appb-000016
的交替最小化问题。因而，可以利用快速傅里叶变换法(FFT)对上述公式(3)进行求解，得到关于k的闭式表达 式，k的闭式表达式以公式(5)表示：
Figure PCTCN2022090747-appb-000017
其中，F和F -1表示FFT和逆FFT，
Figure PCTCN2022090747-appb-000018
和
Figure PCTCN2022090747-appb-000019
分别代表水平和垂直方向的梯度变换。
通过上述方式可以较为方便地确定与原始图像匹配的目标模糊核，使得目标模糊核更接近于真实模糊核，能够减小模糊核的未知对图像重建造成的影响，实现对具有任意模糊核的图像进行重建。需要说明的是，在其他实施例，还可以将第一图像与第二图像的像素值差异满足其他预设条件的情况下对应的中间模糊核确定为目标模糊核；也可以采用其他方法对目标模糊核k进行计算求解，不限于此。
在一些实施例中，步骤S103可以包括但不限于包括对目标模糊核进行降维处理和拉伸处理，以使得退化图像与原始图像的维度相同。
上述退化图像和原始图像的维度相同具体可以是指退化图像的宽与原始图像的宽相同，退化图像的高与原始图像的高相同。也就是说，退化图像与原始图像的维度相同可以是指彼此的宽和高分别相同，而退化图像和原始图像的维度相同时，退化图像和原始图像的通道数可以相同也可以不同。
请参阅图3，在一些具体实施例中，对目标模糊核进行降维处理和拉伸处理可以包括但不限于包括步骤S301至步骤S303：
步骤S301，将目标模糊核转化为列向量；
步骤S302，对列向量进行降维处理，得到降维后的线性向量；
步骤S303，对降维后的线性向量进行拉伸处理，得到退化图像。
具体地，由于目标模糊核的尺寸为w×w，原始图像的尺寸为W×H×C，其中，C为通道数。首先将目标模糊核k转化为列向量，其中，列向量的表示形式可以为w 2+1。通过主成分分析法(PCA)对列向量进行降维处理，例如，通过主成分分析法将列向量降至t维，得到降维之后的线性向量。进而，对降维之后的线性向量进行拉伸处理，将线性向量拉伸成尺寸为w×H×t的退化图像。此时，退化图像与原始图像处于相同维度，在相同维度下，退化图像与原始图像的宽度和高度均相同，而退化图像与原始图像的通道数可以相同或者不同，并不做限制。通过对目标模糊核k进行变维处理，使得退化图像与原始图像处于相同维度，从而能够将退化图像和原始图像进行连接，在后续步骤中将退化图像和原始图像一起输入至已训练的图像恢复模型中进行处理，使得已训练的图像恢复模型在图像重建过程中能够融合退化图像的图像先验信息，提高重建图像的图像质量。
需要说明的是，除了上述的采用主成分分析法对列向量进行降维处理，还可以是采用其他方式对列向量进行降维处理，不限于此。
请参阅图4，在具体的应用场景中，图像恢复模型包括至少一个残差密集块(RDB)和至少一个卷积层。根据目标模糊核生成的退化图像与原始图像一起作为已训练的图像恢复模型的输入，在图像恢复模型对原始图像LR和退化图像P处理的过程中，使得图像恢复模型内的模块能够融合退化图像的图像信息，通过已训练的图像恢复模型对原始图像LR和退化图像P处理，得到重建图像SR，使得重建图像SR的图像分辨率大于原始图像LR。
具体地，图像恢复模型包括M个残差密集块，在图像恢复模型对原始图像和退化图像处理的过程中，M个残差密集块的中的任意一个RDB模块的输入信息均包括退化图像，M为大于或者等于1的整数。在一些其他实施例中，也可以是M个残差密集块中的部分残差密集块的输入包括退化信息，不限于此。
进一步地，为了提高图像处理质量，也可以是在输入退化图像之前，对退化图像进行卷积处理，再将经过卷积处理之后的退化图像和原始图像一起输入到已训练的图像恢复模型中，以提高重建图像的图像质量。
请参阅图5，在一些实施中，图像恢复模型包括N个残差密集块，在训练得到图像恢复模型的过程中，N个残差密集块中的任意一个残差密集块中的任意两层处于可连接状态，N为 大于或者等于1的整数。
具体地，通过残差密集块(RDB)可以充分利用原始图像和退化图像的所有分层特征。需要说明的是，残差密集块包含密集连通层和带有局部残差学习(LRL)的局部特征融合(LFF)。另外，残差密集块还支持残差密集块之间的连续记忆。例如，一个残差密集块的输出可以直接访问下一个残差密集块的任意层，从而使状态连续传递任意一个残差密集块中的任意两层之间可以直接连接，能够充分提取所有层的层次特征，也方便特征信息之间的融合。残差密集块的每个卷积层都可以访问所有的后续层，传递需要保留的信息。将前面的残差密集块与当前残差密集块的所有前面层的状态连接，局部特征融合通过自适应地保存信息来提取局部密集特征。此外，局部特征融合还通过稳定更大网络的训练来实现极高的增长率。在提取多层局部密集特征之后，可以进行全局特征融合(GFF)以全局方式自适应地保留分层特征。
本申请实施例通过获取原始图像，利用预设的图像退化模型对原始图像进行退化处理，确定与原始图像匹配的目标模糊核。进而，对目标模糊核进行变维处理，生成退化图像，将原始图像和退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像。相较于相关技术，本申请通过确定与原始图像匹配的目标模糊核，使得目标模糊核更接近于真实模糊核，能够减小模糊核的未知对图像重建造成的影响，实现对具有任意模糊核的图像进行图像重建。另外，本申请将根据目标模糊核生成的退化图像与原始图像一起作为已训练的图像恢复模型的输入，使得在图像恢复模型对原始图像和退化图像处理的过程中，图像恢复模型内的模块能够融合退化图像的图像信息，提高重建图像的图像质量。
请参阅图6，本申请实施例还提供一种图像重建装置，可以实现上述图像重建方法，该装置包括：
图像获取模块601，用于获取原始图像；
图像处理模块602，所述图像处理模块具体用于：
利用预设的图像退化模型对原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
对目标模糊核进行变维处理，生成退化图像；
将所述原始图像和退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。
本申请实施例还提供了一种电子设备，电子设备包括存储器、处理器、其中，存储器中存储有程序，程序被处理器执行时实现一种图像重建方法，其中，所述图像重建方法包括：获取原始图像；利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；对所述目标模糊核进行变维处理，生成退化图像；将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。该电子设备可以为包括平板电脑、车载电脑等任意智能终端。
下面结合图7对电子设备的硬件结构进行详细说明。如图7所示，电子设备包括：
处理器701，可以采用通用的CPU(CentralProcessingUnit，中央处理器)、微处理器、应用专用集成电路(ApplicationSpecificIntegratedCircuit，ASIC)、或者一个或多个集成电路等方式实现，用于执行相关程序，以实现本申请实施例所提供的技术方案；
存储器702，可以采用ROM(ReadOnlyMemory，只读存储器)、静态存储设备、动态存储设备或者RAM(RandomAccessMemory，随机存取存储器)等形式实现。存储器702可以存储操作系统和其他应用程序，在通过软件或者固件来实现本说明书实施例所提供的技术方案时，相关的程序代码保存在存储器702中，并由处理器701来调用执行本申请实施例的图像重建方法。
该电子设备还可以包括：
输入/输出接口703，用于实现信息输入及输出；
通信接口704，用于实现本设备与其他设备的通信交互，可以通过有线方式(例如USB、网线等)实现通信，也可以通过无线方式(例如移动网络、WIFI、蓝牙等)实现通信；和
总线705，在设备的各个组件(例如处理器701、存储器702、输入/输出接口703和通信接口704)之间传输信息；
其中处理器701、存储器702、输入/输出接口703和通信接口704通过总线705实现彼此之间在设备内部的通信连接。
本申请实施例还提供了一种计算机可读存储介质，计算机可读存储有计算机程序，在计算机程序被计算机执行时，计算机用于执行一种图像重建方法，其中，图像重建方法包括：获取原始图像；利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；对所述目标模糊核进行变维处理，生成退化图像；将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。此外，所述计算机可读存储介质可以是非易失性，也可以是易失性。
存储器作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序以及非暂态性计算机可执行程序。此外，存储器可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施方式中，存储器可选包括相对于处理器远程设置的存储器，这些远程存储器可以通过网络连接至该处理器。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
本申请实施例描述的实施例是为了更加清楚的说明本申请实施例的技术方案，并不构成对于本申请实施例提供的技术方案的限定，本领域技术人员可知，随着技术的演变和新应用场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
本领域技术人员可以理解的是，图1-3中示出的技术方案并不构成对本申请实施例的限定，可以包括比图示更多或更少的步骤，或者组合某些步骤，或者不同的步骤。
以上所描述的装置实施例仅仅是示意性的，其中作为分离部件说明的单元可以是或者也可以不是物理上分开的，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、设备中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。
本申请的说明书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，上述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
上述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括多指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例的方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序的介质。
以上参照附图说明了本申请实施例的优选实施例，并非因此局限本申请实施例的权利范围。本领域技术人员不脱离本申请实施例的范围和实质内所作的任何修改、等同替换和改进，均应在本申请实施例的权利范围之内。

Claims (20)

  1. 一种图像重建方法，其中，包括：
    获取原始图像；
    利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
    对所述目标模糊核进行变维处理，生成退化图像；
    将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。
  2. 根据权利要求1所述的图像重建方法，其中，所述利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核，包括：
    将所述原始图像与中间模糊核相乘并经过下采样处理，得到第一图像；
    将所述第一图像加上噪声，得到第二图像；
    根据所述第一图像与所述第二图像的像素值差异确定所述目标模糊核；
    其中，所述中间模糊核是确定所述目标模糊核过程中设置的模糊核。
  3. 根据权利要求2所述的图像重建方法，其中，根据所述第一图像与所述第二图像的像素值差异确定所述目标模糊核，包括：
    将所述第一图像与所述第二图像的像素值差异满足预设条件的情况下对应的所述中间模糊核确定为所述目标模糊核。
  4. 根据权利要求1所述的图像重建方法，其中，所述对所述目标模糊核进行变维处理，生成退化图像，包括：
    对所述目标模糊核进行降维处理和拉伸处理，以使所述退化图像与所述原始图像的维度相同。
  5. 根据权利要求4所述的图像重建方法，其中，对所述目标模糊核进行降维处理和拉伸处理，包括：
    将所述目标模糊核转化为列向量；
    对所述列向量进行降维处理，得到降维后的线性向量；
    对所述降维后的线性向量进行拉伸处理，得到所述退化图像。
  6. 根据权利要求1所述的图像重建方法，其中，所述图像恢复模型包括N个残差密集块，在训练得到所述图像恢复模型的过程中，所述N个残差密集块中的任意一个残差密集块中的任意两层处于可连接状态，N为大于或者等于1的整数。
  7. 根据权利要求1所述的图像重建方法，其中，所述图像恢复模型包括M个残差密集块，在所述图像恢复模型对所述原始图像和所述退化图像处理的过程中，所述M个残差密集块的中的任意一个残差密集块的输入信息均包括所述退化图像，M为大于或者等于1的整数。
  8. 根据权利要求1所述的图像重建方法，其中，所述图像恢复模型是根据训练集合中的训练图像集训练得到的，其中，所述训练集合包括输入图像，所述输入图像对应的标准图像，以及根据所述输入图像确定的退化参考图像，其中，所述输入图像是对所述标准图像进行退化处理后得到的图像，所述输入图像的分辨率小于所述标准图像的分辨率，所述退化参考图像是根据与所述输入图像匹配的模糊核确定的，所述退化参考图像与所述输入图像的维度相同。
  9. 一种图像重建装置，其中，所述装置包括：
    图像获取模块，用于获取原始图像；
    图像处理模块，所述图像处理模块具体用于：利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
    对所述目标模糊核进行变维处理，生成退化图像；
    将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到 重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。
  10. 一种电子设备，其中，所述电子设备包括存储器和处理器，其中，所述存储器中存储有程序，所述程序被所述处理器执行时所述处理器用于执行一种图像重建方法，其中，所述图像重建方法包括：
    获取原始图像；
    利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
    对所述目标模糊核进行变维处理，生成退化图像；
    将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。
  11. 根据权利要求10所述的电子设备，其中，所述利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核，包括：
    将所述原始图像与中间模糊核相乘并经过下采样处理，得到第一图像；
    将所述第一图像加上噪声，得到第二图像；
    根据所述第一图像与所述第二图像的像素值差异确定所述目标模糊核；
    其中，所述中间模糊核是确定所述目标模糊核过程中设置的模糊核。
  12. 根据权利要求11所述的电子设备，其中，根据所述第一图像与所述第二图像的像素值差异确定所述目标模糊核，包括：
    将所述第一图像与所述第二图像的像素值差异满足预设条件的情况下对应的所述中间模糊核确定为所述目标模糊核。
  13. 根据权利要求10所述的电子设备，其中，所述对所述目标模糊核进行变维处理，生成退化图像，包括：
    对所述目标模糊核进行降维处理和拉伸处理，以使所述退化图像与所述原始图像的维度相同。
  14. 根据权利要求13所述的电子设备，其中，对所述目标模糊核进行降维处理和拉伸处理，包括：
    将所述目标模糊核转化为列向量；
    对所述列向量进行降维处理，得到降维后的线性向量；
    对所述降维后的线性向量进行拉伸处理，得到所述退化图像。
  15. 根据权利要求10所述的电子设备，其中，所述图像恢复模型包括N个残差密集块，在训练得到所述图像恢复模型的过程中，所述N个残差密集块中的任意一个残差密集块中的任意两层处于可连接状态，N为大于或者等于1的整数。
  16. 一种计算机可读存储介质，其中，所述计算机可读存储有计算机程序，在所述计算机程序被计算机执行时，所述计算机用于执行一种图像重建方法，其中，所述图像重建方法包括：：
    获取原始图像；
    利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核；
    对所述目标模糊核进行变维处理，生成退化图像；
    将所述原始图像和所述退化图像输入到预先训练得到的图像恢复模型中进行处理，得到重建图像，其中，所述重建图像的分辨率大于所述原始图像的分辨率。
  17. 根据权利要求16所述的计算机可读存储介质，其中，所述利用预设的图像退化模型对所述原始图像进行退化处理，确定与原始图像匹配的目标模糊核，包括：
    将所述原始图像与中间模糊核相乘并经过下采样处理，得到第一图像；
    将所述第一图像加上噪声，得到第二图像；
    根据所述第一图像与所述第二图像的像素值差异确定所述目标模糊核；
    其中，所述中间模糊核是确定所述目标模糊核过程中设置的模糊核。
  18. 根据权利要求17所述的计算机可读存储介质，其中，根据所述第一图像与所述第二图像的像素值差异确定所述目标模糊核，包括：
    将所述第一图像与所述第二图像的像素值差异满足预设条件的情况下对应的所述中间模糊核确定为所述目标模糊核。
  19. 根据权利要求16所述的计算机可读存储介质，其中，所述对所述目标模糊核进行变维处理，生成退化图像，包括：
    对所述目标模糊核进行降维处理和拉伸处理，以使所述退化图像与所述原始图像的维度相同。
  20. 根据权利要求19所述的计算机可读存储介质，其中，对所述目标模糊核进行降维处理和拉伸处理，包括：
    将所述目标模糊核转化为列向量；
    对所述列向量进行降维处理，得到降维后的线性向量；
    对所述降维后的线性向量进行拉伸处理，得到所述退化图像。
PCT/CN2022/090747 2022-02-16 2022-04-29 图像重建方法、装置、电子设备及存储介质 WO2023155305A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210143565.XA CN114494021A (zh) 2022-02-16 2022-02-16 图像重建方法、装置、电子设备及存储介质
CN202210143565.X 2022-02-16

Publications (1)

Publication Number Publication Date
WO2023155305A1 true WO2023155305A1 (zh) 2023-08-24

Family

ID=81482828

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/090747 WO2023155305A1 (zh) 2022-02-16 2022-04-29 图像重建方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN114494021A (zh)
WO (1) WO2023155305A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115937794B (zh) * 2023-03-08 2023-08-15 成都须弥云图建筑设计有限公司 小目标对象检测方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170205344A1 (en) * 2016-01-15 2017-07-20 The Mitre Corporation Active hyperspectral imaging system
CN112001866A (zh) * 2020-10-28 2020-11-27 季华实验室 多退化模型太赫兹图像复原方法、装置、存储介质和终端
CN112669214A (zh) * 2021-01-04 2021-04-16 东北大学 一种基于交替方向乘子算法的模糊图像超分辨率重建方法
CN113139904A (zh) * 2021-04-29 2021-07-20 厦门大学 一种图像盲超分辨率方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170205344A1 (en) * 2016-01-15 2017-07-20 The Mitre Corporation Active hyperspectral imaging system
CN112001866A (zh) * 2020-10-28 2020-11-27 季华实验室 多退化模型太赫兹图像复原方法、装置、存储介质和终端
CN112669214A (zh) * 2021-01-04 2021-04-16 东北大学 一种基于交替方向乘子算法的模糊图像超分辨率重建方法
CN113139904A (zh) * 2021-04-29 2021-07-20 厦门大学 一种图像盲超分辨率方法及系统

Also Published As

Publication number Publication date
CN114494021A (zh) 2022-05-13

Similar Documents

Publication Publication Date Title
Giraldo et al. Graph moving object segmentation
EP3298576B1 (en) Training a neural network
Zhang et al. Image super-resolution based on structure-modulated sparse representation
Ma et al. Low rank prior and total variation regularization for image deblurring
Ghorai et al. Multiple pyramids based image inpainting using local patch statistics and steering kernel feature
Ma et al. Sparse representation and position prior based face hallucination upon classified over-complete dictionaries
WO2020043296A1 (en) Device and method for separating a picture into foreground and background using deep learning
Thakur et al. PReLU and edge‐aware filter‐based image denoiser using convolutional neural network
Fanaee et al. Face image super-resolution via sparse representation and wavelet transform
Niu et al. Machine learning-based framework for saliency detection in distorted images
CN114331886A (zh) 一种基于深度特征的图像去模糊方法
Xie et al. Feature dimensionality reduction for example-based image super-resolution
Hu et al. Single-image superresolution based on local regression and nonlocal self-similarity
WO2023155305A1 (zh) 图像重建方法、装置、电子设备及存储介质
CN113129212B (zh) 图像超分辨率重建方法、装置、终端设备及存储介质
WO2017070841A1 (zh) 图像处理方法和装置
Jiang et al. Enhanced frequency fusion network with dynamic hash attention for image denoising
Li et al. Detail-enhanced image inpainting based on discrete wavelet transforms
Zou et al. Restoration of hyperspectral image contaminated by poisson noise using spectral unmixing
Hua et al. Dynamic scene deblurring with continuous cross-layer attention transmission
Bao et al. Half quadratic splitting method combined with convolution neural network for blind image deblurring
Chithra et al. 3D LiDAR point cloud image codec based on Tensor
Xia et al. Embedded conformal deep low-rank auto-encoder network for matrix recovery
Wei et al. Multi-focus image fusion based on nonsubsampled compactly supported shearlet transform
Tojo et al. Image Denoising Using Multi Scaling Aided Double Decker Convolutional Neural Network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22926618

Country of ref document: EP

Kind code of ref document: A1