WO2022247232A1 - Procédé et appareil d'amélioration d'image, dispositif terminal et support de stockage - Google Patents

Procédé et appareil d'amélioration d'image, dispositif terminal et support de stockage Download PDF

Info

Publication number
WO2022247232A1
WO2022247232A1 PCT/CN2021/137821 CN2021137821W WO2022247232A1 WO 2022247232 A1 WO2022247232 A1 WO 2022247232A1 CN 2021137821 W CN2021137821 W CN 2021137821W WO 2022247232 A1 WO2022247232 A1 WO 2022247232A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
network
processed
layer
image enhancement
Prior art date
Application number
PCT/CN2021/137821
Other languages
English (en)
Chinese (zh)
Inventor
陈翔宇
刘翼豪
章政文
乔宇
董超
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2022247232A1 publication Critical patent/WO2022247232A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the present application relates to the field of deep learning technology, and in particular to an image enhancement method, device, terminal equipment and storage medium.
  • Image enhancement tasks generally include operations such as dehazing, denoising, deraining, super-resolution, decompression artifacts, deblurring, and high dynamic range (High Dynamic Range, HDR) reconstruction.
  • HDR High Dynamic Range
  • the U-net structure of the network as a special convolutional neural network (Convolutional Neural Network Networks, referred to as CNN), although it can well extract the spatial features of different scales of the input image in the processing of image enhancement tasks.
  • CNN convolutional Neural Network Networks
  • the pure U-net structure network cannot specifically process the input features, so it is difficult to handle the features with large differences in the enhancement task at the same time, which affects the image enhancement effect, resulting in poor image quality in some areas of the enhanced image.
  • the present application provides an image enhancement method, device, terminal equipment, and storage medium, which can improve the quality of an enhanced image in an image enhancement task.
  • the present application provides an image enhancement method, including: acquiring an image to be processed; inputting the image to be processed into a trained image enhancement model for processing, outputting an enhanced image, the image enhancement model includes a main network and a conditional network, and the main network It is a U-net structure.
  • the image enhancement model includes a main network and a conditional network, and the main network It is a U-net structure.
  • the main network includes M downsampling layers and M upsampling layers
  • the conditional network includes a shared convolution layer and M+1 feature extraction modules
  • the M+1 feature extraction modules include different numbers of downsampling operations ; Extract multiple feature tensors of different scales from the image to be processed through the conditional network, including:
  • the intermediate features are extracted from the image to be processed through the shared convolutional layer; the intermediate features are respectively input into M+1 feature extraction modules for processing, and M+1 feature tensors of different scales are obtained.
  • the main network also includes a first SFT layer and a plurality of residual modules, and the residual module includes alternately arranged second SFT layers and convolutional layers;
  • the first SFT layer is connected to the input side of M downsampling layers and M
  • multiple residual modules are interspersed between M downsampling layers and M upsampling layers, and M+1 feature tensors of different scales are input to the first SFT layer and the second SFT layer respectively.
  • the SFT layer of the corresponding scale in the SFT layer In the SFT layer of the corresponding scale in the SFT layer.
  • the image enhancement model also includes a weight network, the weight network includes skip connections and multi-layer convolutional layers; the enhanced image is obtained by fusing the output of the main network with the original features, and the original features are obtained from the weight network extracted from the image to be processed.
  • the image to be processed is an LDR image
  • the enhanced image is an HDR image
  • the image enhancement model is obtained after training the preset image enhancement initial model using a preset loss function and a training set; wherein, the training set includes a plurality of LDR image samples and the HDR corresponding to each LDR image sample Image sample, the preset loss function is used to describe the L1 loss between the value obtained by the Tanh function of the HDR predicted image and the value obtained by the Tanh function of the HDR image sample, and the HDR predicted image is the initial model of the image enhancement An image is obtained after processing the LDR image sample.
  • an image enhancement device including:
  • the acquiring unit is used to acquire the image to be processed.
  • the processing unit is used to input the image to be processed into the trained image enhancement model for processing, and output the enhanced image.
  • the image enhancement model includes a main network and a conditional network.
  • the main network is a U-net structure.
  • the conditional network extracts multiple feature tensors of different scales from the image to be processed, and inputs the image to be processed and multiple feature tensors of different scales to the network layer of the corresponding scale in the main network for processing to obtain an enhanced image.
  • the image enhancement model also includes a weight network, and the weight network includes a skip connection and a multi-layer convolutional layer; the enhanced image is obtained by fusing the output of the main network with the original feature, and the original feature is obtained by weight
  • the network extracts from the image to be processed.
  • the present application provides a terminal device, including: a memory and a processor, where the memory is used to store a computer program; and the processor is used to execute the method described in any one of the above first aspects when calling the computer program.
  • the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the above-mentioned first aspects is implemented.
  • an embodiment of the present application provides a computer program product, which, when the computer program product runs on a processor, causes the processor to execute the method described in any one of the above-mentioned first aspects.
  • the image enhancement method, device, terminal equipment and storage medium provided by this application add a conditional network to the main network of the U-net structure, and use the conditional network to extract multiple feature tensors of different scales from the image to be processed and input them to the
  • the main network after the main network extracts the spatial features of different scales from the image to be processed, it can specialize the spatial features of different scales based on the multiple feature tensors of different scales, so as to retain the spatial features of different scales. effective information, thereby improving the quality of the enhanced image.
  • FIG. 1 is a network architecture diagram 1 of an image enhancement model provided by an embodiment of the present application.
  • Fig. 2 is a network architecture diagram 2 of an image enhancement model provided by the embodiment of the present application.
  • FIG. 3 is a schematic flowchart of a method for an HDR reconstruction task provided in an embodiment of the present application
  • FIG. 4 is a schematic diagram of a reconstruction effect of an HDR reconstruction task provided by an embodiment of the present application.
  • Fig. 5 is a schematic structural diagram of an image enhancement device provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the present application provides an image enhancement method. After the image to be processed is acquired, the image to be processed is input into the image enhancement model provided by the application for processing, and the enhanced image is output.
  • the image enhancement model provided by this application is to add a conditional network to the main network of the U-net structure, and use the conditional network to extract multiple feature tensors of different scales from the image to be processed as adjustment information and input them into the main network.
  • the main network After the main network extracts the spatial features of different scales from the image to be processed, it can specialize the spatial features of different scales based on the multiple feature tensors of different scales, so as to retain the effective information of the spatial features of different scales, so that Effectively improve the image enhancement effect and improve the quality of the enhanced image.
  • an exemplary image enhancement model provided by the present application is introduced with reference to FIG. 1 .
  • the image enhancement model is deployed in an image processing device, which may be a mobile terminal such as a smart phone, a tablet computer, or a camera, or a device capable of processing image data such as a desktop computer, a robot, or a server.
  • the image enhancement model provided by this application includes a main network and a condition network.
  • the main network adopts a U-net structure, which includes M downsampling layers and M upsampling layers connected to the M downsampling layers by jumping.
  • the U-net structure network performs image enhancement tasks, it gradually extracts spatial features of different scales through layer-by-layer down-sampling layers, and restores the spatial features of corresponding scales through layer-by-layer up-sampling layers to identify Enhanced information corresponding to the pixel to achieve image enhancement.
  • a conditional network is added to the main network of the U-net structure.
  • the conditional network can be used to extract multiple feature tensors of different scales from the image to be processed, and the multiple feature tensors of different scales are used as adjustment
  • the information is input into the main network, so that the main network can specialize the spatial features of different scales based on the feature tensors of different scales, so as to retain the effective information of the spatial features of different scales, thereby effectively improving the image enhancement effect. Improves the quality of enhanced images.
  • the conditional network can be designed based on the number of spatial features of different scales processed by the main network. For example, M downsampling layers of the main network mean that the main network can process spatial features of M+1 scales including the original scale. Then, correspondingly, the conditional network can output at most M+1 feature tensors of different scales corresponding to the spatial features of the M+1 scales.
  • the conditional network may include shared convolutional layers and M+1 feature extraction modules, and the M+1 feature extraction modules include different numbers of downsampling operation.
  • the conditional network first extracts intermediate features from the image to be processed by sharing a convolutional layer (for example, including multiple convolutional layers); then the intermediate features are respectively input into M+1 feature extraction modules for processing , so as to obtain M+1 feature tensors of different scales.
  • the main network includes 2 downsampling (Down) layers and corresponding 2 upsampling (Up) layers.
  • the spatial features processed by the main network include the original scale (large scale) spatial features extracted from the image to be processed, and the intermediate scale (smaller than the large scale) obtained after the first layer of downsampling layer is downsampled on the original scale. ) spatial features, the small-scale (smaller than the intermediate scale) spatial features obtained after the second downsampling layer performs downsampling on the intermediate scale.
  • the conditional network can include 3 feature extraction modules
  • the first feature extraction module can be composed of a convolution (Conv) layer (take 3 convolution layers as an example in Figure 1), which includes 0 downsampling operations
  • the scale of the output feature tensor 1 is the original scale, which is used to specifically process the spatial features of the original scale.
  • the second feature extraction module can be composed of a convolutional layer and a downsampling layer (take 2 convolutional layers and a downsampling layer as an example in Figure 1), that is, it includes a downsampling operation, and the output feature sheet
  • the scale of quantity 2 is the intermediate scale, which is used to specifically process the spatial features of the intermediate scale.
  • the third feature extraction module can be composed of a convolutional layer and 2 downsampling layers (take 1 convolutional layer and 2 downsampling layers as an example in Figure 1), which includes 2 downsampling operations, and the output feature sheet
  • the scale of quantity 3 is a small scale, which is used for specific processing of small-scale spatial features.
  • the spatial feature transform (Spatial Feature Transform, SFT) layer can be designed in the main network to make the feature tensors of different scales output by the conditional network act on the spatial features of the corresponding scale.
  • the main network further includes a first SFT layer and multiple residual modules.
  • the first SFT layer is connected to the input side of the M downsampling layers and the output side of the M upsampling layers.
  • the residual module includes alternately arranged second SFT layers and convolutional layers; multiple residual modules are interspersed between M downsampling layers and M upsampling layers, and M+1 feature tensors of different scales are respectively input To the SFT layer of the corresponding scale in the first SFT layer and the second SFT layer, to perform spatial feature transformation on the spatial features of different scales, and realize the specific processing of the spatial features of different scales.
  • the main network includes a convolutional layer, the first SFT layer (SFT layer1), a convolutional layer, a downsampling layer Down1, and two residual modules (Residual block), downsampling layer Down2, N (N is an integer greater than or equal to 1) residual modules, convolutional layer, upsampling layer Up1, two residual modules, upsampling layer Up2, SFT layer1, two convolutions Floor.
  • the residual module includes two sets of alternately arranged second SFT layers (ie, SFT layer2) and convolutional layers.
  • Feature tensor 1 is input to the first SFT layer SFT layer11 and SFT layer5, feature tensor 2 is input to SFT layer2 in the two residual modules between Down1 and Down2, and two residual modules between Up1 and Up2 SFT layer2 in difference module.
  • Feature tensor 3 is input to SFT layer2 in N residual modules located between Down2 and Up1.
  • the SFT layer can include two sets of convolutional layers (in Figure 1, each set of convolutional layers includes two convolutional layers as an example), and the feature tensor output by the conditional network is processed by a set of convolutional layers Get the modulation parameter a.
  • the modulation parameter a is multiplied by the output feature of the previous layer of the SFT layer to obtain the transformed feature.
  • the feature tensor is processed by another set of convolutional layers to obtain the modulation parameter b.
  • the modulation parameter b is added to the transform feature to obtain the output feature of the SFT layer.
  • the modulation parameters (a, b) are learned from the feature tensor output by the conditional network through the SFT layer, and then adaptive affine transformation is performed on the spatial features of the corresponding scale based on the modulation parameters (a, b), so as to realize Specific processing of different spatial features to retain more effective spatial information.
  • the image enhancement model of the present application may also include a weight network. That is to further increase the weight network on the main network.
  • the weight network is used to extract raw features from the image to be processed.
  • the weight network includes multiple convolutional layers (four convolutional layers are taken as an example in FIG. 2 ), and the input of the multi-layer convolutional layer is skip-connected to the output.
  • the enhanced image output by the image enhancement model is obtained by performing feature fusion on the output of the main network and the original features. Among them, feature fusion can be performed on the output of the main network and the original features by means of superposition.
  • the weight network By designing the weight network, the original features can be learned from the image to be processed without manual estimation, so that more and more accurate original features can be retained in the image enhancement task. Moreover, the weight network structure provided by this application is simple, easy to optimize, and can reduce the training difficulty of the image enhancement network under the condition that the original features are fully preserved.
  • the network framework provided by this application is universal. It can be applied to any image enhancement tasks or tasks that use image enhancement effects as evaluation indicators. Image defogging, denoising, deraining, super-resolution, decompression artifacts, deblurring, HDR reconstruction and other image enhancement tasks.
  • the initial model can be trained by designing corresponding training sets and loss functions, so as to obtain image enhancement models suitable for different image enhancement tasks.
  • the image enhancement initial model can be trained to obtain an image enhancement model that can be applied to super-resolution image enhancement tasks.
  • the initial image enhancement model is trained to obtain an image enhancement model that can be applied to the image enhancement task of deblurring.
  • the image enhancement model may be pre-trained by the image processing device, or the file corresponding to the image enhancement model may be transplanted to the image processing device after being pre-trained by other devices. That is to say, the execution subject for training the image enhancement model and the execution subject for performing the image enhancement task using the image enhancement model may be the same or different. For example, when other devices are used to train the image enhancement initial model, after the other devices complete the training of the image enhancement initial model, the model parameters are fixed to obtain the corresponding file of the image enhancement model. This file is then ported to an image processing device.
  • the following uses the HDR reconstruction task to illustrate the training process and effect of the image enhancement model provided by this application.
  • an image enhancement initial model is constructed first. That is, after building the U-net initial network, design the corresponding conditional initial network and weight initial network based on the initial main network, and add the conditional initial network and weight initial network to the initial main network.
  • the training set includes a plurality of image sample pairs, and each image sample pair includes an LDR image sample and an HDR image sample corresponding to the LDR image sample.
  • image sample pairs may be collected by a mobile phone, a camera, or the like. It is also possible to use an open source algorithm to obtain corresponding LDR image samples based on the HDR image samples to be made public.
  • the pixels in the generally highlighted area (ie, overexposed area) and the pixels in the non-highlighted area (ie, normally exposed area) have a large difference, therefore, it is easy to Causes the initial model to focus on areas with larger pixel values during training. That is to say, during the training process of the initial model, it may focus on restoring the brightness and texture details of the highlighted area, while ignoring the noise and quantization loss problems that may exist in the non-highlighted area.
  • the present application provides a loss function Tanh_L1, which is used to describe the L1 loss between the value obtained by the Tanh function for the HDR prediction image and the value obtained by the Tanh function for the HDR image sample. That is, the Tanh function is added on the basis of the L1 loss function, where the HDR prediction image is the image obtained after the initial image enhancement model processes the LDR image sample.
  • Tanh_L1 The expression of Tanh_L1 is as follows:
  • Y represents the HDR prediction image obtained by processing the LDR image sample through the image enhancement initial model
  • H represents the HDR image sample corresponding to the LDR image sample.
  • the Tanh function can perform non-linear compression on pixel values, so after being applied to the L1 loss function, it can balance the pixels in the highlighted area and the pixels in the non-highlighted area in the LDR image sample, and reduce the pixels due to the highlighted area and the non-highlighted area.
  • the pixel difference in the region is too large, which will affect the training effect of the initial model.
  • the gradient descent method can be used to iteratively train the image enhancement initial model.
  • the model converges (that is, the value of Tanh_L1 is not decreasing)
  • the trained image enhancement model can be obtained.
  • the image processing device when using the image enhancement model for HDR reconstruction.
  • the image processing device acquires the LDR image to be processed, it can input the LDR image to the image enhancement model, and the LDR image is respectively input to the weight network, the main network and the conditional network.
  • the original features are extracted from the LDR image by the weight network;
  • the feature tensors of different scales are extracted from the LDR image by the conditional network;
  • the feature tensors of different scales are input to the main network, and the main network is used for spatial features of different scales of the LDR image
  • the feature tensors of different scales are used to specifically process the spatial features of different scales to obtain the output features of the main network.
  • the output feature is fused with the original feature to obtain an HDR image.
  • the reconstruction effect of HDR reconstruction can be referred to as shown in FIG. 4 .
  • this application when using the image enhancement model provided by this application to perform HDR reconstruction tasks, firstly, it is not necessary to collect LDR images with different exposures in the same scene. After the model training is completed, only a single LDR image is needed to restore the corresponding HDR image. . Secondly, compared with the previous HDR reconstruction method based on a single LDR image, this application adopts an end-to-end training method, which has a simple model, high training efficiency, and good model effect.
  • the image enhancement model provided by this application enters the conditional network in the main network, the main network can perform specific processing on spatial features of different scales, and the Tanh_L1 loss function is used in the training process, so that the image enhancement provided by this application
  • the model is able to simultaneously process denoising and dequantization losses on non-highlight regions while restoring brightness and texture details in highlight regions. That is to say, in the HDR reconstruction task, the image enhancement model provided by this application can jointly realize the HDR reconstruction task, denoising task, and dequantization loss task.
  • the image enhancement model can be directly added to the camera's post-processing process to improve the camera's shooting quality from the perspective of software.
  • the image enhancement model can also be used as an image/video post-enhancement means to enhance the image quality of the existing LDR data.
  • the embodiment of the present application provides an image enhancement device, the embodiment of the device corresponds to the foregoing method embodiment, for the convenience of reading, the embodiment of the device no longer compares the foregoing method embodiment
  • the details in the present invention will be described one by one, but it should be clear that the device in this embodiment can correspondingly implement all the content in the foregoing method embodiments.
  • FIG. 5 is a schematic structural diagram of an image enhancement device provided by an embodiment of the present application.
  • the image enhancement device provided by this embodiment includes: an acquisition unit 501 and a processing unit 502 .
  • the obtaining unit 501 is configured to obtain an image to be processed.
  • the processing unit 502 is configured to input the image to be processed into a trained image enhancement model for processing, and output the enhanced image.
  • the image enhancement model includes a main network and a conditional network, the main network is a U-net structure, and the When processing the image to be processed, a plurality of feature tensors of different scales are extracted from the image to be processed through the conditional network, and the image to be processed and the feature tensors of multiple different scales are respectively input Go to the network layer of the corresponding scale in the main network for processing to obtain the enhanced image.
  • the main network includes M downsampling layers and M upsampling layers
  • the conditional network includes a shared convolution layer and M+1 feature extraction modules
  • the M+1 feature extraction modules include Different numbers of downsampling operations; extracting multiple feature tensors of different scales from the image to be processed through the conditional network, including:
  • Extract intermediate features from the image to be processed through the shared convolutional layer input the intermediate features to the M+1 feature extraction modules for processing, and obtain M+1 feature sheets of different scales quantity.
  • the main network further includes a first SFT layer and a plurality of residual modules, and the residual module includes alternately arranged second SFT layers and convolutional layers;
  • the first SFT layer is connected to the M The input side of the downsampling layer and the output side of the M upsampling layers, the plurality of residual modules are interspersed between the M downsampling layers and the M upsampling layers, M+1
  • the feature tensors of different scales are respectively input into SFT layers of corresponding scales in the first SFT layer and the second SFT layer.
  • the image enhancement model also includes a weight network, and the weight network includes a skip connection and a multi-layer convolution layer; the enhanced image is obtained by merging the output of the main network with the original features , the original feature is extracted from the image to be processed through the weight network.
  • the image to be processed is an LDR image
  • the enhanced image is an HDR image
  • the image enhancement model is obtained by using a preset loss function and a training set to train a preset image enhancement initial model; wherein, the training set includes a plurality of LDR image samples and each of the LDR image samples The HDR image sample corresponding to the sample, the preset loss function is used to describe the L1 loss between the value obtained by the Tanh function of the HDR predicted image and the value obtained by the Tanh function of the HDR image sample, the HDR predicted image is The image enhancement initial model obtains an image after processing the LDR image sample.
  • the image enhancement device provided in this embodiment can execute the above-mentioned method embodiment, and its implementation principle and technical effect are similar, and details are not repeated here.
  • the embodiment of the present application also provides a terminal device.
  • the terminal device 6 of this embodiment includes: a processor 60 , a memory 61 , and a computer program 62 stored in the memory 61 and operable on the processor 60 .
  • the processor 60 executes the computer program 62
  • the steps in the above embodiments of the image enhancement method are implemented, for example, steps S101 to S104 shown in FIG. 1 .
  • the processor 60 executes the computer program 62
  • the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the modules 401 to 403 shown in FIG. 4 are realized.
  • the computer program 62 can be divided into one or more modules/units, and the one or more modules/units are stored in the memory 61 and executed by the processor 60 to complete this application.
  • the one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program 62 in the terminal device 6 .
  • FIG. 6 is only an example of the terminal device 6, and does not constitute a limitation on the terminal device 6. It may include more or less components than those shown in the figure, or combine certain components, or different components.
  • the terminal device 6 may also include an input and output device, a network access device, a bus, and the like.
  • the processor 60 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the storage 61 may be an internal storage unit of the terminal device 6 , such as a hard disk or memory of the terminal device 6 .
  • the memory 61 can also be an external storage device of the terminal device 6, such as a plug-in hard disk equipped on the terminal device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Further, the memory 61 may also include both an internal storage unit of the terminal device 6 and an external storage device.
  • the memory 61 is used to store the computer program and other programs and data required by the terminal device 6 .
  • the memory 61 can also be used to temporarily store data that has been output or will be output.
  • the terminal device provided in this embodiment can execute the foregoing method embodiment, and its implementation principle and technical effect are similar, and details are not repeated here.
  • the embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in the foregoing method embodiment is implemented.
  • the embodiment of the present application further provides a computer program product, which, when the computer program product runs on a terminal device, enables the terminal device to implement the method described in the foregoing method embodiments when executed.
  • the above integrated units are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the procedures in the methods of the above embodiments in the present application can be completed by instructing related hardware through computer programs, and the computer programs can be stored in a computer-readable storage medium.
  • the computer program When executed by a processor, the steps in the above-mentioned various method embodiments can be realized.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form.
  • the computer-readable storage medium may at least include: any entity or device capable of carrying computer program codes to a photographing device/terminal device, a recording medium, a computer memory, a read-only memory (Read-Only Memory, ROM), a random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunication signals, and software distribution media.
  • a photographing device/terminal device a recording medium
  • a computer memory a read-only memory (Read-Only Memory, ROM), a random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunication signals, and software distribution media.
  • ROM read-only memory
  • RAM random access Memory
  • electrical carrier signals telecommunication signals
  • software distribution media such as U disk, mobile hard disk, magnetic disk or optical disk, etc.
  • computer readable media may not be electrical carrier signals and telecommunication signals under legislation and patent practice.
  • the disclosed device/device and method can be implemented in other ways.
  • the device/device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the term “if” may be construed, depending on the context, as “when” or “once” or “in response to determining” or “in response to detecting “.
  • the phrase “if determined” or “if [the described condition or event] is detected” may be construed, depending on the context, to mean “once determined” or “in response to the determination” or “once detected [the described condition or event] ]” or “in response to detection of [described condition or event]”.
  • references to "one embodiment” or “some embodiments” or the like in the specification of the present application means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application.
  • appearances of the phrases “in one embodiment,” “in some embodiments,” “in other embodiments,” “in other embodiments,” etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless specifically stated otherwise.
  • the terms “including”, “comprising”, “having” and variations thereof mean “including but not limited to”, unless specifically stated otherwise.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

La présente demande se rapporte au domaine technique de l'apprentissage profond et concerne un procédé et un appareil d'amélioration d'image, un dispositif terminal et un support de stockage, pouvant améliorer la qualité d'une image améliorée dans une tâche d'amélioration d'image. Le procédé d'amélioration d'image consiste : à acquérir une image à traiter; et à entrer ladite image dans un modèle d'amélioration d'image formé pour le traitement et à délivrer en sortie une image améliorée, le modèle d'amélioration d'image comprenant un réseau maître et un réseau conditionnel, le réseau maître étant d'une structure en forme de U et, lorsque ladite image est traitée, à extraire une pluralité de tenseurs de caractéristique de différentes échelles à partir de ladite image au moyen du réseau conditionnel et à entrer respectivement ladite image et la pluralité de tenseurs de caractéristique de différentes échelles dans des couches de réseau d'échelles correspondantes dans le réseau maître pour le traitement pour obtenir l'image améliorée.
PCT/CN2021/137821 2021-05-27 2021-12-14 Procédé et appareil d'amélioration d'image, dispositif terminal et support de stockage WO2022247232A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110584556.XA CN113298740A (zh) 2021-05-27 2021-05-27 一种图像增强方法、装置、终端设备及存储介质
CN202110584556.X 2021-05-27

Publications (1)

Publication Number Publication Date
WO2022247232A1 true WO2022247232A1 (fr) 2022-12-01

Family

ID=77325578

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/137821 WO2022247232A1 (fr) 2021-05-27 2021-12-14 Procédé et appareil d'amélioration d'image, dispositif terminal et support de stockage

Country Status (2)

Country Link
CN (1) CN113298740A (fr)
WO (1) WO2022247232A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113298740A (zh) * 2021-05-27 2021-08-24 中国科学院深圳先进技术研究院 一种图像增强方法、装置、终端设备及存储介质
CN117157665A (zh) * 2022-03-25 2023-12-01 京东方科技集团股份有限公司 视频处理方法及装置、电子设备、计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190172230A1 (en) * 2017-12-06 2019-06-06 Siemens Healthcare Gmbh Magnetic resonance image reconstruction with deep reinforcement learning
CN111353939A (zh) * 2020-03-02 2020-06-30 中国科学院深圳先进技术研究院 一种基于多尺度特征表示与权值共享卷积层的图像超分辨率方法
CN112270644A (zh) * 2020-10-20 2021-01-26 西安工程大学 基于空间特征变换和跨尺度特征集成的人脸超分辨方法
CN112419152A (zh) * 2020-11-23 2021-02-26 中国科学院深圳先进技术研究院 一种图像超分辨率方法、装置、终端设备和存储介质
CN113298740A (zh) * 2021-05-27 2021-08-24 中国科学院深圳先进技术研究院 一种图像增强方法、装置、终端设备及存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830816B (zh) * 2018-06-27 2020-12-04 厦门美图之家科技有限公司 图像增强方法及装置
RU2709661C1 (ru) * 2018-09-19 2019-12-19 Общество с ограниченной ответственностью "Аби Продакшн" Обучение нейронных сетей для обработки изображений с помощью синтетических фотореалистичных содержащих знаки изображений

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190172230A1 (en) * 2017-12-06 2019-06-06 Siemens Healthcare Gmbh Magnetic resonance image reconstruction with deep reinforcement learning
CN111353939A (zh) * 2020-03-02 2020-06-30 中国科学院深圳先进技术研究院 一种基于多尺度特征表示与权值共享卷积层的图像超分辨率方法
CN112270644A (zh) * 2020-10-20 2021-01-26 西安工程大学 基于空间特征变换和跨尺度特征集成的人脸超分辨方法
CN112419152A (zh) * 2020-11-23 2021-02-26 中国科学院深圳先进技术研究院 一种图像超分辨率方法、装置、终端设备和存储介质
CN113298740A (zh) * 2021-05-27 2021-08-24 中国科学院深圳先进技术研究院 一种图像增强方法、装置、终端设备及存储介质

Also Published As

Publication number Publication date
CN113298740A (zh) 2021-08-24

Similar Documents

Publication Publication Date Title
Zamir et al. Learning enriched features for fast image restoration and enhancement
CN111402130B (zh) 数据处理方法和数据处理装置
Shi et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network
EP3948764B1 (fr) Procédé et appareil d'entraînement de modèle de réseau neuronal pour améliorer le détail d'image
WO2021164234A1 (fr) Procédé de traitement d'image et dispositif de traitement d'image
CN112308200B (zh) 神经网络的搜索方法及装置
CN112233038A (zh) 基于多尺度融合及边缘增强的真实图像去噪方法
CN112767290B (zh) 图像融合方法、图像融合装置、存储介质与终端设备
WO2022247232A1 (fr) Procédé et appareil d'amélioration d'image, dispositif terminal et support de stockage
WO2022242122A1 (fr) Procédé et appareil d'optimisation vidéo, équipement terminal, et support d'enregistrement
CN111932480A (zh) 去模糊视频恢复方法、装置、终端设备以及存储介质
Guan et al. Srdgan: learning the noise prior for super resolution with dual generative adversarial networks
Xu et al. Exploiting raw images for real-scene super-resolution
CN116547694A (zh) 用于对模糊图像去模糊的方法和系统
Zhang et al. Deep motion blur removal using noisy/blurry image pairs
CN110717864B (zh) 一种图像增强方法、装置、终端设备及计算机可读介质
CN113628134B (zh) 图像降噪方法及装置、电子设备及存储介质
Hua et al. Dynamic scene deblurring with continuous cross-layer attention transmission
CN111383188A (zh) 一种图像处理方法、系统及终端设备
CN113658050A (zh) 一种图像的去噪方法、去噪装置、移动终端及存储介质
CN111953888B (zh) 暗光成像方法、装置、计算机可读存储介质及终端设备
US20230060988A1 (en) Image processing device and method
Zhang et al. A new image filtering method: Nonlocal image guided averaging
WO2023273515A1 (fr) Procédé de détection de cible, appareil, dispositif électronique et support de stockage
CN115937121A (zh) 基于多维度特征融合的无参考图像质量评价方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21942795

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21942795

Country of ref document: EP

Kind code of ref document: A1