WO2020186372A1 - 一种逆色调映射方法、系统、设备及计算机可读介质 - Google Patents

一种逆色调映射方法、系统、设备及计算机可读介质 Download PDF

Info

Publication number
WO2020186372A1
WO2020186372A1 PCT/CN2019/075313 CN2019075313W WO2020186372A1 WO 2020186372 A1 WO2020186372 A1 WO 2020186372A1 CN 2019075313 W CN2019075313 W CN 2019075313W WO 2020186372 A1 WO2020186372 A1 WO 2020186372A1
Authority
WO
WIPO (PCT)
Prior art keywords
component
illumination
restoration
reflection
reflection component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/075313
Other languages
English (en)
French (fr)
Inventor
王荣刚
王超
王振宇
高文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US16/610,469 priority Critical patent/US20220051375A1/en
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to PCT/CN2019/075313 priority patent/WO2020186372A1/zh
Publication of WO2020186372A1 publication Critical patent/WO2020186372A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/56—Processing of colour picture signals
    • H04N1/60—Colour correction or control
    • H04N1/603—Colour correction or control controlled by characteristics of the picture signal generator or the picture reproducer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/40—Picture signal circuits
    • H04N1/407—Control or modification of tonal gradation or of extreme levels, e.g. background level
    • H04N1/4072—Control or modification of tonal gradation or of extreme levels, e.g. background level dependent on the contents of the original
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20208—High dynamic range [HDR] image processing

Definitions

  • This specification relates to the field of computer technology, in particular to an inverse tone mapping method, system, device and computer readable medium.
  • inverse tone mapping technology is a key link in the application field of 4K television technology and has high research value.
  • inverse tone mapping is an ill-conditioned problem, and it is necessary to restore the information lost in the quantization and compression process of low dynamic range images.
  • inverse tone mapping usually proposes a network model, through which the conversion of low dynamic range images to high dynamic range images is completed.
  • the above-mentioned method has drawbacks. It cannot completely recover the information lost in the low dynamic range image.
  • the network model cannot take into account the recovery of different missing information, such as overexposed areas, underexposed areas, and color information.
  • the embodiments of this specification provide an inverse tone mapping method, system, device, and computer readable medium, which are used to improve the problem that the inverse tone mapping image restoration effect in the prior art cannot reach the expected problem.
  • the embodiment of this specification provides an inverse tone mapping method, the method includes:
  • the illumination component represents the global illumination condition of the image
  • the reflection component represents the color and texture details of the image
  • the illumination component is restored, wherein the illumination component is restored according to an illumination component restoration network based on a full convolutional network.
  • the illumination component is restored according to an illumination component restoration network based on a full convolutional network, wherein each layer of the illumination component restoration network includes a convolutional layer and an activation layer, and the activation The activation function of the layer uses SELU.
  • the illumination component is restored according to an illumination component restoration network based on a full convolutional network, wherein:
  • the illumination component restoration network includes first to seventh illumination component restoration layers;
  • the number of characteristic channels of the first to sixth illumination component restoration layers is 64, and the number of characteristic channels of the seventh illumination component restoration layer is 3;
  • the size of the convolution kernel of the first to sixth illumination component restoration layers is 3*3, and the step size is 1, and the size of the convolution kernel of the seventh illumination component restoration layer is 1*1, and the step size is 1.
  • the illumination component is restored according to the illumination component restoration network based on the full convolutional network, wherein the edge filling is performed in a mirror symmetry manner.
  • the illumination component is restored according to the illumination component restoration network based on the full convolutional network, wherein a residual is introduced, and the input and output of the illumination component restoration network are added, and the residual is learned The restoration of the illumination component is completed.
  • the reflection component is restored, wherein the reflection component is restored according to a reflection component restoration network based on a U-Net structure.
  • the reflection component is restored according to the reflection component recovery network based on the U-Net structure, wherein:
  • the reflection component recovery network includes first to tenth reflection component recovery layers
  • the first to fifth reflection component recovery layers and the tenth reflection component recovery layer are convolution layers, and the sixth to ninth reflection component recovery layers are deconvolution layers;
  • the number of characteristic channels of the first to tenth reflection component recovery layers are 64, 128, 256, 512, 1024, 512, 256, 128, 64, 3, respectively;
  • the size of the convolution kernel of the first to fourth reflection component recovery layers is 3*3, and the step size is 2, and the size of the convolution kernel of the fifth to ninth reflection component recovery layers is 3*3, and the step size is 1.
  • the size of the convolution kernel of the tenth reflection component recovery layer is 1*1 and the step size is 1.
  • the reflection component is restored according to the reflection component recovery network based on the U-Net structure, wherein, in the deconvolution layer of the reflection component recovery network, the feature is expanded by upsampling by bilinear interpolation. The resolution of the picture, and then convolution operation.
  • the reflection component is recovered according to a reflection component recovery network based on the U-Net structure, wherein, in the reflection component recovery network, each layer adds a batch normalization operation.
  • This application also proposes an inverse tone mapping system, which includes:
  • a component decomposition module configured to decompose the original image into illumination components and reflection components, wherein the illumination components represent the global illumination conditions of the image, and the reflection components represent the color and texture details of the image;
  • An illumination component restoration module which is configured to restore the illumination component and obtain the illumination component restoration result
  • a reflection component restoration module configured to restore the reflection component and obtain a reflection component restoration result
  • the component merging module is configured to merge the restoration result of the illumination component and the restoration result of the reflection component to obtain a restored result image.
  • This application also proposes a computer-readable medium on which computer-readable instructions are stored, and the computer-readable instructions can be executed by a processor to implement the method described in the embodiments of this specification.
  • This application also proposes a device for information processing on the user equipment side.
  • the device includes a memory for storing computer program instructions and a processor for executing the program instructions.
  • the device executes, the device is triggered to execute the method described in the embodiment of this specification.
  • Fig. 1 is a flowchart of a method according to an embodiment of the present specification
  • Fig. 2 is a schematic diagram of a light component restoration network structure according to an embodiment of the present specification
  • Fig. 3 is a schematic diagram of a reflection component recovery network structure according to an embodiment of the present specification
  • Fig. 4 is a block diagram of a system structure according to an embodiment of the present specification.
  • Fig. 5 is an illumination component obtained according to an embodiment of this specification for the original image shown in Fig. 4;
  • Fig. 6 is a reflection component obtained according to an embodiment of this specification for the original image shown in Fig. 4;
  • Fig. 7 is a light component restoration result obtained according to an embodiment of this specification for the light component shown in Fig. 5;
  • FIG. 8 is a reflection component restoration result obtained according to an embodiment of this specification for the reflection component shown in FIG. 6;
  • FIG. 9 is a restoration result image acquired according to an embodiment of this specification based on FIG. 7 and FIG. 8;
  • Fig. 10 is a block diagram of a system structure according to an embodiment of the present specification.
  • inverse tone mapping usually proposes a network model, and completes the conversion from low dynamic range images to high dynamic range images through the network model.
  • the above-mentioned method has drawbacks. It cannot completely recover the information lost in the low dynamic range image.
  • the network model cannot take into account the recovery of different missing information, such as overexposed areas, underexposed areas, and color information.
  • the embodiment of this specification proposes an inverse tone mapping method.
  • this application first makes a detailed analysis of actual application scenarios.
  • one of the disadvantages of inverse tone mapping is that it cannot take into account the recovery of different missing information, such as over-exposed areas, under-exposed areas, and color information. Therefore, in an embodiment of the present specification, different parameter models are used to restore different feature attributes of the image, and then the restoration results of all models are combined to obtain the final image restoration result.
  • Rtinex theory is a theory widely used in digital image processing. It believes that digital images can be decomposed into illumination components and reflection components.
  • the illumination components mainly represent the global illumination conditions of the image, and the reflection components Indicates the color and texture details of the image, which are independent and do not affect each other. Therefore, in an embodiment of this specification, the original image (low dynamic range image) is decomposed into illumination components and reflection components based on the Rtinex theory, where the illumination component represents the global illumination conditions and dynamic range of the image, and the reflection component represents the color of the image. And details.
  • the two components are restored separately, and finally the restoration results of the two components are combined based on the Rtinex theory to obtain the final restoration result image (high dynamic range image).
  • the restoration operations of the illumination component and the reflection component respectively represent the expansion of the dynamic range and the restoration of the detail texture. Recovering the two components separately is equivalent to decomposing the inverse tone mapping operation into two subtasks, and the two subtasks are independent of each other and do not affect each other. This not only improves the shortcomings of a single network model that cannot take into account the recovery of different missing information, but also reduces the complexity of the inverse tone mapping operation, and the recovery effect is more robust.
  • the method includes the following steps.
  • S110 Decompose the original image (low dynamic range image) into illumination components and reflection components;
  • S140 Combine the restoration result of the illumination component and the restoration result of the reflection component to obtain a restoration result image (high dynamic range image).
  • the illumination component mainly represents the global illumination conditions of the image
  • the reflection component represents the color and texture details of the image
  • the two independently do not affect each other. Therefore, in an embodiment of this specification, the difference between the illumination component and the reflection component Features and different recovery needs adopt different recovery strategies. Specifically, different network models are used to restore the illumination component and the reflection component respectively.
  • the illumination component since the illumination component represents global information, the structural integrity of the illumination component needs to be ensured to reduce information loss. In order to reduce the loss of information, down sampling needs to be avoided. Therefore, in an embodiment of this specification, for the illumination component, a full convolutional network without down-sampling is used for recovery. Specifically, in an embodiment, the illumination component is restored according to an illumination component restoration network based on a full convolutional network.
  • the illumination component recovery network based on the full convolutional network includes multiple layers, where each layer includes a convolutional layer and an activation layer.
  • the activation function of the activation layer is determined according to the results of field practice.
  • the activation function of the activation layer uses SELU. It should be noted here that in other embodiments of the present invention, the activation layer of each layer of the illumination component recovery network may also use other activation functions.
  • the network structure setting of the light component recovery network with relatively better recovery effect in the general application scenario is determined according to the experimental records of the actual application scenario. In this way, the user can restore the illumination component based on the set illumination component restoration network.
  • the specific structure of the illumination component restoration network for restoring illumination components is set as:
  • the illumination component restoration network includes first to seventh illumination component restoration layers;
  • the number of feature channels of the first to sixth illumination component restoration layers is 64, and the number of feature channels of the seventh illumination component restoration layer is 3;
  • the convolution kernel size of the first to sixth illumination component restoration layers is 3*3, and the step size is 1, and the convolution kernel size of the seventh illumination component restoration layer is 1*1 and the step size is 1.
  • the above-mentioned specific settings of the light component restoration network structure are only specific settings of the light component restoration network structure with relatively good restoration effects in certain application scenarios, and do not represent the light component restoration network of all embodiments of this specification. Must adopt this network structure setting.
  • the specific network structure settings of the light component recovery network can be set according to specific original image characteristics and/or recovery requirements.
  • a mirror symmetry method is used for edge filling in the process of restoring the illumination component.
  • the residual network can improve the learning efficiency and reduce the learning difficulty
  • the residual error is introduced into the illumination component recovery network, and the illumination The input and output of the component restoration network are added together, and the illumination component is restored by learning the residual.
  • the illumination component restoration network includes seven illumination component restoration layers from 210 to 270.
  • the number of feature channels of layers 210-260 is 64, the size of the convolution kernel is 3*3, and the step size is 1; the number of feature channels of layer 270 is 3, the size of convolution kernel is 1*1, and the step size is 1.
  • the reflection component has a large amount of color and texture information, which is crucial to the recovery of overexposed and underexposed areas. Therefore, in one embodiment, multi-scale information is used to restore the reflected component.
  • U-Net is used as the illumination component recovery network, that is, the reflection component is recovered by the network based on the reflection component recovery of the U-Net structure.
  • the resolution of the feature map is expanded by upsampling by bilinear interpolation, and then the convolution operation is performed.
  • each layer adds a batch normalization operation.
  • the network structure setting of the reflection component recovery network with relatively better recovery effect in the general application scenario is determined according to the experimental records of the actual application scenario. In this way, the user can restore the reflected component based on the set reflection component recovery network.
  • the specific structure of the reflection component recovery network based on the U-Net structure for recovering the reflection component is set as:
  • the reflection component recovery network includes first to tenth reflection component recovery layers
  • the first to fifth reflection component recovery layers and the tenth reflection component recovery layer are convolutional layers, and the sixth to ninth reflection component recovery layers are deconvolutional layers;
  • the characteristic channel numbers of the first to tenth reflection component recovery layers are 64, 128, 256, 512, 1024, 512, 256, 128, 64, 3 respectively;
  • the convolution kernel size of the first to fourth reflection component recovery layers is 3*3, and the step size is 2, and the convolution kernel size of the fifth to ninth reflection component recovery layers is 3*3, and the step size is 1.
  • the size of the convolution kernel of the reflection component recovery layer is 1*1 and the step size is 1.
  • the reflection component recovery network includes ten light component recovery layers 301 to 310.
  • Layers 301 to 305 and layer 310 are convolutional layers, and layers 306 to 309 are deconvolutional layers;
  • the characteristic channel numbers of layers 301 to 310 are 64, 128, 256, 512, 1024, 512, 256, 128, 64, 3;
  • the size of the convolution kernel of layers 301-304 is 3*3, the step size is 2, the size of the convolution kernels of layers 305-309 is 3*3, the step size is 1, and the size of the convolution kernel of layer 310 is 1* 1.
  • the step size is 1.
  • the original image (low dynamic range image) is shown in FIG. 4.
  • the original image shown in FIG. 4 is decomposed into the illumination components shown in FIG. And the reflection component as shown in Figure 6.
  • the illumination component is restored, and the illumination component restoration result shown in Figure 7 is obtained.
  • the reflection component is restored, and the reflection component restoration result shown in Figure 8 is obtained.
  • the restored result image (high dynamic range image) as shown in FIG. 9 is obtained.
  • the embodiment of this specification also proposes an inverse tone mapping system. Specifically, as shown in FIG. 10, in an embodiment, the system includes:
  • a component decomposition module 410 which is configured to decompose the original image into illumination components and reflection components based on Retinex theory
  • the illumination component restoration module 420 is configured to restore the illumination component and obtain the illumination component restoration result
  • the reflection component recovery module 430 is configured to recover the reflection component and obtain the reflection component recovery result
  • the component merging module 440 is configured to merge the restoration result of the illumination component and the restoration result of the reflection component based on the Retinex theory to obtain the restored result image.
  • the embodiment of this specification also proposes a computer-readable medium on which computer-readable instructions are stored, and the computer-readable instructions can be executed by a processor to implement the method described in the embodiment of this specification .
  • the embodiment of this specification also proposes a device for information processing on the user equipment side.
  • the device includes a memory for storing computer program instructions and a processor for executing the program instructions, Wherein, when the computer program instruction is executed by the processor, the device is triggered to execute the method described in the embodiment of this specification.
  • a programmable logic device Programmable Logic Device, PLD
  • FPGA Field Programmable Gate Array
  • HDL Hardware Description Language
  • ABEL Advanced Boolean Expression Language
  • AHDL Altera Hardware Description Language
  • HDCal JHDL
  • Lava Lava
  • Lola MyHDL
  • PALASM RHDL
  • Verilog Verilog
  • the controller can be implemented in any suitable manner.
  • the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program codes (such as software or firmware) executable by the (micro)processor. , Logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers and embedded microcontrollers.
  • controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicon Labs C8051F320, the memory controller can also be implemented as a part of the memory control logic.
  • controller in addition to implementing the controller in a purely computer-readable program code manner, it is entirely possible to program the method steps to make the controller use logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded The same function can be realized in the form of a microcontroller, etc. Therefore, such a controller can be regarded as a hardware component, and the devices included in it for implementing various functions can also be regarded as a structure within the hardware component. Or even, the device for realizing various functions can be regarded as both a software module for realizing the method and a structure within a hardware component.
  • a typical implementation device is a computer.
  • the computer may be, for example, a personal computer, a laptop computer, a cell phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or Any combination of these devices.
  • the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
  • a computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.
  • the computing device includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.
  • processors CPU
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • the memory may include non-permanent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer readable media.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash memory
  • Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology.
  • the information can be computer-readable instructions, data structures, program modules, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.
  • program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types.
  • This application can also be practiced in distributed computing environments. In these distributed computing environments, remote processing devices connected through a communication network perform tasks.
  • program modules can be located in local and remote computer storage media including storage devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种逆色调映射方法、系统、设备及计算机可读介质。本申请实施例的方法包括：将原始图像分解为光照分量以及反射分量，其中，所述光照分量表示图像的全局光照条件，所述反射分量表示图像的颜色和纹理细节；对所述光照分量进行恢复，获取光照分量恢复结果；对所述反射分量进行恢复，获取反射分量恢复结果；合并所述光照分量恢复结果以及所述反射分量恢复结果，获取恢复结果图像。相较于现有技术，根据本发明实施例的方法进行逆色调映射，可以大大提高图像的恢复效果。

Description

一种逆色调映射方法、系统、设备及计算机可读介质 技术领域
本说明书涉及计算机技术领域，尤其涉及一种逆色调映射方法、系统、设备及计算机可读介质。
背景技术
目前，4K电视技术和相关应用正在快速发展。在4K电视标准中，高动态范围的播放是一个重要的组成部分。但是由于目前大多数媒体资源仍为低动态范围的存储方式，所以需要对媒体资源进行逆色调映射，将媒体资源从低动态范围转换到高动态范围。
在现有技术中，逆色调映射技术是4K电视技术应用领域的一个关键环节，具有很高的研究价值。在实际应用场景中，逆色调映射是一个病态问题，需要恢复低动态范围图像在量化和压缩过程中损失的信息。一般的，逆色调映射通常是提出一个网络模型，通过模型完成低动态范围图像到高动态范围图像的转换。但是，上述方式存在弊端，其无法完全恢复低动态范围图像中损失的信息，尤其的，网络模型不能很好兼顾不同缺失信息的恢复，诸如过曝区域、欠曝区域和颜色信息。
发明内容
有鉴于此，本说明书实施例提供了一种逆色调映射方法、系统、设备及计算机可读介质，用于改善现有技术中逆色调映射图像恢复效果无法达到预期的问题。
本说明书实施例采用下述技术方案：
本说明书实施例提供一种逆色调映射方法，所述方法包括：
将原始图像分解为光照分量以及反射分量，其中，所述光照分量表示图像的全局光照条件，所述反射分量表示图像的颜色和纹理细节；
对所述光照分量进行恢复，获取光照分量恢复结果；
对所述反射分量进行恢复，获取反射分量恢复结果；
合并所述光照分量恢复结果以及所述反射分量恢复结果，获取恢复结果图像。
在一实施例中，对所述光照分量进行恢复，其中，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复。
在一实施例中，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复，其中，所述光照分量恢复网络的每一层包含一个卷积层和一个激活层，所述激活层的激活函数采用SELU。
在一实施例中，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复，其中：
按照从输入到输出的次序，所述光照分量恢复网络包括第一至第七光照分量恢复层；
所述第一至第六光照分量恢复层的特征通道数为64，所述第七光照分量恢复层的特征通道数为3；
所述第一至第六光照分量恢复层的卷积核尺寸为3*3、步长为1，所述第七光照分量恢复层的卷积核尺寸为1*1、步长为1。
在一实施例中，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复，其中，采用镜像对称的方式做边缘填充。
在一实施例中，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复，其中，引入残差，将所述光照分量恢复网络的输入和输出相加，通过学习残差来完成所述光照分量的恢复。
在一实施例中，对所述反射分量进行恢复，其中，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量。
在一实施例中，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量，其中：
按照从输入到输出的次序，所述反射分量恢复网络包括第一至第十反射分量恢复层；
所述第一至第五反射分量恢复层以及第十反射分量恢复层为卷积层，所述第六至第九反射分量恢复层为反卷积层；
所述第一至第十反射分量恢复层的特征通道数分别为64、128、256、512、1024、512、256、128、64、3；
所述第一至第四反射分量恢复层的卷积核尺寸为3*3、步长为2，所述第五至第九反射分量恢复层的卷积核尺寸为3*3、步长为1，所述第十反射分量恢复层的卷积核尺寸为1*1、步长为1。
在一实施例中，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量，其中，在所述反射分量恢复网络的反卷积层中，先通过双线性插值上采样，扩大特征图的分辨率，再进行卷积操作。
在一实施例中，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量，其中，在所述反射分量恢复网络中，每一层都加入批归一化操作。
本申请还提出了一种逆色调映射系统，所述系统包括：
分量分解模块，其配置为将原始图像分解为光照分量以及反射分量，其中，所述光照分量表示图像的全局光照条件，所述反射分量表示图像的颜色和纹理细节；
光照分量恢复模块，其配置为对所述光照分量进行恢复，获取光照分量恢复结果；
反射分量恢复模块，其配置对所述反射分量进行恢复，获取反射分量恢复结果；
分量合并模块，其配置为合并所述光照分量恢复结果以及所述反射分量恢复结果，获取恢复结果图像。
本申请还提出了一种计算机可读介质，其上存储有计算机可读指令，所述计算机可读指令可被处理器执行以实现本说明书实施例所述的方法。
本申请还提出了一种用于在用户设备端进行信息处理的设备，该设备包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发该设备执行本说明书实施例所述的方法。
本说明书实施例采用的上述至少一个技术方案能够达到以下有益效果：相较于现有技术，根据本发明实施例的方法进行逆色调映射，可以大大提高图像的恢复效果。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1为根据本说明书一实施例的方法执行流程图；
图2为根据本说明书一实施例的光照分量恢复网络结构示意图；
图3为根据本说明书一实施例的反射分量恢复网络结构示意图；
图4为根据本说明书一实施例的系统结构框图；
图5为针对图4所示原始图像，根据本说明书一实施例获取的光照分量；
图6为针对图4所示原始图像，根据本说明书一实施例获取的反射分量；
图7为针对图5所示光照分量，根据本说明书一实施例获取的光照分量恢复结果；
图8为针对图6所示反射分量，根据本说明书一实施例获取的反射分量恢复结果；
图9为基于图7以及图8，根据本说明书一实施例获取的恢复结果图像；
图10为根据本说明书一实施例的系统结构框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施例及相应的附图对本申请技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
在现有技术中，逆色调映射通常是提出一个网络模型，通过网络模型完成 低动态范围图像到高动态范围图像的转换。但是，上述方式存在弊端，其无法完全恢复低动态范围图像中损失的信息，尤其的，网络模型不能很好兼顾不同缺失信息的恢复，诸如过曝区域、欠曝区域和颜色信息。
针对上述问题，本说明书实施例提出了一种逆色调映射方法。为了提出本说明书实施例的方法，本申请首先对实际应用场景做详细分析。在现有技术中，逆色调映射的弊端之一在于不能很好兼顾不同缺失信息的恢复，诸如过曝区域、欠曝区域和颜色信息。因此，在本说明书一实施例中，采用不同的参数模型分别针对图像的不同特征属性进行恢复，然后综合所有模型的恢复结果获取最终的图像恢复结果。
具体的，在实际应用场景中，Rtinex理论是一种广泛应用于数字图像处理中的理论，它认为数字图像可以被分解为光照分量和反射分量，光照分量主要表示图像的全局光照条件，反射分量表示图像的颜色和纹理细节，两者独立互不影响。因此，在本说明书一实施例中，基于Rtinex理论将原始图像(低动态范围图像)分解为光照分量以及反射分量，其中，光照分量表示图像的全局光照条件和动态范围，反射分量表示图像的颜色和细节信息。分别对两个分量进行恢复，最后基于Rtinex理论合并两个分量的恢复结果，从而获取最终的恢复结果图像(高动态范围图像)。
在上述步骤中，光照分量以及反射分量的恢复操作分别代表动态范围的扩展和细节纹理的恢复。针对两个分量分别进行恢复，相当于将逆色调映射操作分解为两个子任务，并且两个子任务相互独立、互不影响。这样不仅改善了单一网络模型不能很好兼顾不同缺失信息的恢复的弊端，而且减轻了逆色调映射操作的复杂程度，恢复效果更加鲁棒。进一步的，相较于现有技术，由于本说明书实施例的逆色调映射操作的复杂程度被大大降低，其实现过程中所需的参数设置也被大大简化，这就使得专业能力相对较低的普通用户也能完成逆色调映射操作，从而大大提高了逆色调映射操作实用性以及可推广性。
以下结合附图，详细说明本说明书各实施例提供的技术方案。如图1所示， 在一实施例中，方法包括以下步骤。
S110，将原始图像(低动态范围图像)分解为光照分量以及反射分量；
S120，对光照分量进行恢复，获取光照分量恢复结果；
S130，对反射分量进行恢复，获取反射分量恢复结果；
S140，合并光照分量恢复结果以及反射分量恢复结果，获取恢复结果图像(高动态范围图像)。
进一步的，由于光照分量主要表示图像的全局光照条件，反射分量表示图像的颜色和纹理细节，两者独立互不影响，因此，在本说明书一实施例中，分别针对光照分量以及反射分量的不同特征及其不同的恢复需求采用不同的恢复策略。具体的，采用不同的网络模型分别针对光照分量以及反射分量进行恢复。
具体的，由于光照分量表示着全局信息，对于光照分量需要保证其结构的完整性，减少信息损失。而为了减少信息的损失，需要避免下采样。因此，在本说明书一实施例中，针对光照分量，采用没有下采样的全卷积网络来进行恢复。具体的，在一实施例中，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复。
进一步的，在一实施例中，基于全卷积网络的光照分量恢复网络包含多个层，其中，每一层包含一个卷积层和一个激活层。在一实施例中，根据现场实践结果确定激活层的激活函数。在一实施例中，激活层的激活函数采用SELU。这里需要说明的是，在本发明其他实施例中，光照分量恢复网络各层的激活层也可以采用其他激活函数。
进一步的，在一实施例中，为了进一步简化参数设置，在一实施例中，根据实际应用场景的实验记录确定一般应用场景下恢复效果相对较优的光照分量恢复网络的网络结构设置。这样用户就可以基于已设置好的光照分量恢复网络进行光照分量的恢复。
具体的，在一实施例中，用于恢复光照分量的光照分量恢复网络的具体结 构设置为：
按照从输入到输出的次序，所述光照分量恢复网络包括第一至第七光照分量恢复层；
第一至第六光照分量恢复层的特征通道数为64，第七光照分量恢复层的特征通道数为3；
第一至第六光照分量恢复层的卷积核尺寸为3*3、步长为1，第七光照分量恢复层的卷积核尺寸为1*1、步长为1。
这里需要说明的是，上述光照分量恢复网络结构的具体设置只是在某些应用场景下恢复效果相对较优的光照分量恢复网络结构的具体设置，并不代表本说明书所有实施例的光照分量恢复网络都必须采用该网络结构设置。在实际应用场景中，可以根据具体的原始图像特征和/或恢复需求设定光照分量恢复网络的具体网络结构设置。
进一步的，在一实施例中，在恢复光照分量的过程中，为了减少信息的损失，需要保证恢复前后的特征图尺寸不变。具体的，在一实施例中，为了保持特征图尺寸不变，在恢复光照分量的过程中采用镜像对称的方式做边缘填充。
进一步的，考虑到残差网络可以提高学习效率降低学习难度，在一实施例中，为了提高训练光照分量恢复网络时的稳定性、提高训练效率，在光照分量恢复网络中引入残差，将光照分量恢复网络的输入和输出相加，通过学习残差来完成光照分量的恢复。
具体的，如图2所示，在一实施例中，光照分量恢复网络包括210～270七个光照分量恢复层。层210～260的特征通道数为64，卷积核尺寸为3*3、步长为1；层270的特征通道数为3，卷积核尺寸为1*1、步长为1。
进一步的，不同于光照分量，反射分量有大量的颜色纹理信息，这一部分信息对过曝欠曝区域的恢复至关重要。因此，在一实施例中，利用多尺度的信息来恢复反射分量。具体的，在一实施例中，为了利用多尺度信息来恢复反射分量，采用U-Net作为光照分量恢复网络，即，根据基于U-Net结构的反射分 量恢复网络恢复反射分量。
进一步的，在一实施例中，为了避免棋盘伪影，在反射分量恢复网络的反卷积层中，先通过双线性插值上采样，扩大特征图的分辨率，再进行卷积操作。
进一步的，在一实施例中，为了加快收敛速度，在反射分量恢复网络中，每一层都加入批归一化操作。
进一步的，在一实施例中，为了进一步简化参数设置，在一实施例中，根据实际应用场景的实验记录确定一般应用场景下恢复效果相对较优的反射分量恢复网络的网络结构设置。这样用户就可以基于已设置好的反射分量恢复网络进行反射分量的恢复。
具体的，在一实施例中，用于恢复反射分量的基于U-Net结构的反射分量恢复网络的具体结构设置为：
按照从输入到输出的次序，反射分量恢复网络包括第一至第十反射分量恢复层；
第一至第五反射分量恢复层以及第十反射分量恢复层为卷积层，第六至第九反射分量恢复层为反卷积层；
第一至第十反射分量恢复层的特征通道数分别为64、128、256、512、1024、512、256、128、64、3；
第一至第四反射分量恢复层的卷积核尺寸为3*3、步长为2，第五至第九反射分量恢复层的卷积核尺寸为3*3、步长为1，第十反射分量恢复层的卷积核尺寸为1*1、步长为1。
具体的，如图3所示，在一实施例中，反射分量恢复网络包括301～310十个光照分量恢复层。层301～305以及层310为卷积层，层306～309为反卷积层；层301～310的特征通道数分别为64、128、256、512、1024、512、256、128、64、3；层301～304的卷积核尺寸为3*3、步长为2，层305～309的卷积核尺寸为3*3、步长为1，层310的卷积核尺寸为1*1、步长为1。
具体的，在一应用场景中，原始图像(低动态范围图像)如图4所示，根 据本说明书一实施例的方法，将图4所示的原始图像分解为如图5所示的光照分量以及如图6所示的反射分量。对光照分量进行恢复，获得如图7所示的光照分量恢复结果。对反射分量进行恢复，得到如图8所示的反射分量恢复结果。合并光照分量恢复结果以及反射分量恢复结果，获得如图9所示的恢复结果图像(高动态范围图像)。
进一步的，基于本说明书实施例的方法，本说明书实施例还提出了一种逆色调映射系统。具体的，如图10所示，在一实施例中，系统包括：
分量分解模块410，其配置为基于Retinex理论将原始图像分解为光照分量以及反射分量；
光照分量恢复模块420，其配置为对光照分量进行恢复，获取光照分量恢复结果；
反射分量恢复模块430，其配置对反射分量进行恢复，获取反射分量恢复结果；
分量合并模块440，其配置为基于Retinex理论合并光照分量恢复结果以及反射分量恢复结果，获取恢复结果图像。
基于本说明书实施例的方法，本说明书实施例还提出了一种计算机可读介质，其上存储有计算机可读指令，计算机可读指令可被处理器执行以实现本说明书实施例所述的方法。
基于本说明书实施例的方法，本说明书实施例还提出了一种用于在用户设备端进行信息处理的设备，该设备包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发该设备执行本说明书实施例所述的方法。
在20世纪90年代，对于一个技术的改进可以很明显地区分是硬件上的改进(例如，对二极管、晶体管、开关等电路结构的改进)还是软件上的改进(对于方法流程的改进)。然而，随着技术的发展，当今的很多方法流程的改进已经可以视为硬件电路结构的直接改进。设计人员几乎都通过将改进的方法流程 编程到硬件电路中来得到相应的硬件电路结构。因此，不能说一个方法流程的改进就不能用硬件实体模块来实现。例如，可编程逻辑器件(Programmable Logic Device,PLD)(例如现场可编程门阵列(Field Programmable Gate Array，FPGA))就是这样一种集成电路，其逻辑功能由用户对器件编程来确定。由设计人员自行编程来把一个数字系统“集成”在一片PLD上，而不需要请芯片制造厂商来设计和制作专用的集成电路芯片。而且，如今，取代手工地制作集成电路芯片，这种编程也多半改用“逻辑编译器(logic compiler)”软件来实现，它与程序开发撰写时所用的软件编译器相类似，而要编译之前的原始代码也得用特定的编程语言来撰写，此称之为硬件描述语言(Hardware Description Language，HDL)，而HDL也并非仅有一种，而是有许多种，如ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby Hardware Description Language)等，目前最普遍使用的是VHDL(Very-High-Speed Integrated Circuit Hardware Description Language)与Verilog。本领域技术人员也应该清楚，只需要将方法流程用上述几种硬件描述语言稍作逻辑编程并编程到集成电路中，就可以很容易得到实现该逻辑方法流程的硬件电路。
控制器可以按任何适当的方式实现，例如，控制器可以采取例如微处理器或处理器以及存储可由该(微)处理器执行的计算机可读程序代码(例如软件或固件)的计算机可读介质、逻辑门、开关、专用集成电路(Application Specific Integrated Circuit，ASIC)、可编程逻辑控制器和嵌入微控制器的形式，控制器的例子包括但不限于以下微控制器：ARC 625D、Atmel AT91SAM、Microchip PIC18F26K20以及Silicone Labs C8051F320，存储器控制器还可以被实现为存储器的控制逻辑的一部分。本领域技术人员也知道，除了以纯计算机可读程序代码方式实现控制器以外，完全可以通过将方法步骤进行逻辑编程来使得控制 器以逻辑门、开关、专用集成电路、可编程逻辑控制器和嵌入微控制器等的形式来实现相同功能。因此这种控制器可以被认为是一种硬件部件，而对其内包括的用于实现各种功能的装置也可以视为硬件部件内的结构。或者甚至，可以将用于实现各种功能的装置视为既可以是实现方法的软件模块又可以是硬件部件内的结构。
上述实施例阐明的系统、装置、模块或单元，具体可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。一种典型的实现设备为计算机。具体的，计算机例如可以为个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任何设备的组合。
为了描述的方便，描述以上装置时以功能分为各种单元分别描述。当然，在实施本申请时可以把各单元的功能在同一个或多个软件和/或硬件中实现。
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中 的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、商品或者设 备中还存在另外的相同要素。
本申请可以在由计算机执行的计算机可执行指令的一般上下文中描述，例如程序模块。一般地，程序模块包括执行特定任务或实现特定抽象数据类型的例程、程序、对象、组件、数据结构等等。也可以在分布式计算环境中实践本申请，在这些分布式计算环境中，由通过通信网络而被连接的远程处理设备来执行任务。在分布式计算环境中，程序模块可以位于包括存储设备在内的本地和远程计算机存储介质中。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于系统实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
以上所述仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (13)

  1. 一种逆色调映射方法，其特征在于，所述方法包括：
    将原始图像分解为光照分量以及反射分量，其中，所述光照分量表示图像的全局光照条件，所述反射分量表示图像的颜色和纹理细节；
    对所述光照分量进行恢复，获取光照分量恢复结果；
    对所述反射分量进行恢复，获取反射分量恢复结果；
    合并所述光照分量恢复结果以及所述反射分量恢复结果，获取恢复结果图像。
  2. 根据权利要求1所述的方法，其特征在于，对所述光照分量进行恢复，其中，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复。
  3. 根据权利要求2所述的方法，其特征在于，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复，其中，所述光照分量恢复网络的每一层包含一个卷积层和一个激活层，所述激活层的激活函数采用SELU。
  4. 根据权利要求2或3所述的方法，其特征在于，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复，其中：
    按照从输入到输出的次序，所述光照分量恢复网络包括第一至第七光照分量恢复层；
    所述第一至第六光照分量恢复层的特征通道数为64，所述第七光照分量恢复层的特征通道数为3；
    所述第一至第六光照分量恢复层的卷积核尺寸为3*3、步长为1，所述第七光照分量恢复层的卷积核尺寸为1*1、步长为1。
  5. 根据权利要求2～4中任一项所述的方法，其特征在于，根据基于全卷积网络的光照分量恢复网络对所述光照分量进行恢复，其中，采用镜像对称的方式做边缘填充。
  6. 根据权利要求2～5中任一项所述的方法，其特征在于，根据基于全卷 积网络的光照分量恢复网络对所述光照分量进行恢复，其中，引入残差，将所述光照分量恢复网络的输入和输出相加，通过学习残差来完成所述光照分量的恢复。
  7. 根据权利要求1～6中任一项所述的方法，其特征在于，对所述反射分量进行恢复，其中，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量。
  8. 根据权利要求7所述的方法，其特征在于，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量，其中：
    按照从输入到输出的次序，所述反射分量恢复网络包括第一至第十反射分量恢复层；
    所述第一至第五反射分量恢复层以及第十反射分量恢复层为卷积层，所述第六至第九反射分量恢复层为反卷积层；
    所述第一至第十反射分量恢复层的特征通道数分别为64、128、256、512、1024、512、256、128、64、3；
    所述第一至第四反射分量恢复层的卷积核尺寸为3*3、步长为2，所述第五至第九反射分量恢复层的卷积核尺寸为3*3、步长为1，所述第十反射分量恢复层的卷积核尺寸为1*1、步长为1。
  9. 根据权利要求7或8所述的方法，其特征在于，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量，其中，在所述反射分量恢复网络的反卷积层中，先通过双线性插值上采样，扩大特征图的分辨率，再进行卷积操作。
  10. 根据权利要求7～9中任一项所述的方法，其特征在于，根据基于U-Net结构的反射分量恢复网络恢复所述反射分量，其中，在所述反射分量恢复网络中，每一层都加入批归一化操作。
  11. 一种逆色调映射系统，其特征在于，所述系统包括：
    分量分解模块，其配置为将原始图像分解为光照分量以及反射分量，其中，所述光照分量表示图像的全局光照条件，所述反射分量表示图像的颜色和纹理 细节；
    光照分量恢复模块，其配置为对所述光照分量进行恢复，获取光照分量恢复结果；
    反射分量恢复模块，其配置对所述反射分量进行恢复，获取反射分量恢复结果；
    分量合并模块，其配置为合并所述光照分量恢复结果以及所述反射分量恢复结果，获取恢复结果图像。
  12. 一种计算机可读介质，其上存储有计算机可读指令，所述计算机可读指令可被处理器执行以实现权利要求1至10中任一项所述的方法。
  13. 一种用于在用户设备端进行信息处理的设备，该设备包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发该设备执行权利要求1至10中任一项所述的方法。
PCT/CN2019/075313 2019-03-18 2019-03-18 一种逆色调映射方法、系统、设备及计算机可读介质 Ceased WO2020186372A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US16/610,469 US20220051375A1 (en) 2019-03-18 2019-02-18 An inverse tone mapping method, system, device and computer readable medium
PCT/CN2019/075313 WO2020186372A1 (zh) 2019-03-18 2019-03-18 一种逆色调映射方法、系统、设备及计算机可读介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/075313 WO2020186372A1 (zh) 2019-03-18 2019-03-18 一种逆色调映射方法、系统、设备及计算机可读介质

Publications (1)

Publication Number Publication Date
WO2020186372A1 true WO2020186372A1 (zh) 2020-09-24

Family

ID=72519516

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/075313 Ceased WO2020186372A1 (zh) 2019-03-18 2019-03-18 一种逆色调映射方法、系统、设备及计算机可读介质

Country Status (2)

Country Link
US (1) US20220051375A1 (zh)
WO (1) WO2020186372A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11682492B2 (en) * 2019-06-07 2023-06-20 Cornell University Energy-efficient on-chip classifier for detecting physiological conditions
CN114612727A (zh) * 2022-03-23 2022-06-10 西北工业大学 基于双边特征金字塔网络与多尺度鉴别的本质图像分解方法研究
CN115205157B (zh) * 2022-07-29 2024-04-26 如你所视(北京)科技有限公司 图像处理方法和系统、电子设备和存储介质
US20250037256A1 (en) * 2023-07-28 2025-01-30 Paypal, Inc. Interpretive and qualitative assessments of document image quality for document image submissions

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105469373A (zh) * 2014-09-04 2016-04-06 南京理工大学 一种Retinex图像增强方法及系统
CN105654437A (zh) * 2015-12-24 2016-06-08 广东迅通科技股份有限公司 一种对低照度图像的增强方法
US20170256092A1 (en) * 2016-03-02 2017-09-07 Fuji Xerox Co., Ltd. Image processing apparatus, image processing system, image processing method, and non-transitory computer readable medium
CN107578383A (zh) * 2017-08-29 2018-01-12 北京华易明新科技有限公司 一种低照度图像增强处理方法
CN108305236A (zh) * 2018-01-16 2018-07-20 腾讯科技（深圳）有限公司 图像增强处理方法及装置
CN109785263A (zh) * 2019-01-14 2019-05-21 北京大学深圳研究生院 一种基于Retinex的逆色调映射图像转换方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105469373A (zh) * 2014-09-04 2016-04-06 南京理工大学 一种Retinex图像增强方法及系统
CN105654437A (zh) * 2015-12-24 2016-06-08 广东迅通科技股份有限公司 一种对低照度图像的增强方法
US20170256092A1 (en) * 2016-03-02 2017-09-07 Fuji Xerox Co., Ltd. Image processing apparatus, image processing system, image processing method, and non-transitory computer readable medium
CN107578383A (zh) * 2017-08-29 2018-01-12 北京华易明新科技有限公司 一种低照度图像增强处理方法
CN108305236A (zh) * 2018-01-16 2018-07-20 腾讯科技（深圳）有限公司 图像增强处理方法及装置
CN109785263A (zh) * 2019-01-14 2019-05-21 北京大学深圳研究生院 一种基于Retinex的逆色调映射图像转换方法

Also Published As

Publication number Publication date
US20220051375A1 (en) 2022-02-17

Similar Documents

Publication Publication Date Title
WO2020186372A1 (zh) 一种逆色调映射方法、系统、设备及计算机可读介质
CN112036236B (zh) 一种基于GhostNet的图像检测方法、设备及介质
WO2019144855A1 (zh) 图像处理方法、存储介质和计算机设备
JP2023523047A (ja) 画像処理方法、装置、コンピュータ機器及び記憶媒体
WO2020001022A1 (zh) 图片展示方法、计算设备及计算机存储介质
CN110544214A (zh) 一种图像修复方法、装置及电子设备
CN111353939B (zh) 一种基于多尺度特征表示与权值共享卷积层的图像超分辨率方法
CN114299088A (zh) 图像处理方法及装置
CN117635822A (zh) 一种模型训练的方法、装置、存储介质及电子设备
CN117372781A (zh) 一种基于持续学习的图像分类方法及装置
US20250191122A1 (en) Image generation method for eliminating splicing seams, computer device and storage medium
CN116485944A (zh) 图像处理方法及装置、计算机可读存储介质和电子设备
CN117808857B (zh) 一种自监督360°深度估计方法、装置、设备及介质
CN116884005B (zh) 一种结合卷积网络与图特征的快速语义分割方法与系统
CN114359935A (zh) 一种模型训练以及表格识别方法及装置
CN107613046B (zh) 滤镜管道系统、图像数据处理方法、装置以及电子设备
JP7826519B2 (ja) 画像ノイズ低減処理方法、装置、デバイス、記憶媒体及びプログラム製品
CN113888415B (zh) 一种模型训练以及图像修复方法及装置
CN107609051A (zh) 一种图像渲染方法、装置以及电子设备
CN112001268A (zh) 人脸校准方法及设备
TWI804850B (zh) 多深度資訊之融合方法與融合系統
CN116341642A (zh) 一种数据处理的方法、装置、存储介质及电子设备
CN115512033A (zh) 一种3d场景渲染方法、装置、电子设备及存储介质
US20260024174A1 (en) Image reconstruction model training and image reconstruction
JP7577899B1 (ja) 合成写真データを伝送する方法、装置、システムおよび電子デバイス

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19920493

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19920493

Country of ref document: EP

Kind code of ref document: A1