CN116645296A - Non-uniform low-light image enhancement method and system under zero reference sample - Google Patents
Non-uniform low-light image enhancement method and system under zero reference sample Download PDFInfo
- Publication number
- CN116645296A CN116645296A CN202310732969.7A CN202310732969A CN116645296A CN 116645296 A CN116645296 A CN 116645296A CN 202310732969 A CN202310732969 A CN 202310732969A CN 116645296 A CN116645296 A CN 116645296A
- Authority
- CN
- China
- Prior art keywords
- illumination
- image
- map
- enhancement
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 239000013074 reference sample Substances 0.000 title claims abstract description 19
- 238000005286 illumination Methods 0.000 claims abstract description 150
- 230000006870 function Effects 0.000 claims abstract description 66
- 238000013507 mapping Methods 0.000 claims abstract description 57
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 40
- 238000013135 deep learning Methods 0.000 claims abstract description 14
- 238000010586 diagram Methods 0.000 claims description 25
- 230000003044 adaptive effect Effects 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 6
- 230000006835 compression Effects 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 239000000523 sample Substances 0.000 claims description 3
- 230000008569 process Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 7
- 238000009826 distribution Methods 0.000 description 5
- 230000001965 increasing effect Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007500 overflow downdraw method Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000003321 amplification Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000003723 Smelting Methods 0.000 description 1
- 102100029469 WD repeat and HMG-box DNA-binding protein 1 Human genes 0.000 description 1
- 101710097421 WD repeat and HMG-box DNA-binding protein 1 Proteins 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000004800 variational method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a non-uniform low-illumination image enhancement method and a system under a zero reference sample, an image decomposition network is established based on a robust Retinex model and deep learning, an input image is decomposed into an illumination image, a reflection image and a noise image according to the image decomposition network, a self-adaptive brightness mapping function is constructed, an optimal brightness mapping curve corresponding to the illumination image is obtained, the enhanced illumination image is obtained by utilizing the optimal brightness mapping curve corresponding to the illumination image, and the enhanced illumination image is obtained by multiplying the enhanced illumination image and the reflection image pixel by pixel.
Description
Technical Field
The invention mainly relates to the technical field of image processing, in particular to a non-uniform low-illumination image enhancement method and system under a zero reference sample.
Background
The image is an important carrier for information transmission, and the quality of the image directly influences the visual experience of people and the accuracy of information transmission. However, in actual production and life, due to the fact that the acquired images are easy to cause due to factors such as illumination environment and acquisition equipment, the acquired images have the characteristics of low illumination, uneven distribution, multiple noise points and the like, the accuracy of visual experience and visual tasks based on the images is affected, and practical application requirements are difficult to meet. The method has important application value in the fields of night monitoring, industrial detection, automatic driving and the like.
The existing non-uniform low-light image enhancement method mainly comprises the following steps: traditional methods and deep learning methods. The conventional method includes gray mapping method, model method, fusion method, and deep learning method is classified into supervised learning and unsupervised learning according to whether the corresponding image in good environment is required. The gray mapping method comprises histogram equalization, gamma correction and the like, and the whole brightness of the image is improved by carrying out nonlinear stretching on pixels with different brightness in the image. The method is simple to operate, but the problems of noise amplification, image distortion and the like are easy to cause because a physical model of imaging is not considered. The model method comprises an atmospheric scattering model and a Retinex model, and the atmospheric light transmission model and the human eye perception characteristics of the colors and the brightness of the image are respectively utilized, so that a better enhancement effect can be provided, but model parameters are required to be manually set, the algorithm complexity is high, and the optimization process is time-consuming. The fusion method is an enhancement method for fusing a plurality of images with different exposure degrees, and has a good enhancement effect due to the fact that the fusion method has a plurality of image sources with different exposure degrees, but image preprocessing is needed, and the enhancement effect depends on the quality of different images for fusion to a certain extent. The deep learning method learns the mapping relation between the low-light image and the normal-light image through a neural network, or acquires the enhanced image through self-learning of the low-light image, and the method utilizes the strong learning and adaptation capability of the neural network, does not need to manually adjust parameters, but has poor interpretability, and easily causes the problem of uncontrollable noise and distortion.
The invention patent with publication number of CN111798400B discloses a no-reference low-light image enhancement method and system based on generation of an countermeasure network. The data set of the method is unpaired low-light and normal-light image blocks, and the generator network for low-light enhancement and the arbiter network for countermeasure training are trained alternately until Nash equalization is achieved. This approach does not require supervised training of pairs of images, but still relies on normal illumination images in the construction of the training dataset.
The invention patent with the publication number of CN110232661B discloses a low-illumination color image enhancement method based on Retinex and convolutional neural networks, which is characterized in that an RGB color image is decomposed into a reflection image and an illumination image through a decomposition network, denoising and color recovery processing are carried out on the reflection image based on a reflection image recovery network, illumination adjustment parameters provided by an illumination adjustment network and a user are used for enhancing the illumination image, and finally dot multiplication is carried out on the reflection image and the recovered reflection image to obtain an enhancement result. The method can obtain a better enhancement effect, but the illumination adjustment parameters are required to be manually input, and the adaptability of different scenes is not high.
Disclosure of Invention
The non-uniform low-light image enhancement method and system under the zero reference sample provided by the invention solve the technical problem that the existing non-uniform low-light image enhancement effect is poor.
In order to solve the technical problems, the non-uniform low-light image enhancement method under a zero reference sample provided by the invention comprises the following steps:
an image decomposition network is built based on a robust Retinex model and deep learning.
The input image is decomposed into an illumination map, a reflection map, and a noise map according to an image decomposition network.
And constructing an adaptive brightness mapping function to obtain an optimal brightness mapping curve corresponding to the illumination map.
And obtaining the enhanced illumination map by utilizing an optimal brightness mapping curve corresponding to the illumination map.
And multiplying the enhanced illumination map with the reflection map pixel by pixel to obtain an enhanced image.
Further, the robust Retinex model is specifically:
wherein S represents an input image, R, I and N represent a reflection diagram, an illumination diagram and a noise diagram under a robust Retinex model respectively,representing pixel-by-pixel multiplication.
Further, the specific formula of the loss function of the robust Retinex model is:
f dec ＝argminL dec ＝argmin(L rec +λ 1 L ref +λ 2 L ill +λ 3 L n )，
wherein f dec Loss function, L, representing a robust Retinex model dec ,L rec ,L ref ,L ill ,L n Respectively represents a mixed no-reference decomposition loss, a reconstruction loss, a reflection estimation loss, an illumination estimation loss and a noise estimation loss, lambda 1 ,λ 2 ,λ 3 Respectively preset reflection estimation loss weight, illumination estimation loss weight and noise estimation loss weight, wherein S represents an input image, R, I and N represent a reflection diagram, an illumination diagram and a noise diagram under a robust Retinex model respectively 1 Represents L 1 The norm of the sample is calculated, I.I F Represents F norms, c E{ r, g, b }, RGB three channels of the image, H (·) represents histogram equalization,representing the sum of gradients in the horizontal and vertical directions, and β represents a custom constant.
Further, the specific formula of the adaptive luminance mapping function is:
wherein g (·) represents a brightness map curve, I represents an illumination map, tanh (I) represents a hyperbolic tangent function, and the value range is [ -1,1]Omega controls the weight, k, of low light enhancement 1 And k 2 For adjusting the amplitude of the low light enhancement and the high light compression, respectively.
Further, constructing an adaptive luminance mapping function, and obtaining an optimal luminance mapping curve corresponding to the illumination map includes:
according to the self-adaptive brightness mapping function, an illumination enhancement network is established, the illumination enhancement network comprises seven convolution layers and a full connection layer, the convolution layers are connected through a ReLU activation function and a maximum pooling layer, and the full connection layer outputs the optimal parameters omega and k 1 ,k 2 。
Training the illumination enhancement network, and adjusting parameters of the self-adaptive brightness mapping function according to the trained illumination enhancement network, so as to obtain an optimal brightness mapping curve corresponding to the illumination map.
Further, the specific formula of the objective function of the training illumination enhancement network is:
f enh ＝argmin(L E +η 1 L N +η 2 L S )，
wherein f dec Representing an objective function of a luminance enhancement network, L E ,L N ,L S Respectively represent exposure control loss, naturalness loss and illumination smoothness loss, eta 1 ,η 2 Respectively representing the naturalness loss function weight and the illumination smoothness loss function weight, sign (·) represents a sign function, 1 is taken when greater than 0,1 is taken when less than 0, 0 is taken when 0, E represents the exposure, O i And I i Representing the luminance values of the ith image block in the enhanced and original illumination respectively, Ω (i) representing the four-neighborhood space centered on pixel i, O j ,I j Representing luminance values of the four-neighborhood space of the enhanced and original illumination map respectively,representing the sum of gradients in the horizontal and vertical directions of the enhanced illumination map, and β represents a custom constant.
Further, the specific formula for obtaining the enhanced image by multiplying the enhanced illumination map with the reflection map pixel by pixel is as follows:
wherein the method comprises the steps ofRepresenting enhanced images +.>Representation ofThe enhanced illumination map, R, represents the reflection map under the robust Retinex model.
The invention provides a non-uniform low-light image enhancement system under a zero reference sample, which comprises:
the method comprises the steps of a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the non-uniform low-light image enhancement method under a zero reference sample.
The invention aims to solve the problems of low illumination, uneven distribution and more noise of an acquired image caused by factors such as illumination environment, shooting equipment limitation and the like, and provides a non-uniform low illumination image enhancement method under a zero reference sample. In order to enhance the brightness of the image and prevent noise from being amplified, a depth Retinex network is constructed by combining a robust Retinex model and deep learning, and the original image is decomposed into a reflection image, an illumination image and a noise image. By quantitatively describing noise components contained in an image, noise in the image is suppressed. In order to solve the problems of low brightness and uneven distribution of an image, a self-adaptive brightness mapping curve is provided, a brightness enhancement network is constructed to obtain optimal mapping curve parameters according to an input illumination graph, different gray scale change rates are distributed to pixels with different brightness at different positions in the image, a brightness enhanced and natural illumination graph is obtained, and the brightness enhanced and natural illumination graph is fused with a noise-free reflection graph to obtain a final enhancement result. Because the invention does not need to be paired with high-quality images and can adaptively adjust the brightness of the images according to different illumination environments, the invention can obtain contrast and brightness improvement and natural enhancement results.
The invention provides a non-uniform low-illumination image enhancement method and a system under a zero reference sample, an image decomposition network is established based on a robust Retinex model and deep learning, an input image is decomposed into an illumination image, a reflection image and a noise image according to the image decomposition network, a self-adaptive brightness mapping function is constructed, an optimal brightness mapping curve corresponding to the illumination image is obtained, the enhanced illumination image is obtained by utilizing the optimal brightness mapping curve corresponding to the illumination image, and the enhanced illumination image is obtained by multiplying the enhanced illumination image and the reflection image pixel by pixel.
The beneficial effects of the invention include:
(1) The image decomposition network is established by combining a robust Retinex model and deep learning, and noise is suppressed in the decomposition process.
(2) A hybrid, non-reference decomposition loss function is designed for directing a decomposition network to decompose an image into a reflection map, an illumination map, and a noise map.
(3) An adaptive brightness mapping curve is constructed, different parameters in the curve are used for controlling the amplitude of low light enhancement and high light compression, and the low light enhancement and high light compression of the illumination pattern can be synchronously realized.
(4) An adaptive illumination enhancement network is established for obtaining parameters of an optimal adaptive luminance mapping curve according to different illumination patterns.
Drawings
FIG. 1 is a flowchart of a non-uniform low-light image enhancement method under a zero reference sample according to a second embodiment of the present invention;
FIG. 2 is an exploded network architecture diagram of a second embodiment of the present invention;
FIG. 3 is a graph of an adaptive luminance mapping according to a second embodiment of the present invention;
FIG. 4 is a schematic diagram of the result of an image enhancement method according to the second embodiment of the present invention, wherein (a) and (b) are the original input image and the enhanced result after enhancement, respectively;
FIG. 5 is a schematic diagram of the result of enhancement of a charge level image according to a third embodiment of the present invention;
fig. 6 is a block diagram of a non-uniform low-light image enhancement system with zero reference samples according to an embodiment of the present invention.
Reference numerals:
10. a memory; 20. a processor.
Detailed Description
The present invention will be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments are shown, for the purpose of illustrating the invention, but the scope of the invention is not limited to the specific embodiments shown.
Embodiments of the invention are described in detail below with reference to the attached drawings, but the invention can be implemented in a number of different ways, which are defined and covered by the claims.
Example 1
The non-uniform low-light image enhancement method under the zero reference sample provided by the embodiment of the invention comprises the following steps:
step S101, an image decomposition network is built based on a robust Retinex model and deep learning.
Step S102, according to the image decomposition network, the input image is decomposed into an illumination map, a reflection map and a noise map.
Step S103, constructing an adaptive brightness mapping function to obtain an optimal brightness mapping curve corresponding to the illumination map.
Step S104, obtaining the enhanced illumination map by utilizing the optimal brightness mapping curve corresponding to the illumination map.
Step S105, the enhanced illumination map is multiplied with the reflection map pixel by pixel to obtain an enhanced image.
According to the non-uniform low-illumination image enhancement method under the zero reference sample, an image decomposition network is established based on a robust Retinex model and deep learning, an input image is decomposed into an illumination image, a reflection image and a noise image according to the image decomposition network, a self-adaptive brightness mapping function is constructed, an optimal brightness mapping curve corresponding to the illumination image is obtained, the enhanced illumination image is obtained by utilizing the optimal brightness mapping curve corresponding to the illumination image, the enhanced illumination image is obtained by multiplying the enhanced illumination image with the reflection image pixel by pixel, the technical problem that the enhancement effect of the existing non-uniform low-illumination image is poor is solved, and as the pair of high-quality images is not needed and the brightness of the image can be self-adaptively adjusted according to different illumination environments, the embodiment of the invention can obtain natural, contrast-enhanced and brightness-enhanced images according to the non-uniform low-illumination image, has strong generalization and can be suitable for different scenes.
Specifically, the invention provides a non-uniform low-illumination image enhancement method under a zero reference sample for solving the problems of non-uniformity, low illumination and noise caused by factors such as illumination environment, acquisition equipment limitation and the like. The method mainly comprises two parts: consider image decomposition of noise and adaptive illumination enhancement. In image decomposition, optimizing the decomposition of a robust Retinex model based on deep learning and mixed non-reference decomposition loss function, and suppressing the amplification of noise by separating out the noise map and the structure of the reflection map and keeping smooth operation; in the illumination enhancement, a self-adaptive brightness mapping curve suitable for non-uniform low illumination enhancement is provided, and parameters of the self-adaptive brightness mapping curve are self-adaptively adjusted through an illumination enhancement network, so that the low illumination enhancement and the high illumination compression of a non-uniform illumination image can be realized at the same time.
Example two
The invention provides a method for solving the problem of non-uniform low-light image enhancement caused by factors such as light environment, shooting equipment limitation and the like, and the whole idea is shown in figure 1, and the method comprises the following steps:
(1) An image decomposition network is established based on a robust Retinex model and deep learning, and noise is suppressed in the decomposition process.
(2) The hybrid no-reference decomposition loss function is designed to direct the image decomposition network to decompose the input image into an illumination map, a reflection map, and a noise map.
(3) An adaptive brightness mapping function is designed in consideration of uneven brightness distribution of an original image, and the amplitude of different brightness adjustment in an illumination graph is controlled by adjusting parameters of the function.
(4) In order to obtain the optimal brightness mapping curve according to different input images, an illumination enhancement network is established, and parameters of the mapping curve are adaptively adjusted according to the input illumination images.
(5) And obtaining an illumination graph with enhanced brightness according to the self-adaptive brightness mapping curve, and multiplying the illumination graph with the reflection graph pixel by pixel to obtain a final enhancement result.
The specific implementation scheme is as follows:
(1) An image decomposition network is established based on a robust Retinex model and deep learning, and noise is suppressed in the decomposition process.
The classical Retinex model separates an image based on color constancy into a reflection map and an illumination map, wherein the reflection map characterizes the high frequency information in the image, which is an inherent property of the image, independent of the illumination conditions, and the illumination map characterizes the low frequency information of the image, determining the maximum dynamic range of the luminance of each pixel.
Wherein, the liquid crystal display device comprises a liquid crystal display device,representing the original non-uniform low-light map, reflection map and illumination map, respectively, < >>Representing pixel-by-pixel multiplication.
Although the classical Retinex model achieves a good effect in image enhancement, it is easy to cause noise of the image to be amplified synchronously during enhancement, since the noise problem in the image is not considered. Considering the process that image denoising can be regarded as separating an image without noise pollution from an original image, the embodiment of the invention adds noise components on the basis of a classical Retinex model, and a robust Retinex model is obtained.
Wherein R, I and N respectively represent a reflection diagram, an illumination diagram and a noise diagram under a robust Retinex model.
The method for decomposing three different components from a single image is a typical ill-posed problem, and most of the existing decomposition methods adopt a variational method, and the method can effectively decompose the different components in the image, but needs to manually adjust parameters according to different images, the complexity of an algorithm is increased, the optimization process takes longer time, and a neural network has strong self-learning capability and self-adaption advantages. The decomposition network comprises 8 layers of convolution layers, wherein the 2 nd layer and the 3 rd layer are connected with the 5 th layer and the 6 th layer through jump connection, and a ReLU is adopted as an activation function of the middle layer so as to avoid gradient disappearance and accelerate training speed. In the output layer of the network, the output channel of the network for estimating the reflection map and the noise map is 3, the activation function adopts Sigmoid and Tanh respectively, so that the reflection map and the noise map are respectively limited in the ranges of [0,1] and [ -1,1], the output channel of the network for estimating the illumination map is 1, and the activation function adopts Sigmoid.
(2) The hybrid no-reference decomposition loss function is designed to direct the image decomposition network to decompose the input image into an illumination map, a reflection map, and a noise map.
In actual production and life, it is generally difficult to obtain a pair of high quality images. In order to effectively decompose the original non-uniform low-light image under the condition of no reference, a mixed non-reference decomposition loss function is designed through reconstruction loss, reflection estimation loss, illumination estimation loss and noise estimation loss.
Wherein L is dec ,L rec ,L ref ,L ill ,L n Respectively represents a mixed no-reference decomposition loss, a reconstruction loss, a reflection estimation loss, an illumination estimation loss and a noise estimation loss, lambda 1 ,λ 2 ,λ 3 The weights of reflection estimation, illumination estimation, and noise estimation losses are respectively represented by constants set in advance.
The different loss functions are specifically expressed as follows:
wherein I II 1 Represents L 1 The norm of the sample is calculated, I.I 1 Representing the F-norm. c ε { r, g, b } represents the RGB three channels of the image. H (·) represents the histogram equalization,the sum of gradients in the horizontal and vertical directions is represented, β represents a constant, and 10 is taken.
L rec : the reconstruction loss is used for measuring the difference between an original image and an image obtained by inverse operation of a robust Retinex model of three components of the decomposition, so as to ensure the effectiveness of the decomposition, and the decomposed result can restore the original image.
L rec : the reflection estimation loss aims to obtain a reflection diagram with high contrast, rich information and noise suppression. The first term of the equation is to ensure that the acquired reflection map contains enough information, and the second term considers that noise in the image is generally less graded, and that the non-noise content is more graded, and that minimizing the second term suppresses noise in the reflection map.
L ill : the illumination estimation loss, the first term of the equation, based on the bright channel prior, regards the illumination map as the maximum brightness channel in the image. The second consideration is to ensure that the image as a whole is smooth from a good illumination map and can reflect the structure of the image. By at least one ofThe term adds a weight associated with the reflectogram such that the illumination smoothness is less weighted where the reflectogram gradient is large and more weighted where the gradient is small.
L n : noise estimation is lost in order to suppress the intensity of noise. By solving the F norm of the point multiplication of the original image and the noise image, the information and the characteristics of the original image can be utilized to restrict the overall strength of the noise.
To decompose the original image into a reflection map, an illumination map, and a noise map, the minimization of the hybrid no-reference loss function is an objective function.
f dec ＝argminL＝argmin(L rec +λ 1 L ref +λ 2 L ill +λ 3 L n ) (5)
Wherein f dec Representing the objective function of decomposing the network.
And obtaining a denoised reflection diagram, an illumination diagram and a noise diagram by optimizing an objective function, as shown in fig. 2.
(3) An adaptive brightness mapping function is designed in consideration of uneven brightness distribution of an original image, and the amplitude of different brightness adjustment in an illumination graph is controlled by adjusting parameters of the function.
Considering that the perception of external brightness by the human eye varies non-linearly, assume B 1 B is an illumination map of the original input image 2 For a light map of a well-exposed image, the mapping relationship between two images can be expressed as:
B 2 ＝g(B 1 ) (6)
where g (-) represents the luminance mapping curve, as shown in fig. 3.
In order to realize natural and reasonable mapping of brightness, the embodiment of the invention provides three conditions which are required to be met by the brightness mapping curve of the non-uniform image: 1) The function is monotonically increasing, and the brightness sequence difference of the output image is consistent with the brightness sequence difference of the input image; 2) The function can enhance the brightness of the low light area, inhibit the brightness of the high light area, and normalize the value range to [0,1] to prevent information overflow; 3) The function is steerable, with the gradient decreasing with increasing brightness for pixels in low light regions and increasing with increasing brightness for pixels in high light regions.
To accommodate the non-uniformity of image brightness and meet the above conditions, an adaptive brightness mapping curve is designed based on the hyperbolic tangent function:
wherein, tan h (I) represents hyperbolic tangent function with value range of [ -1,1]Omega controls the weight, k, of low light enhancement 1 、k 2 The exposure can be simulated for adjusting the amplitude of the low light enhancement and the high light compression, respectively.
(4) In order to obtain the optimal brightness mapping curve according to different input images, an illumination enhancement network is established, and parameters of the mapping curve are adaptively adjusted according to the input illumination images.
In order to adaptively adjust parameters of a brightness mapping curve according to different illumination graphs, a lightweight illumination enhancement network is constructed. The network has simple structure and comprises 7 convolution layers and a full connection layer, wherein the convolution layers are connected through a ReLU activation function and a maximum pooling layer, and the full connection layer outputs the optimal parameters omega and k 1 ,k 2 . And combining the three conditions of the formulated brightness mapping curve to give the illumination enhancement loss function of the network training.
L enc ＝L E +η 1 L N +η 2 L S (9)
Wherein L is E ,L N ,L S The light smoothness loss, η, respectively representing exposure control loss, naturalness loss and structure retention 1 ,η 2 The weights of the different loss functions are shown separately, and constants are taken in the experiment.
In order to increase the brightness of an image and maintain the brightness value near good exposure, the brightness map is divided into K4×4 non-overlapping image blocks, and an exposure loss is designed.
Wherein sign (·) represents a sign function, 1 being taken at greater than 0 and-1 being taken at less than 0, beingAnd taking 0 when the number is 0.E represents good exposure, can be taken [0.5,0.7 ]]。O i And I i Representing the luminance values of the i-th image block in the enhanced and original illumination map, respectively.
In order to maintain consistency of the luminance order difference of the enhanced image with that of the original image, a naturalness loss function is given by measuring the luminance differences of the image block in the four adjacent domains centered on the pixel i and the image block i.
Wherein Ω (i) represents a four-neighborhood space centered on pixel i, O j ,I j Representing luminance values of the four-neighborhood space of the enhanced and original illumination patterns, respectively.
Furthermore, the enhanced luminance map still needs to satisfy the overall smoothness and structure that can reflect the image, for which the illumination estimation loss in the decomposition second term remains consistent, giving a structure-preserving illumination smoothness loss.
Wherein the method comprises the steps ofRepresenting the sum of gradients in the horizontal and vertical directions of the enhanced illumination map.
By minimizing the illumination enhancement loss function, the objective function of the brightness enhancement network is obtained.
f enh ＝argmin(L E +η 1 L N +η 2 L S ) (13)
Wherein f dec Representing the objective function of the brightness enhancement network.
(5) And obtaining an illumination graph with enhanced brightness according to the self-adaptive brightness mapping curve, and multiplying the illumination graph with the reflection graph pixel by pixel to obtain a final enhancement result.
And obtaining a denoised reflection image and an enhanced brightness image, and obtaining an enhanced result according to the inverse operation of the Retinex model.
Wherein the method comprises the steps ofRepresenting the enhanced image. FIG. 4 is a schematic diagram of the result of enhancing a non-uniform low-light image using the method of the present embodiment. Wherein (a) and (b) in fig. 4 represent the original input image and the enhanced enhancement result schematic, respectively.
Example III
In the embodiment, 2650m of a certain ironworks 3 The blast furnace is an experimental platform, and the burden surface image in the blast furnace smelting process is acquired from the site, so that the brightness and contrast of the burden surface image are improved by utilizing the non-uniform low-illumination image enhancement method provided by the invention. The specific implementation steps are as follows:
1. establishing a robust Retinex model of the blast furnace burden surface image, and setting parameters of an image decomposition network, iteration times and the like;
2. obtaining a reflection diagram, an illumination diagram and a noise diagram under different training iteration times by minimizing a mixed non-reference decomposition loss function;
3. continuously updating the reflection graph, the illumination graph and the noise graph until the iteration times or the loss function is minimum;
4. inputting an original illumination map into a brightness enhancement network, setting network parameters, iteration times and the like;
5. continuously optimizing the brightness mapping process by minimizing the loss function of the illumination enhancement network until the set iteration times are reached or the loss function is minimum, so as to obtain the optimal self-adaptive brightness mapping curve parameters;
6. and multiplying the noiseless reflection image with the enhanced brightness image pixel by pixel to obtain a final enhanced structure.
FIG. 5 is a schematic diagram showing the result of image enhancement of a non-uniform low-light level image using the method of this example. It should be noted that, the blast furnace burden surface image enhancement provided by the embodiment of the invention only provides an industrial implementation case, but is not limited to the blast furnace burden surface, and because the embodiment of the invention does not need paired high-quality images as references, and can adaptively adjust the image brightness according to different illumination environments, the contrast and brightness of different illumination environments and application scenes can be improved, and the generalization is good.
Referring to fig. 6, a non-uniform low-light image enhancement system under zero reference samples according to an embodiment of the present invention includes a memory 10, a processor 20, and a computer program stored in the memory 10 and executable on the processor 20, wherein the steps of the non-uniform low-light image enhancement method under zero reference samples according to the present embodiment are implemented when the processor 20 executes the computer program.
The specific working process and working principle of the non-uniform low-light image enhancement system under the zero reference sample of the embodiment can be referred to the working process and working principle of the non-uniform low-light image enhancement method under the zero reference sample of the embodiment.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A method of non-uniform low-light image enhancement with zero reference samples, the method comprising:
establishing an image decomposition network based on a robust Retinex model and deep learning;
according to the image decomposition network, decomposing an input image into an illumination map, a reflection map and a noise map;
constructing a self-adaptive brightness mapping function to obtain an optimal brightness mapping curve corresponding to the illumination map;
obtaining an enhanced illumination map by utilizing an optimal brightness mapping curve corresponding to the illumination map;
and multiplying the enhanced illumination map with the reflection map pixel by pixel to obtain an enhanced image.
2. The non-uniform low-light image enhancement method under zero reference sample according to claim 1, wherein the robust Retinex model is specifically:
wherein S represents an input image, R, I and N represent a reflection diagram, an illumination diagram and a noise diagram under a robust Retinex model respectively,representing pixel-by-pixel multiplication.
3. The non-uniform low-light image enhancement method according to claim 2, wherein the specific formula of the loss function of the robust Retinex model is:
f dec ＝arg minL dec ＝arg min(L rec +λ 1 L ref +λ 2 L ill +λ 3 L n )，
wherein f dec Loss function, L, representing a robust Retinex model dec ,L rec ,L ref ,L ill ,L n Respectively represents a mixed no-reference decomposition loss, a reconstruction loss, a reflection estimation loss, an illumination estimation loss and a noise estimation loss, lambda 1 ,λ 2 ,λ 3 Respectively preset reflection estimated loss weight, illumination estimated loss weight and noise estimated lossWeights, S, represent the input image, R, I, N represent the reflection, illumination and noise maps, respectively, under the robust Retinex model, |·|| 1 Represents L 1 The norm of the sample is calculated, I.I F Representing F norm, c ε { r, g, b }, RGB three channels of the image, H (&) represents histogram equalization,representing the sum of gradients in the horizontal and vertical directions, and β represents a custom constant.
4. A method for non-uniform low-light image enhancement under zero reference samples according to any of claims 1-3, wherein the specific formula of the adaptive luminance mapping function is:
wherein g (·) represents a brightness map curve, I represents an illumination map, tanh (I) represents a hyperbolic tangent function, and the value range is [ -1,1]Omega controls the weight, k, of low light enhancement 1 And k 2 For adjusting the amplitude of the low light enhancement and the high light compression, respectively.
5. The method of non-uniform low-light image enhancement under zero reference samples according to claim 4, wherein constructing an adaptive luminance mapping function to obtain an optimal luminance mapping curve corresponding to a light map comprises:
according to the self-adaptive brightness mapping function, an illumination enhancement network is established, the illumination enhancement network comprises seven convolution layers and a full connection layer, the convolution layers are connected through a ReLU activation function and a maximum pooling layer, and the full connection layer outputs the optimal parameters omega and k 1 ,k 2 ；
Training the illumination enhancement network, and adjusting parameters of the self-adaptive brightness mapping function according to the trained illumination enhancement network, so as to obtain an optimal brightness mapping curve corresponding to the illumination map.
6. The method of non-uniform low-light image enhancement under zero reference samples according to claim 5, wherein the specific formula for training the objective function of the light enhancement network is:
f enh ＝arg min(L E +η 1 L N +η 2 L S )，
wherein f dec Representing an objective function of a luminance enhancement network, L E ,L N ,L S Respectively represent exposure control loss, naturalness loss and illumination smoothness loss, eta 1 ,η 2 Respectively representing the naturalness loss function weight and the illumination smoothness loss function weight, sign (·) represents a sign function, 1 is taken when greater than 0,1 is taken when less than 0, 0 is taken when 0, E represents the exposure, O i And I i Representing the luminance values of the ith image block in the enhanced and original illumination respectively, Ω (i) representing the four-neighborhood space centered on pixel i, O j ,I j Representing luminance values of the four-neighborhood space of the enhanced and original illumination map respectively,representation enhancementThe sum of gradients in the horizontal and vertical directions of the subsequent illumination map, β, represents a custom constant.
7. The method for enhancing a non-uniform low-light image under a zero reference sample according to claim 6, wherein the specific formula for obtaining the enhanced image by multiplying the enhanced light map with the reflection map pixel by pixel is:
wherein the method comprises the steps ofRepresenting enhanced images +.>Representing the enhanced illumination map, R represents the reflection map under the robust Retinex model.
8. A non-uniform low-light image enhancement system under a zero reference sample, the system comprising:
memory (10), a processor (20) and a computer program stored on the memory (10) and executable on the processor (20), characterized in that the processor (20) implements the steps of the method according to any of the preceding claims 1 to 7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310732969.7A CN116645296A (en) | 2023-06-20 | 2023-06-20 | Non-uniform low-light image enhancement method and system under zero reference sample |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310732969.7A CN116645296A (en) | 2023-06-20 | 2023-06-20 | Non-uniform low-light image enhancement method and system under zero reference sample |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116645296A true CN116645296A (en) | 2023-08-25 |
Family
ID=87640054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310732969.7A Pending CN116645296A (en) | 2023-06-20 | 2023-06-20 | Non-uniform low-light image enhancement method and system under zero reference sample |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116645296A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117541489A (en) * | 2023-10-18 | 2024-02-09 | 南京航空航天大学 | Physical driving contrast learning method for low-light image enhancement |
| CN117893455A (en) * | 2024-03-11 | 2024-04-16 | 杭州海康威视数字技术股份有限公司 | Image brightness and contrast adjusting method |
-
2023
- 2023-06-20 CN CN202310732969.7A patent/CN116645296A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117541489A (en) * | 2023-10-18 | 2024-02-09 | 南京航空航天大学 | Physical driving contrast learning method for low-light image enhancement |
| CN117893455A (en) * | 2024-03-11 | 2024-04-16 | 杭州海康威视数字技术股份有限公司 | Image brightness and contrast adjusting method |
| CN117893455B (en) * | 2024-03-11 | 2024-06-04 | 杭州海康威视数字技术股份有限公司 | Image brightness and contrast adjusting method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116645296A (en) | Non-uniform low-light image enhancement method and system under zero reference sample | |
| CN110197463B (en) | High dynamic range image tone mapping method and system based on deep learning | |
| CN112614077B (en) | Unsupervised low-illumination image enhancement method based on generation countermeasure network | |
| CN109343692B (en) | Mobile device display power saving method based on image segmentation | |
| WO2023236565A1 (en) | Low-light image enhancement method based on reinforcement learning and aesthetic evaluation | |
| JP7353803B2 (en) | Image processing device, image processing method, and program | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| CN110009574B (en) | Method for reversely generating high dynamic range image from low dynamic range image | |
| CN115660964A (en) | Nighttime road image enhancement method based on threshold partition weighted brightness component | |
| CN111563854B (en) | Particle swarm optimization method for underwater image enhancement processing | |
| Lei et al. | A novel intelligent underwater image enhancement method via color correction and contrast stretching✰ | |
| CN115984535A (en) | Low-illumination image judgment and image enhancement method for drilling operation site | |
| CN116109509A (en) | Real-time low-illumination image enhancement method and system based on pixel-by-pixel gamma correction | |
| CN114693548B (en) | Dark channel defogging method based on bright area detection | |
| CN116188339A (en) | Retinex and image fusion-based scotopic vision image enhancement method | |
| CN112614063B (en) | Image enhancement and noise self-adaptive removal method for low-illumination environment in building | |
| CN116993616A (en) | Single low-illumination scene image enhancement method and enhancement system | |
| Zhao et al. | Color channel fusion network for low-light image enhancement | |
| He et al. | Low-light image enhancement combined with attention map and u-net network | |
| CN112085668A (en) | Image tone mapping method based on regional self-adaptive self-supervised learning | |
| JP2021086284A (en) | Image processing device, image processing method, and program | |
| US20210374916A1 (en) | Storage medium storing program, image processing apparatus, and training method of machine learning model | |
| CN115937019A (en) | Non-uniform defogging method combining LSD (local Scale decomposition) quadratic segmentation and deep learning | |
| Shi et al. | LCA-Net: A Context-Aware Light-Weight Network For Low-Illumination Image Enhancement | |
| CN111325690B (en) | Self-adaptive underwater image enhancement method based on differential evolution algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |