CN113129236A - Single low-light image enhancement method and system based on Retinex and convolutional neural network - Google Patents
Single low-light image enhancement method and system based on Retinex and convolutional neural network Download PDFInfo
- Publication number
- CN113129236A CN113129236A CN202110449727.8A CN202110449727A CN113129236A CN 113129236 A CN113129236 A CN 113129236A CN 202110449727 A CN202110449727 A CN 202110449727A CN 113129236 A CN113129236 A CN 113129236A
- Authority
- CN
- China
- Prior art keywords
- component
- image
- neural network
- illumination
- retinex
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 36
- 238000005286 illumination Methods 0.000 claims abstract description 68
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000003062 neural network model Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 36
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 10
- 238000005457 optimization Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 19
- 238000012545 processing Methods 0.000 description 17
- 230000000694 effects Effects 0.000 description 13
- 238000012800 visualization Methods 0.000 description 11
- 235000008733 Citrus aurantifolia Nutrition 0.000 description 4
- 235000011941 Tilia x europaea Nutrition 0.000 description 4
- 239000004571 lime Substances 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000014759 maintenance of location Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 1
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- RTAQQCXQSZGOHL-UHFFFAOYSA-N Titanium Chemical compound [Ti] RTAQQCXQSZGOHL-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000009916 joint effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012113 quantitative test Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of computer vision images, and provides a single low-illumination image enhancement method and system based on Retinex and a convolutional neural network. The method comprises the steps of obtaining an image, preprocessing the image, separating three channels, and obtaining a hue component, a saturation component and a brightness component; obtaining an illumination component according to the lightness component by adopting the trained deep convolution neural network model; calculating a reflection component of the illumination component by utilizing a Retinex theory; and recombining the reflection component with the hue component and the saturation component to obtain a three-channel image in the HSV color space.
Description
Technical Field
The invention belongs to the technical field of computer vision images, and particularly relates to a single low-illumination image enhancement method and system based on Retinex and a convolutional neural network.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
With the development of science and technology, it is a common means to acquire information from images. Many computer vision algorithms such as object detection, recognition and tracking are becoming more common. However, not all images acquired by the sensors can be used directly for these algorithms. For example, the quality of the image acquired in a low-light environment is degraded, such as low visibility, low contrast, color distortion, high noise, and the like. The information that humans can get through these photographs is very little. The direct application of these photographs to computer vision algorithms can affect the performance of the algorithms. Therefore, how to improve the quality of images acquired under complex environments is a research focus in recent years in the field of computer vision.
A low-light image generally refers to an image of low quality that is acquired in a low-light environment. The main purpose of low-light image enhancement is to increase the brightness and contrast of a low-light image (or an underexposed image), highlight the main information of the image, and mainly realize the technology from the perspective of software. The enhanced image can be better used for computer vision tasks and can provide valuable information. The low-illumination image enhancement method has wide application prospects in the fields of monitoring, automatic driving and the like.
Traditional low-illumination image enhancement methods are mainly classified into four categories: (1) a method for enhancing images based on spatial domain. Such methods mainly change the distribution range of image pixel values to achieve image enhancement. Such methods are mostly based on histogram equalization, gamma correction and fuzzy logic transformations, etc., such as histogram equalization and limited contrast adaptive histogram equalization. (2) Transform domain based image enhancement methods, such methods mainly convert the image into the frequency domain where it is enhanced with a suitable filter function. Most of the methods are based on frequency domain and wavelet transform domain, and the used filter functions mainly include low-pass filter, band-pass filter, high-pass filter and the like. A representative method is a wavelet transform method. (3) An image enhancement method based on image fusion. The algorithm mainly realizes image enhancement in a mode of fusing a plurality of images. The method is based on a plurality of different images of the same scene, and combines parts with good visual effect in the different images to form a high-brightness image. Representative methods such as high dynamic lighting rendering. (4) An image enhancement method based on Retinex theory. The Retinex theory refers to that an image is formed by the joint action of illumination and an object, and the image can be expressed by the product of illumination components and reflection components, wherein the reflection components are inherent properties of the object and have consistent invariance under different illumination conditions. Most of these methods use the original image to obtain the reflection component (also the illumination component and the reflection component). Representative methods are single-scale Retinex using gaussian filtering of the illumination component of the image; the method comprises the following steps of performing Gaussian filtering on an image in multiple scales and increasing a color influence factor to obtain a multi-scale Retinex with color recovery; the illumination component is estimated using a structural prior, the reflection component is solved using Retinex theory and used as LIME of the final result, and so on.
The methods can achieve better effect on partial images, but are limited by models, lack of generalization capability and difficult to be applied to wider scenes.
Convolutional neural networks have great advantages in processing image tasks, have been applied to various types of computer vision tasks, and have been with great success. In the field of low-light image processing, there are also many methods based on convolutional neural networks. LLNet establishes an autoencoder to enhance the image; msr-net realizes image enhancement by learning the mapping relation between light and dark images; Retinex-Net establishes a decomposition network and an enhancement network, trains the network using a paired image dataset, and the like. The method has strong generalization capability and can adapt to a plurality of scenes, but the effect of the method has a great relationship with a paired data set used for training a network model, the establishment of the bright and dark image to the data set is a difficult work, and the brightness of a real image (namely a normal image) does not have a specific standard.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention provides a method and a system for enhancing a single low-light image based on Retinex and a convolutional neural network, which can effectively enhance the single low-light image, do not generate color distortion, can keep the texture details of an original image, and have good generalization capability on different data sets.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a single low-light image enhancement method based on Retinex and a convolutional neural network.
A single low-light image enhancement method based on Retinex and a convolutional neural network comprises the following steps:
acquiring an image, preprocessing the image, separating three channels and acquiring a hue component, a saturation component and a brightness component;
obtaining an illumination component according to the lightness component by adopting the trained deep convolution neural network model;
calculating a reflection component of the illumination component by utilizing a Retinex theory;
and recombining the reflection component with the hue component and the saturation component to obtain a three-channel image in the HSV color space.
And converting the three-channel image in the HSV color space into an RGB color space to obtain an enhanced low-illumination image.
Further, the process of obtaining the enhanced low-light image comprises: and converting the three-channel image in the HSV color space into an RGB color space, and adjusting the pixel range of the three-channel image from [0,1] to [0,255] to obtain the enhanced low-illumination image.
Further, the process of obtaining the hue component, the saturation component and the brightness component includes: normalizing the pixels of the image to [0,1], converting the normalized image from a color space RGB to an HSV space, and separating three channels to obtain hue components, saturation and brightness components.
Further, the process of training the deep convolutional neural network model includes:
constructing a deep convolutional neural network model, and establishing a target loss function of the neural network based on Retinex theory and prior hypothesis;
obtaining a logarithm brightness component after the brightness component is subjected to logarithm and normalization, and obtaining a bright channel prior component through the brightness component;
and (3) bringing the illumination component, the lightness component and the bright channel prior component into a target loss function, calculating an error, and realizing gradient updating on the weight and the parameters of the neural network by using an Adam optimization algorithm through the target loss function until the error is smaller than a set threshold or iteration reaches a preset number of times, and ending the model training.
The target loss function is:
E＝Lossis+λ1Lossr+λ2Lossrs+λ3Losslc
therein, LossisRepresenting the Loss function of illumination smoothness, LossrRepresenting the reflection Loss function, LossrsRepresenting Loss of reflection smoothness function, LosslcRepresenting the prior loss function of the bright channel, λ1,λ2,λ3Respectively, reflection loss, reflection component smoothness loss, and bright channel prior loss.
As an embodiment, three weight values may be taken: lambda [ alpha ]1＝0.05，λ2＝0.1，λ30.5. It should be noted that the numerical values of the three weight values are only one embodiment of the present invention, and should not be construed as limiting the present invention.
The second aspect of the invention provides a single low-light image enhancement system based on Retinex and a convolutional neural network.
A single low-light image enhancement system based on Retinex and a convolutional neural network comprises:
an acquisition and pre-processing module configured to: acquiring an image, preprocessing the image, separating three channels and acquiring a hue component, a saturation component and a brightness component;
an illumination component obtaining module configured to: obtaining an illumination component according to the lightness component by adopting the trained deep convolution neural network model;
a reflected component obtaining module configured to: calculating a reflection component of the illumination component by utilizing a Retinex theory;
a reassembly module configured to: and recombining the reflection component with the hue component and the saturation component to obtain a three-channel image in the HSV color space.
An output module configured to: and converting the three-channel image in the HSV color space into an RGB color space to obtain an enhanced low-illumination image.
A third aspect of the invention provides a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the Retinex and convolutional neural network-based single-low-illumination image enhancement method as described in the first aspect above.
A fourth aspect of the invention provides a computer apparatus.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the Retinex and convolutional neural network-based single low-light image enhancement method according to the first aspect when executing the program.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, the low-illumination image is converted from the color space RGB to the color space HSV, the channels are separated, only the brightness component is enhanced, the color information is kept in the hue (H) and the saturation (S), and the color distortion in the image enhancing process is avoided.
2. The invention establishes the objective function of the deep learning neural network through Retinex theory and a series of prior, realizes the estimation of illumination component from the brightness component of a single image, further estimates the reflection component, thereby obtaining the enhanced image, completes the unsupervised image enhancement process, and has strong generalization capability.
3. The invention uses fractional order differentiation in the objective function of the neural network, and improves the retention capacity of the enhanced image details.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a schematic flow chart of a single low-light image enhancement method based on Retinex and a convolutional neural network;
FIG. 2 is a diagram of a deep convolutional network architecture in an embodiment of the present invention;
FIG. 3 is a schematic illustration of an object imaging process of the present invention;
FIG. 4(a) is an input low-light image;
FIG. 4(b) is a visualization effect diagram after the image enhancement processing of the DICM data set of the BIMEF algorithm;
FIG. 4(c) is a visualization effect diagram after the image enhancement processing of the DICM data set of the DONG algorithm;
FIG. 4(d) is a visualization effect diagram after the image enhancement processing of the DICM data set of the LIME algorithm;
FIG. 4(e) is a visualization effect diagram after the image enhancement processing of the DICM data set by the MF algorithm;
FIG. 4(f) is a visualization effect diagram after the image enhancement processing of the DICM data set of the NPE algorithm;
FIG. 4(g) is a visualization effect diagram after SIRE algorithm DICM data set image enhancement processing;
FIG. 4(h) is a visualization effect diagram after ULE algorithm DICM data set image enhancement processing;
FIG. 4(i) is a visualization effect diagram after the image enhancement processing of the Retinex-Net algorithm DICM data set;
FIG. 4(j) is a visualization effect diagram after image enhancement processing of the KinD algorithm DICM data set;
fig. 4(k) is a visualization effect diagram after the DICM dataset image enhancement processing according to the method of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Example one
As shown in fig. 1, the embodiment provides a single low-light image enhancement method based on Retinex and convolutional neural network, and the embodiment is illustrated by applying the method to a server, it is understood that the method may also be applied to a terminal, and may also be applied to a system including a terminal and a server, and is implemented by interaction between the terminal and the server. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network server, cloud communication, middleware service, a domain name service, a security service CDN, a big data and artificial intelligence platform, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein. In this embodiment, the method includes the steps of:
step 1: acquiring an image, preprocessing the image, separating three channels and acquiring a hue component, a saturation component and a brightness component;
the method comprises the following steps of preprocessing the acquired low-illumination image:
step 1.1, in the stage of reading in the low-illumination image, the pixel value range of the read image is changed into [0,1] by using a special read-in function; or create a normalization function to normalize pixels of the low-light image from [0,255] to [0,1 ].
Step 1.2: converting the image obtained in the step 1.1 from a color space RGB to a color space HSV, and extracting three channel components: hue (H), saturation (S), lightness (V). The specific implementation method comprises the following steps:
step 1.2.1: calculating the maximum channel (c) of the imagemax) Minimum channel (c)min) And a contrast (Δ), the calculation formula being: c. Cmax＝max(R,G,B)，cmin＝min(R,G,B)，Δ＝cmax-cmin(wherein R, G, B are the three channels of the color space RGB, with values at [0,1]]In (d) of (a);
step 1.2.2: the hue (H), saturation (S), lightness (V) are calculated according to the following formula:
V＝cmax。
step 1.3: the luminance component is logarithmized using a function y ═ log (x +1), and the result is normalized to obtain a logarithmized luminance component (v).
Step 1.4: calculating to obtain a prior component (v) of a bright channel by a formulalight) The formula is as follows:
where Ω is a k × k region centered at (i, j),
and vp,qRepresenting the pixel at the specified location.
Step 2: obtaining an illumination component according to the lightness component by adopting the trained deep convolution neural network model;
the method comprises the following specific steps of constructing a deep convolutional neural network model:
step 2.1: and constructing a deep convolutional network. The network structure of the invention is shown in fig. 2, the input is the lightness component (v) obtained in step 1.3, the output is the illumination component (l), the first 4 layers of the network model are composed of a convolution operation and a ReLU function, the latter layer only has 1 convolution operation, and finally, a sigmoid layer is connected. The specific information of each layer of the network model is as follows:
TABLE 1 deep convolutional network
Step 2.2: and establishing an objective function of the neural network based on Retinex theory and prior hypothesis. Fig. 3 shows the imaging process of an object, which the Retinex theory shows as:
where S represents the acquired (or observed) image, R is the reflected component of the object, L is the illumination component of the environment,
representing multiplication by element. Under the assumption that the spatial variation of the illumination is smooth, a loss of illumination smoothness is proposed:
wherein l is the illumination component output by the network,
is v is1(being positive) derivative of order, v2Is a positive number and N is the total number of pixels of the image. By applying constraint to the fractional order gradient of the illumination component, the illumination component obtained by the neural network is smooth when the space changes.
In order to avoid too large difference between the obtained illumination image and the original image, the reflection loss is designed:
where l is the illumination component of the network output, v is the lightness component of the input, v is the luminance component of the network output2Is a positive number.
In order to make the spatial variation of the reflection component as smooth as possible, to guarantee its clarity and visual effect, a loss of reflection smoothness is proposed:
in order to have the value of the reflection component (R) between [0,1], it is necessary to guarantee that the illumination component (L) is greater than (V), thus proposing a bright channel prior penalty:
wherein v islightIs the bright channel prior component calculated from the luma component v,
in the above loss, the present invention uses fractional order gradient and fractional order differential to improve the retention of image texture details by the model.
And synthesizing the loss functions of all parts, wherein the target loss function of the model is as follows:
E＝Lossis+λ1Lossr+λ2Lossrs+λ3Losslc
wherein λ is1,λ2,λ3The weights are respectively reflection loss, reflection component smoothness loss and bright channel prior loss, and the values of the three weights are respectively: lambda [ alpha ]1＝0.05，λ2＝0.1，λ30.5. The resulting illumination components can be spatially smoothed with 4 lossy constraints, preserving texture details.
Step 2.3: randomly initializing a network weight, and determining a weight optimization algorithm as an Adam optimization algorithm.
Wherein a neural network is trained. Inputting the lightness component (v) obtained in the step 1.3 into a convolution neural network, and obtaining an illumination component (l), a lightness component (v) and a bright channel prior component (v) by the neural networklight) And (3) carrying the error into the objective function in the step 2.2, calculating the error, and realizing gradient updating on the weight and the parameters of the neural network by using an Adam optimization algorithm through the objective function, wherein the iteration is stopped when the error meets the requirement or the iteration reaches a preset number.
And step 3: calculating a reflection component of the illumination component by utilizing a Retinex theory;
step 3.1: and (3) inputting the brightness component (v) obtained in the step (3.1) into the deep convolutional neural network trained in the step (3) to obtain an output illumination component (l).
Step 3.2: and (3) gamma correction is carried out on the illumination component obtained in the step (3.1), and then the lightness component (V) and the illumination component (L) obtained in the step (1.3) are subjected to indexing processing to obtain a lightness component (V) and an illumination component (L) after indexing. Obtaining a calculation formula of the reflection component according to Retinex theory: and R is V/L. The reflection component (R) is calculated by a calculation formula.
And 4, step 4: recombining the reflection component with the hue component and the saturation component to obtain a three-channel image in an HSV color space; and converting the three-channel image in the HSV color space into an RGB color space to obtain an enhanced low-illumination image.
Specifically, the reflection component (R) is used as lightness component (V), and combined with hue (H) and saturation (S) obtained in step 1.2 to restore an image enhanced in color space HSV, and then the image is converted from color space HSV to color space RGB, and then the image value range is changed from [0,1] to [0,255] by a special read-out function or a created mapping function.
Through the steps, the enhanced image can be obtained.
We performed tests on DICM low-light image datasets and evaluated network performance using peak Signal-to-Noise ratio psnr (peak Signal to Noise ratio) and structural similarity ssim (structural similarity index). Meanwhile, visualization and quantitative comparison are carried out with the current advanced algorithm, including a natural color retention enhancement algorithm NPE, a DONG based on defogging, an MF based on fusion, a LIME based on illumination component estimation, a BIMEF based on illumination component estimation and multi-exposure fusion, an SRIE based on reflection component and illumination component estimation, and an ULE, Retinex-Net and KinD based on deep learning.
The environment of the experiment is CPU processor Intel (R) Xeon (R) CPU E5-2620 v3@2.40GHz, memory 128G, video card Nvidia GeForce GTX TITAN X, and video memory 12G. The software system is Windows10, the deep learning framework of the pytorech, python3.8, CUDA version 10.0, cuDNN version 7.4. The development software used was Pycharm2020 and Matlab2020 a.
The learning rate of the parameter setting of this experiment is 0.001, and the total iteration number is 1000.
Fig. 4(a) - (k) illustrate the visual comparison results of different algorithms on a DICM dataset. Fig. 4(a) is an input low-light image, fig. 4(b) -fig. 4(j) are results of other methods, and fig. 4(k) is a method proposed by the present invention. Fig. 4 (b): BIMEF, fig. 4 (d): LIME, fig. 4 (e): MF, fig. 4 (f): NPE, fig. 4 (h): ULE and fig. 4 (i): the Retinex-Net method over-enhances the image and the enhanced image suffers from color distortion. Fig. 4 (c): DONG and fig. 4 (g): the consequences of SIRE are unnatural and in some places, excessive enhancement occurs. As can be seen from the image fig. 4(k), the method of the present embodiment retains the image details, makes the image more natural, and does not generate color distortion and excessive enhancement. By contrast, the method of the invention has better overall effect representation, and fine color and structure can be better recovered.
The quantitative test results on the DICM dataset are as follows:
TABLE 2 quantitative comparison on DICM datasets
Example two
The embodiment provides a single low-light image enhancement system based on Retinex and a convolutional neural network.
A single low-light image enhancement system based on Retinex and a convolutional neural network comprises:
an acquisition and pre-processing module configured to: acquiring an image, preprocessing the image, separating three channels and acquiring a hue component, a saturation component and a brightness component;
an illumination component obtaining module configured to: obtaining an illumination component according to the lightness component by adopting the trained deep convolution neural network model;
a reflected component obtaining module configured to: calculating a reflection component of the illumination component by utilizing a Retinex theory;
a reassembly module configured to: and recombining the reflection component with the hue component and the saturation component to obtain a three-channel image in the HSV color space.
An output module configured to: and converting the three-channel image in the HSV color space into an RGB color space to obtain an enhanced low-illumination image.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the single-low-illumination image enhancement method based on Retinex and convolutional neural network as described in the first embodiment above.
Example four
The embodiment provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to implement the steps in the single-low-light image enhancement method based on Retinex and convolutional neural network as described in the first embodiment.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A single low-light image enhancement method based on Retinex and a convolutional neural network is characterized by comprising the following steps:
acquiring an image, preprocessing the image, separating three channels and acquiring a hue component, a saturation component and a brightness component;
obtaining an illumination component according to the lightness component by adopting the trained deep convolution neural network model;
calculating a reflection component of the illumination component by utilizing a Retinex theory;
and recombining the reflection component with the hue component and the saturation component to obtain a three-channel image in the HSV color space.
2. The single-sheet low-illumination image enhancement method based on Retinex and the convolutional neural network as claimed in claim 1, wherein the three-channel image in HSV color space is converted into RGB color space, resulting in an enhanced low-illumination image.
3. The single low-light image enhancement method based on Retinex and convolutional neural network of claim 2, wherein the process of obtaining the enhanced low-light image comprises: and converting the three-channel image in the HSV color space into an RGB color space, and adjusting the pixel range of the three-channel image from [0,1] to [0,255] to obtain the enhanced low-illumination image.
4. The single low-light image enhancement method based on Retinex and convolutional neural network of claim 1, wherein the process of obtaining hue component, saturation component and brightness component comprises: normalizing the pixels of the image to [0,1], converting the normalized image from a color space RGB to an HSV space, and separating three channels to obtain hue components, saturation and brightness components.
5. The single-low-light image enhancement method based on Retinex and the convolutional neural network as claimed in claim 1, wherein the process of deep convolutional neural network model training comprises:
constructing a deep convolutional neural network model, and establishing a target loss function of the neural network based on Retinex theory and prior hypothesis;
obtaining a logarithm brightness component after the brightness component is subjected to logarithm and normalization, and obtaining a bright channel prior component through the brightness component;
and (3) bringing the illumination component, the lightness component and the bright channel prior component into a target loss function, calculating an error, and realizing gradient updating on the weight and the parameters of the neural network by using an Adam optimization algorithm through the target loss function until the error is smaller than a set threshold or iteration reaches a preset number of times, and ending the model training.
6. The single-low-illumination image enhancement method based on Retinex and convolutional neural network of claim 5, wherein the target loss function is:
E＝Lossis+λ1Lossr+λ2Lossrs+λ3Losslc
therein, LossisRepresenting the Loss function of illumination smoothness, LossrRepresenting the reflection Loss function, LossrsRepresenting Loss of reflection smoothness function, LosslcRepresenting the prior loss function of the bright channel, λ1,λ2,λ3The weights are respectively reflection loss, reflection component smoothness loss and bright channel prior loss, and the values of the three weights are respectively: lambda [ alpha ]1＝0.05，λ2＝0.1，λ3＝0.5。
7. The single-sheet low-illumination image enhancement method based on Retinex and the convolutional neural network as claimed in claim 1, wherein the deep convolutional neural network model comprises five convolutional layers and one sigmoid activation layer.
8. A single low-light image enhancement system based on Retinex and a convolutional neural network is characterized by comprising:
an acquisition and pre-processing module configured to: acquiring an image, preprocessing the image, separating three channels and acquiring a hue component, a saturation component and a brightness component;
an illumination component obtaining module configured to: obtaining an illumination component according to the lightness component by adopting the trained deep convolution neural network model;
a reflected component obtaining module configured to: calculating a reflection component of the illumination component by utilizing a Retinex theory;
a reassembly module configured to: and recombining the reflection component with the hue component and the saturation component to obtain a three-channel image in the HSV color space.
An output module configured to: and converting the three-channel image in the HSV color space into an RGB color space to obtain an enhanced low-illumination image.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for single-low-illumination image enhancement based on Retinex and convolutional neural network according to any of claims 1 to 7.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps in the Retinex and convolutional neural network based single low-light image enhancement method of any of claims 1-7 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110449727.8A CN113129236B (en) | 2021-04-25 | 2021-04-25 | Single low-light image enhancement method and system based on Retinex and convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110449727.8A CN113129236B (en) | 2021-04-25 | 2021-04-25 | Single low-light image enhancement method and system based on Retinex and convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113129236A true CN113129236A (en) | 2021-07-16 |
| CN113129236B CN113129236B (en) | 2022-07-12 |
Family
ID=76779830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110449727.8A Active CN113129236B (en) | 2021-04-25 | 2021-04-25 | Single low-light image enhancement method and system based on Retinex and convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113129236B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114202475A (en) * | 2021-11-24 | 2022-03-18 | 北京理工大学 | Adaptive image enhancement method and system |
| CN114266705A (en) * | 2021-11-04 | 2022-04-01 | 山东师范大学 | Low-light image enhancement method and system |
| CN114283288A (en) * | 2021-12-24 | 2022-04-05 | 合肥工业大学智能制造技术研究院 | Method, system, equipment and storage medium for enhancing night vehicle image |
| CN114862737A (en) * | 2022-06-02 | 2022-08-05 | 大连海事大学 | Underwater image enhancement method based on image reflection component and deep learning model |
| CN116824511A (en) * | 2023-08-03 | 2023-09-29 | 行为科技(北京)有限公司 | Tool identification method and device based on deep learning and color space |
| CN116993636A (en) * | 2023-07-10 | 2023-11-03 | 中国地质大学（武汉） | Image enhancement method and device for underground low-illumination deep stratum empty area |
| CN117853783A (en) * | 2023-12-12 | 2024-04-09 | 济南大学 | Single board defect identification method and system based on deep learning |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080101719A1 (en) * | 2006-10-30 | 2008-05-01 | Samsung Electronics Co., Ltd. | Image enhancement method and system |
| CN106600564A (en) * | 2016-12-23 | 2017-04-26 | 潘敏 | Novel image enhancement method |
| CN110298796A (en) * | 2019-05-22 | 2019-10-01 | 中山大学 | Based on the enhancement method of low-illumination image for improving Retinex and Logarithmic image processing |
| WO2020001164A1 (en) * | 2018-06-28 | 2020-01-02 | Boe Technology Group Co., Ltd. | Image enhancement method and apparatus |
| CN110930341A (en) * | 2019-10-17 | 2020-03-27 | 杭州电子科技大学 | Low-illumination image enhancement method based on image fusion |
| CN112465727A (en) * | 2020-12-07 | 2021-03-09 | 北京邮电大学 | Low-illumination image enhancement method without normal illumination reference based on HSV color space and Retinex theory |
-
2021
- 2021-04-25 CN CN202110449727.8A patent/CN113129236B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080101719A1 (en) * | 2006-10-30 | 2008-05-01 | Samsung Electronics Co., Ltd. | Image enhancement method and system |
| CN106600564A (en) * | 2016-12-23 | 2017-04-26 | 潘敏 | Novel image enhancement method |
| WO2020001164A1 (en) * | 2018-06-28 | 2020-01-02 | Boe Technology Group Co., Ltd. | Image enhancement method and apparatus |
| CN110298796A (en) * | 2019-05-22 | 2019-10-01 | 中山大学 | Based on the enhancement method of low-illumination image for improving Retinex and Logarithmic image processing |
| CN110930341A (en) * | 2019-10-17 | 2020-03-27 | 杭州电子科技大学 | Low-illumination image enhancement method based on image fusion |
| CN112465727A (en) * | 2020-12-07 | 2021-03-09 | 北京邮电大学 | Low-illumination image enhancement method without normal illumination reference based on HSV color space and Retinex theory |
Non-Patent Citations (2)
| Title |
|---|
| LIYUN ZHUANG ET.AL: "Image Enhancement by Deep Learning Network Based on derived image and Retinex", 《2019 IEEE 3RD ADVANCED INFORMATION MANAGEMENT,COMMUNICATES,ELECTRONIC AND AUTOMATION CONTROL CONFERENCE》 * |
| 张红颖 等: "空间的RetinexNet低照度图像增强算法", 《激光与光电子学进展》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114266705A (en) * | 2021-11-04 | 2022-04-01 | 山东师范大学 | Low-light image enhancement method and system |
| CN114202475A (en) * | 2021-11-24 | 2022-03-18 | 北京理工大学 | Adaptive image enhancement method and system |
| CN114283288A (en) * | 2021-12-24 | 2022-04-05 | 合肥工业大学智能制造技术研究院 | Method, system, equipment and storage medium for enhancing night vehicle image |
| CN114862737A (en) * | 2022-06-02 | 2022-08-05 | 大连海事大学 | Underwater image enhancement method based on image reflection component and deep learning model |
| CN116993636A (en) * | 2023-07-10 | 2023-11-03 | 中国地质大学（武汉） | Image enhancement method and device for underground low-illumination deep stratum empty area |
| CN116993636B (en) * | 2023-07-10 | 2024-02-13 | 中国地质大学（武汉） | Image enhancement method and device for underground low-illumination deep stratum empty area |
| CN116824511A (en) * | 2023-08-03 | 2023-09-29 | 行为科技(北京)有限公司 | Tool identification method and device based on deep learning and color space |
| CN117853783A (en) * | 2023-12-12 | 2024-04-09 | 济南大学 | Single board defect identification method and system based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113129236B (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113129236B (en) | Single low-light image enhancement method and system based on Retinex and convolutional neural network | |
| Lv et al. | Attention guided low-light image enhancement with a large scale low-light simulation dataset | |
| US20240062530A1 (en) | Deep perceptual image enhancement | |
| Liu et al. | Survey of natural image enhancement techniques: Classification, evaluation, challenges, and perspectives | |
| CN111079764B (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| Zhou et al. | Multi-scale retinex-based adaptive gray-scale transformation method for underwater image enhancement | |
| Guo et al. | Image dehazing via enhancement, restoration, and fusion: A survey | |
| Wang et al. | Joint iterative color correction and dehazing for underwater image enhancement | |
| Steffens et al. | Cnn based image restoration: Adjusting ill-exposed srgb images in post-processing | |
| Fan et al. | Multi-scale depth information fusion network for image dehazing | |
| CN114581318B (en) | Low-illumination image enhancement method and system | |
| CN114529709A (en) | Method and system for training convolutional neural network | |
| CN116157805A (en) | Camera image or video processing pipeline using neural embedding | |
| Wen et al. | Autonomous robot navigation using Retinex algorithm for multiscale image adaptability in low-light environment | |
| Lei et al. | Low-light image enhancement using the cell vibration model | |
| Wang et al. | Single Underwater Image Enhancement Based on $ L_ {P} $-Norm Decomposition | |
| Saleem et al. | A non-reference evaluation of underwater image enhancement methods using a new underwater image dataset | |
| CN115797205A (en) | Unsupervised single image enhancement method and system based on Retinex fractional order variation network | |
| Zhou et al. | Sparse representation with enhanced nonlocal self-similarity for image denoising | |
| Chen et al. | High-dynamic range, night vision, image-fusion algorithm based on a decomposition convolution neural network | |
| CN117974459A (en) | Low-illumination image enhancement method integrating physical model and priori | |
| CN113628143A (en) | Weighted fusion image defogging method and device based on multi-scale convolution | |
| CN117391987A (en) | Dim light image processing method based on multi-stage joint enhancement mechanism | |
| Singh et al. | Multiscale reflection component based weakly illuminated nighttime image enhancement | |
| Soumya et al. | Self-organized night video enhancement for surveillance systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |