CN116363001A - Underwater image enhancement method combining RGB and HSV color spaces - Google Patents
Underwater image enhancement method combining RGB and HSV color spaces Download PDFInfo
- Publication number
- CN116363001A CN116363001A CN202310213964.3A CN202310213964A CN116363001A CN 116363001 A CN116363001 A CN 116363001A CN 202310213964 A CN202310213964 A CN 202310213964A CN 116363001 A CN116363001 A CN 116363001A
- Authority
- CN
- China
- Prior art keywords
- scale
- features
- rgb
- underwater image
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 238000012937 correction Methods 0.000 claims abstract description 12
- 230000004927 fusion Effects 0.000 claims abstract description 11
- 230000002708 enhancing effect Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000012935 Averaging Methods 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 description 10
- 238000013135 deep learning Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 6
- 241000196171 Hydrodictyon reticulatum Species 0.000 description 5
- 101100014407 Pisum sativum GDCSP gene Proteins 0.000 description 5
- 238000006731 degradation reaction Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000000889 atomisation Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an underwater image enhancement method combining RGB and HSV color spaces, which comprises the following steps: performing color correction on the original color underwater image; generating a multi-scale underwater image by using a nearest neighbor interpolation method from the underwater image after color correction; sending the multi-scale underwater image into an encoder, and respectively extracting multi-scale features in two color spaces of RGB and HSV; for the multi-scale features extracted in the encoding stage, a selective kernel feature fusion module is used for selectively fusing the features of the same scale from the RGB and HSV color spaces; and decoding and reconstructing the fused features, gradually reconstructing the underwater image from the minimum scale, and finally obtaining the enhanced underwater image. According to the invention, important information such as hue, saturation and the like in the HSV color space is introduced into the algorithm by adopting the RGB and HSV combined double-color space, so that the robustness of the algorithm is enhanced, and the network generalization is improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an underwater image enhancement method combining RGB and HSV color spaces.
Background
High-quality clear underwater images are important guarantees for performing a series of underwater tasks such as marine exploration, monitoring and the like. However, due to the complex imaging mechanisms under water, the underwater images tend to suffer from varying degrees of degradation. First, the attenuation of light under water is different from that on land, and the propagation attenuation of light under water is affected by wavelength. Red light is longest in visible light, so that during propagation, it disappears first, followed by orange, green and blue light. Most underwater images therefore exhibit a bluish-green hue. Secondly, suspended particles in the water body reflect light, and when the reflected light reaches the camera, a scattering effect is generated. The problems of color distortion, detail blurring, insufficient contrast and the like of the underwater image can occur due to the problems of selective attenuation and scattering of light rays.
At present, the underwater image enhancement technology is divided into two types, one type is a traditional method; one type is a deep learning method. The core idea of the traditional method based on the physical model is to build the physical model for the degradation process of the underwater image, and finally obtain the corresponding enhanced image by reversely reasoning the image degradation process after obtaining the parameters in the model. The core idea of the traditional method based on the non-physical model is that the imaging model of the underwater image is not considered, and the pixels in the image are directly adjusted by the methods of histogram stretching or linear transformation, etc., so that the effects of removing the color cast of the image and improving the contrast of the image are achieved. The conventional method relies on a priori knowledge, and the estimation process of parameters is difficult to consider for different underwater scenes, so that the algorithm lacks universality. In recent years, the deep learning is used for amplifying the wonderful colors in the field of image enhancement, the underwater image enhancement is more flexible and universal by using the deep learning method, and the method can be more suitable for changeable underwater scenes.
The current underwater image enhancement method based on deep learning mostly focuses on a single RGB color space, and often limits the enhancement performance of the algorithm. The RGB color space contains only basic image information, and lacks information about color degradation and insufficient contrast of the underwater image. Meanwhile, a large number of paired samples are needed for training the deep learning algorithm, and the problem of insufficient generalization is often caused by using the deep learning algorithm to enhance the underwater image at present.
Disclosure of Invention
The invention aims to solve the technical problems and defects in the prior art and provides an underwater image enhancement method combining RGB and HSV color spaces.
The method of the invention introduces important information such as hue, saturation and the like in the HSV color space into the algorithm by adopting the RGB and HSV combined double color space so as to enhance the robustness of the algorithm. Meanwhile, in order to solve the problem of uneven distribution of insufficient data, the color correction is performed on the data before the data is sent to a deep learning network so as to improve the generalization performance of the network to the greatest extent. The specific method comprises the steps of firstly smoothing an input image, then adopting a multi-scale input mode to extract information from RGB and HSV spaces respectively, finally enhancing an underwater image, correcting various color cast problems and improving contrast.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a method of underwater image enhancement combining RGB and HSV color spaces, comprising the steps of:
step one: performing color correction on the original color underwater image;
step two: generating a multi-scale underwater image by using a nearest neighbor interpolation method from the underwater image after color correction;
step three: sending the multi-scale underwater image into an encoder, and respectively extracting multi-scale features in two color spaces of RGB and HSV;
step four: selectively fusing the multi-scale features extracted in the encoding stage by using a selective kernel feature fusion module SKFF (selective kernel feature fusion module), wherein the features are from the same scale in the RGB and HSV color spaces;
step five: and decoding and reconstructing the fused features, gradually reconstructing the underwater image from the minimum scale, and finally obtaining the enhanced underwater image.
Further preferably, in the first step, the specific method and the specific steps for performing color correction on the original color underwater image are as follows:
(1) Dividing an input color underwater image X into three channels of red, green and blue according to channel dimensions, and sequentially marking the three channels as X red ，X green ，X blue . Respectively averaging the three channels, and sequentially marking the obtained averages as mu r ，μ g ，μ b . Wherein mu r ＝mean(X red )，μ g ＝mean(X green )，μ b ＝mean(X blue ) Mean () represents the averaging function, i.e
Wherein M and N represent the width and height of the channel, x, respectively i,j Representing the pixel value in the channel with coordinates (i, j).
(2) Sequencing the obtained average value from small to large to obtain a minimum average value mu low Median average mu mid Maximum mean mu high . Wherein mu low ,μ mid ,μ high ＝sort(μ r ,μ g ,μ b ) Sort () represents the ranking function.
(3) And respectively judging the average value of the red, green and blue channels. Adding to each element in the channel of the minimum mean a numerical difference Deltaμ between the median mean and the minimum mean 1 ，Δμ 1 ＝μ mid -μ low The method comprises the steps of carrying out a first treatment on the surface of the The difference Δμ between the maximum and median means is subtracted for each element in the maximum mean channel 2 ，Δμ 2 ＝μ high -μ mid 。
(4) Combining the three red, green and blue channels to obtain a color corrected underwater image operates as follows: z=concat (X red ,X green ,X blue ) Wherein Z represents a color correctedUnderwater image, concat () stands for combining red, green and blue channels into a color image by channel dimension.
Further preferably, in the second step, the specific method for generating the multi-scale underwater image by using the nearest neighbor interpolation method is as follows:
scale_2＝interpolate(input＝scale_1,scale_factor＝0.5,mode＝'nearest')
scale_4＝interpolate(input＝scale_1,scale_factor＝0.25,mode＝'nearest')
wherein, interpolation () represents an interpolation function that accepts three parameters, namely, an input image input, a scale factor_factor, and a scale mode. scale_1 represents the input image, i.e., the original size; scale_2 represents a 2-fold downsampled image of the original size image; scale_4 represents a 4 times downsampled image of the original size image.
In the third step, the specific method for respectively extracting the multi-scale features in the two color spaces of RGB and HSV is as follows:
(1) Firstly, extracting features of an underwater image with scale_1 of an RGB color space by using 3X 3 convolution operation, and then sending the features into a coding block formed by residual convolution to obtain extracted features rgb_encoded_1 of an original scale;
(2) To stabilize the network gradient, first, the shallow convolution module SCM (shallow convolutional module) is used to extract the scale_2 scale underwater image of the RGB color space to obtain the shallow feature rgb_scm_2, meanwhile, the maximum pooling operation with the step length of 2 is used to perform downsampling on the rgb_encoded_1 feature to obtain the downsampled feature mp_rgb_encoded_1, then the shallow feature rgb_scm_2 and the downsampled feature mp_rgb_encoded_1 are sent to the feature attention module FAM (feature attention module) to obtain the feature rgb_fam_2, and finally, the encoding block is used to extract the feature to obtain the feature rgb_encoded_2 downsampled by 2 times of the original size. Likewise, the same operation as that of the scale_2 underwater image is performed on the scale_4 underwater image in the RGB color space, and the feature rgb_encoded_4 downsampled by 4 times the original size is obtained.
(3) Extracting underwater image features in an HSV color space, firstly converting an underwater image with a color space of RGB into the HSV color space, and then performing the same operation as in the step (2). Likewise, three scale features, hsv_endcoded_1, hsv_endcoded_2, hsv_endcoded_4, are extracted under HSV color space.
Further preferably, in the fourth step, the specific method for selectively fusing the features of the same scale in the two color spaces of RGB and HSV by using a selective kernel feature fusion module (SKFF) for the multi-scale features extracted in the encoding stage is as follows:
(1) Element-by-element addition of features of different color spaces of the same scale is performed, and then a global pooling operation is used to obtain an average value vector of each channel
C is the number of channels of the feature.
(2) Deriving compact channel characterization from s using convolution and PReLU activation functions
C is the number of channels of the feature.
(3) Feeding z into two parallel convolution layers to obtain two feature descriptors v 1 And v 2 ，v 1 ,
C is the number of channels of the feature, and then v is found separately using the self-attention mechanism 1 And v 2 Attention score s of (2) 1 Sum s 2 The specific operation is as follows:
c is the number of channels of the feature.
(4) Using the attention score s 1 Sum s 2 For input features L 1 And L 2 Selecting and fusing, wherein the method comprises the following steps: u=s 1 ·L 1 +s 2 ·L 2 . Wherein U represents the output of the SKFF module, L 1 And L 2 Representing the characteristics of different color spaces of the same scale, i.e. the two inputs of the SKFF module, such as rgb_encoded_1 and hsv_encoded_1, respectively. Finally, the SKFF module outputs three scale features containing rich semantic information, namely skff_scale_1, skff_scale_2, and skff_scale_4.
Further preferably, in the fifth step, the specific method of sending the fused features to a decoder, gradually reconstructing the underwater image from the smallest scale, and finally obtaining the enhanced underwater image is as follows:
(1) The decoding block formed by residual convolution is used for decoding the features of the skff_scale_4 scale, the reconstructed features are sent into a convolution layer to stabilize gradients, the decoded features de_scale_4 are obtained, and then the decoded features de_scale_up_2 are obtained by up-sampling 2 times by using a nearest neighbor interpolation method.
(2) And (3) splicing the features of the skff_scale_2 scale to the decoded features de_scale_up_2 after the upsampling in the step (1) according to the dimension of the channel, and repeating the step (1) to obtain the decoded features de_scale_up_1.
(3) Splicing the features of the skff_scale_1 scale to the decoded features de_scale_up_1 after up-sampling in the step (2) according to the dimension of the channel, then sending the features into a decoding block and finally obtaining the decoded features de_scale_1 in a dimension-reducing convolution layer, and finally adding the decoded features de_scale_1 with the original input color underwater image element by element to obtain the enhanced underwater image.
Advantages and beneficial effects of the invention
The method of the invention introduces important information such as hue, saturation and the like in the HSV color space into the algorithm by adopting the RGB and HSV combined double color space so as to enhance the robustness of the algorithm. Meanwhile, in order to solve the problem of uneven distribution of insufficient data, the color correction is performed on the data before the data is sent to a deep learning network so as to improve the generalization of the network to the greatest extent.
Drawings
FIG. 1 is a general flow chart of an embodiment of the present invention.
Fig. 2 shows the experimental results of the algorithm of the present invention and other image enhancement algorithms in the UIEB dataset.
FIG. 3 shows the results of the algorithm of the present invention and other image enhancement algorithms in synthesizing underwater image datasets.
Detailed Description
As shown in fig. 1, an underwater image enhancement method combining RGB and HSV color spaces includes the following steps:
step one: the method for correcting the color of the original color underwater image comprises the following specific steps:
firstly, an input color underwater image X is divided into three channels of red, green and blue according to the channel dimension, and is marked as X red ，X green ，X blue . And respectively calculating the average value of the three channels, wherein the calculated average value is recorded as mu r ，μ g ，μ b . Wherein mu r ＝mean(X red )，μ g ＝mean(X green )，μ b ＝mean(X blue ) Mean () represents the averaging function, i.e
Wherein M and N represent the width and height of the channel, x, respectively i,j Representing the pixel value in the channel with coordinates (i, j). Then sorting the obtained average value from small to large to obtain the minimum average value mu low Median average mu mid Maximum mean mu high . Wherein mu low ,μ mid ,μ high ＝sort(μ r ,μ g ,μ b ) Sort () represents the ranking function. And then, respectively judging the average value of the red, green and blue channels. Adding to each element in the channel of the minimum mean a numerical difference Deltaμ between the median mean and the minimum mean 1 ，Δμ 1 ＝μ mid -μ low The method comprises the steps of carrying out a first treatment on the surface of the The difference Δμ between the maximum and median means is subtracted for each element in the maximum mean channel 2 ，Δμ 2 ＝μ high -μ mid . Then, the three channels of red, green and blue are combined to obtain a color corrected underwater image, which operates as follows: z=concat (X red ,X green ,X blue ) Wherein Z represents color corrected waterLower image, concat () represents the merging of red, green and blue channels into a color image by channel dimension.
Step two: generating a multi-scale underwater image by using a nearest neighbor interpolation method for the underwater image after color correction, wherein the specific content and the operation method are as follows:
scale_2＝interpolate(input＝scale_1,scale_factor＝0.5,mode＝'nearest')
scale_4＝interpolate(input＝scale_1,scale_factor＝0.25,mode＝'nearest')
wherein, interpolation () represents an interpolation function that accepts three parameters, namely, an input image input, a scale factor_factor, and a scale mode. scale_1 represents the input image, i.e., the original size; scale_2 represents a 2-fold downsampled image of the original size image; scale_4 represents a 4 times downsampled image of the original size image.
Step three: sending the multi-scale underwater image into an encoder, and respectively extracting multi-scale features in two color spaces of RGB and HSV; the specific content and method steps are as follows:
(1) Extracting features of which the underwater image scale is scale_1 in the RGB color space by using 3X 3 convolution operation, and then sending the features into a coding block formed by residual convolution to obtain extracted features rgb_encoded_1 of an original scale;
(2) In order to stabilize the network gradient, firstly, extracting an underwater image of scale_2 scale of RGB color space by using a shallow convolution module SCM (shallow convolutional module) to obtain shallow feature rgb_scm_2, simultaneously, downsampling the feature rgb_encoded_1 by using a maximum pooling operation with a step length of 2 to obtain downsampled feature mp_rgb_encoded_1, then, sending the shallow feature rgb_scm_2 and the downsampled feature mp_rgb_encoded_1 into a feature attention module FAM (feature attention module) to obtain feature rgb_fam_2, and finally, extracting the feature by using a coding block to obtain the feature rgb_encoded_2 downsampled by 2 times of the original size; likewise, performing the same operation as that of the scale_2 underwater image in the RGB color space, and obtaining a feature rgb_encoded_4 downsampled by 4 times of the original size;
(3) Extracting underwater image features in an HSV color space, firstly converting an underwater image with a color space of RGB into the HSV color space, and then performing the same operation as in the step (2); likewise, three scale features, hsv_endcoded_1, hsv_endcoded_2, hsv_endcoded_4, are extracted under HSV color space.
Step four: selectively fusing the multi-scale features extracted in the encoding stage by using a selective kernel feature fusion module SKFF (selective kernel feature fusion module), wherein the features are from the same scale in the RGB and HSV color spaces; the specific content and method steps are as follows:
(1) Element-by-element addition of features of different color spaces of the same scale is performed, and then a global pooling operation is used to obtain an average value vector of each channel
C is the number of channels of the feature;
(2) Deriving compact channel characterization from s using convolution and PReLU activation functions
C is the number of channels of the feature;
(3) Feeding z into two parallel convolution layers to obtain two feature descriptors v 1 And v 2 ，v 1 ,
C is the number of channels of the feature, and then v is found separately using the self-attention mechanism 1 And v 2 Attention score s of (2) 1 Sum s 2 The specific operation is as follows:
c is the number of channels of the feature;
(4) Using the attention score s 1 Sum s 2 For input features L 1 And L 2 The specific content and method for selecting and fusing are as follows: u=s 1 ·L 1 +s 2 ·L 2 The method comprises the steps of carrying out a first treatment on the surface of the Wherein U represents the output of the SKFF module，L 1 And L 2 Two inputs representing the same scale and different color spaces, namely SKFF module, such as rgb_encoded_1 and hsv_encoded_1;
(5) The SKFF module outputs three scales of features containing rich semantic information, namely skff_scale_1, skff_scale_2, and skff_scale_4.
Step five: decoding and reconstructing the fused features, gradually reconstructing the underwater image from the minimum scale, and finally obtaining the enhanced underwater image, wherein the specific contents and the method steps are as follows:
(1) Decoding the features of the skff_scale_4 scale by using a decoding block formed by residual convolution, then sending the reconstructed features into a convolution layer to stabilize gradients to obtain decoded features de_scale_4, and then up-sampling the decoded features 2 times by using a nearest neighbor interpolation method to obtain decoded features de_scale_up_2;
(2) Splicing the features of the skff_scale_2 scale to the decoded features de_scale_up_2 sampled in the step (1) according to the dimension of the channel, and repeating the step (1) to obtain the decoded features de_scale_up_1;
(3) Splicing the features of the skff_scale_1 scale to the decoded features de_scale_up_1 after up-sampling in the step (2) according to the dimension of the channel, then sending the features into a decoding block and finally obtaining the decoded features de_scale_1 in a dimension-reducing convolution layer, and finally adding the decoded features de_scale_1 with the original input color underwater image element by element to obtain the enhanced underwater image.
Examples
The following describes embodiments of the invention by performing an underwater image enhancement example on a subset of the test set challengingset of the underwater image dataset UIEB:
(1) Color correction: the input image is preprocessed prior to inputting the original degraded underwater image into the deep learning based network architecture. The most attenuated channels are numerically compensated and the least attenuated channels are numerically limited. Firstly, calculating the average value of three channels, then adding the difference between the median average value and the minimum average value to the channel with the minimum average value, and adding the difference between the maximum average value and the median average value to the channel with the maximum average value. The color corrected image is smoother.
(2) Feature extraction: the network architecture of the present invention is an asymmetric codec architecture, meaning that there are two encoders and only one decoder. In the encoding stage, the present invention extracts features from both RGB and HSV color spaces, respectively. First, a coding feature 1 is obtained for an image of scale 1. And performing shallow convolution operation on the image of the scale 2 to stabilize the gradient, then maximizing the pool of the coding features 1 in the previous step to the same size as the scale 2, sending the two to a feature attention module together for fusion, and then coding the fused features to obtain the coding features 2. The scale 3 processing is consistent with the scale 2 processing, so that 3 different scale coding features can be obtained in the RGB color space. The processing in HSV color space is the same as in RGB color space, so that 3 different scale coding features are also available. A total of 3 different scale 6 coding features are reviewed from the RGB and HSV color spaces, respectively.
(3) Feature selection and fusion: the invention uses the SKFF module to specially select the coding features from different color spaces with the same scale, and selects the most representative features and fuses the features.
(4) And (3) feature reconstruction: the invention gradually reconstructs the image upwards from the fusion characteristic of the scale 3, and acquires the enhanced underwater image.
(5) The method of the invention is compared with other 8 underwater image enhancement algorithms, and the results are shown in table 1;
by comparison with other methods, the method of the invention can be verified to have good superiority compared with other 8 methods: the color cast problem of the underwater image is effectively corrected; the contrast ratio of the underwater image is effectively improved. In addition, the method can well correct various rare color cast.
TABLE 1
| Algorithm | MSE | PSNR | SSIM |
| UDCP | 3787.4201 | 14.9353 | 0.7053 |
| UIBLA | 4193.7638 | 14.5837 | 0.672 |
| GDCP | 3664.5848 | 14.8305 | 0.6471 |
| ODM | 4674.3138 | 12.9323 | 0.6243 |
| Water-Net | 2918.8692 | 16.2425 | 0.7809 |
| UWCNN_Ⅰ | 2158.1204 | 18.6012 | 0.7990 |
| UWCNN_retrain | 1644.1489 | 17.7971 | 0.8036 |
| FUnIE | 3644.5658 | 14.9029 | 0.6708 |
| Ours | 1028.8371 | 20.6022 | 0.8677 |
The real scene underwater image data set UIEB is a high-quality underwater image data set, and the data set covers a wide scene and has higher image quality. The number of synthetic data sets synthesized according to the underwater image imaging principle is large, and the quality is good. The method selects 60 reference-free images and 1000 artificially synthesized underwater images of a real scene underwater image data set UIEB as a test set. And comparing the method of the invention with other most advanced underwater image enhancement methods:
comparison of experimental results of 60 reference-free underwater images at UIEB:
the 60 reference-free underwater images of the UIEB are one image including 60 underwater images having different chromatic aberration, a low-contrast underwater image and a fogged underwater image. The method of the present invention selects 8 enhancement methods and compares the enhancement results of the method of the present invention with all enhancement results contained therein, as shown in fig. 2.
In fig. 2, the enhancement results of the method of the present invention and the enhancement results of 8 methods are shown in the left-most side (a) of the original image, and the rest of the original image is from left to right from (b) UDCP, (c) ODM, (d) uipla, (e) GDCP, (f) Water-Net, (g) uwcnn_typei, (h) uwcnn_retraine, (i) FUnIE, (j) the method of the present invention, respectively.
By image contrast analysis:
(b) UDCP only eliminates the fogging effect of the image and does not correct the severe color cast problem of the underwater image.
(c) ODM overcompensates for the red channel value, resulting in a large area of red texture in the image.
(d) Uilbla fails to correct the color cast phenomenon of underwater images, only moderating the fogging effect of the images.
(e) The enhanced image fogging phenomenon of GDCP is more remarkable.
(f) The enhancement effect of Water-Net is overall darker.
(g) Uwcnn_typei has poor enhancement effect for low-luminance underwater images.
(h) Uwcnn_retrain fails to remove the fogging effect of underwater images.
(i) The enhancement effect of the FUnIE is orange.
(j) The method can well correct various chromatic aberration problems, and the visual effect of the enhanced image is better.
To further demonstrate the performance of the method of the present invention, the method of the present invention was further tested on 1000 artificial synthetic underwater image datasets.
The artificially synthesized underwater image data set is an image synthesized according to the underwater imaging principle, the visual effect of the image is close to that of an underwater image in a real scene, 8 enhancement methods are selected by the method, and the enhancement results of the method are compared with all enhancement results contained in the enhancement results, as shown in fig. 3.
In fig. 3, the left-most side (a) is the original image, and the rest of the images are from left to right respectively from (b) UDCP, (c) ODM, (d) uipla, (e) GDCP, (f) Water-Net, (g) uwcnn_typei, (h) uwcnn_retrain, (i) FUnIE, (j) the method of the invention, (k) reference image (group Truth).
By image contrast analysis:
(b) UDCP fails to enhance the original image and even degrades the original image.
(c) ODM makes the original image clearer, but the enhanced image has a large area of red texture.
(d) Uilbla fails to correct the color cast phenomenon of the original image.
(e) The GDCP produces an overexposed enhanced image.
(f) The enhanced image of Water-Net is overall reddish.
(g) Uwcnn_typei has poor enhancement effect on low-luminance underwater images, and a partial orange phenomenon occurs on individual images.
(h) The enhanced image of uwcnn_retrain is entirely darker and a color loss phenomenon occurs on the individual images.
(i) The FUnIE only alleviates the atomization effect of the primary color image, and fails to correct the problems of color cast, low contrast and the like of the primary color image.
(j) The method can well correct various problems of color cast, low contrast, low saturation and the like, and the visual effect of the enhanced image is better and is closer to that of a reference image.
In addition to visually exhibiting the beneficial effects of the method of the present invention, the robustness of the method of the present invention compared to other underwater image enhancement methods will be described in terms of quantization index. As shown in table 1, 8 indices on the 1000 synthetic underwater image datasets were calculated for the comparative underwater image enhancement method and the method of the present invention, including three indices, mean Square Error (MSE), peak signal to noise ratio (PSNR), and Structural Similarity (SSIM). The data in the table are all calculated using the image enhanced by each method.
As shown in table 1, the method of the present invention achieved the first of three metrics, further verifying the powerful performance of the method of the present invention from the quantitative metrics.
Claims (6)
1. A method of underwater image enhancement combining RGB and HSV color spaces, comprising the steps of:
step one: performing color correction on the original color underwater image;
step two: generating a multi-scale underwater image by using a nearest neighbor interpolation method from the underwater image after color correction;
step three: sending the multi-scale underwater image into an encoder, and respectively extracting multi-scale features in two color spaces of RGB and HSV;
step four, selectively fusing the multi-scale features extracted in the encoding stage by using a selective kernel feature fusion module SKFF to the features with the same scale from the RGB and HSV color spaces;
step five: and decoding and reconstructing the fused features, gradually reconstructing the underwater image from the minimum scale, and finally obtaining the enhanced underwater image.
2. The method for enhancing an underwater image combining RGB and HSV color spaces according to claim 1, wherein in the first step, the specific contents and method steps of performing color correction on the original color underwater image are as follows:
(1) Dividing an input color underwater image X into three channels of red, green and blue according to channel dimensions, and respectively and sequentially marking the three channels as X red ，X green ，X blue The method comprises the steps of carrying out a first treatment on the surface of the Respectively averaging the three channels, and sequentially marking the obtained averages as mu r ，μ g ，μ b The method comprises the steps of carrying out a first treatment on the surface of the Wherein mu r ＝mean(X red )，μ g ＝mean(X green )，μ b ＝mean(X blue ) Mean () represents the averaging function, i.e
Wherein M and N represent the width and height of the channel, x, respectively i,j Representing pixel values in the channel with coordinates (i, j);
(2) Sequencing the obtained average value from small to large to obtain a minimum average value mu low Median average mu mid Maximum mean mu high The method comprises the steps of carrying out a first treatment on the surface of the Wherein mu low ,μ mid ,μ high ＝sort(μ r ,μ g ,μ b ) Sort () represents the ranking function；
(3) Respectively judging the average value of the red, green and blue channels; adding to each element in the channel of the minimum mean a numerical difference Deltaμ between the median mean and the minimum mean 1 ，Δμ 1 ＝μ mid -μ low The method comprises the steps of carrying out a first treatment on the surface of the The difference Δμ between the maximum and median means is subtracted for each element in the maximum mean channel 2 ，Δμ 2 ＝μ high -μ mid ；
(4) Combining the three red, green and blue channels to obtain a color corrected underwater image operates as follows: z=concat (X red ,X green ,X blue ) Where Z represents the color corrected underwater image and concat () represents the merging of the red, green and blue channels into a color image in the channel dimension.
3. The method for enhancing an underwater image combining RGB and HSV color spaces according to claim 1, wherein in the second step, the specific content and method steps for generating the multi-scale underwater image by using nearest neighbor interpolation method are as follows:
scale_2＝interpolate(input＝scale_1,scale_factor＝0.5,mode＝'nearest')
scale_4＝interpolate(input＝scale_1,scale_factor＝0.25,mode＝'nearest')
wherein, interpolation () represents an interpolation function that accepts three parameters, i.e., input image input, scale_factor, and scale mode, respectively; mode scale_1 represents the input image, i.e., the original size; scale_2 represents a 2-fold downsampled image of the original size image; scale_4 represents a 4 times downsampled image of the original size image.
4. The method for enhancing an underwater image combining RGB and HSV color spaces according to claim 1, wherein in the third step, the multi-scale underwater image is sent to an encoder, and specific contents and method steps for respectively extracting multi-scale features in the RGB and HSV color spaces are as follows:
(1) Extracting features of which the underwater image scale is scale_1 in the RGB color space by using 3X 3 convolution operation, and then sending the features into a coding block formed by residual convolution to obtain extracted features rgb_encoded_1 of an original scale;
(2) In order to stabilize network gradient, firstly, extracting an underwater image of scale_2 scale of RGB color space by using a shallow convolution module SCM to obtain shallow feature rgb_scm_2, simultaneously, downsampling the feature rgb_encoded_1 by using a maximum pooling operation with a step length of 2 to obtain downsampled feature mp_rgb_encoded_1, then, sending the shallow feature rgb_scm_2 and the downsampled feature mp_rgb_encoded_1 into a feature attention module FAM to obtain feature rgb_fam_2, and finally, extracting the feature by using a coding block to obtain the feature rgb_encoded_2 downsampled by 2 times of the original size; likewise, performing the same operation as that of the scale_2 underwater image in the RGB color space, and obtaining a feature rgb_encoded_4 downsampled by 4 times of the original size;
(3) Extracting underwater image features in an HSV color space, firstly converting an underwater image with a color space of RGB into the HSV color space, and then performing the same operation as in the step (2); likewise, three scale features, hsv_endcoded_1, hsv_endcoded_2, hsv_endcoded_4, are extracted under HSV color space.
5. The method for enhancing an underwater image combining RGB and HSV color spaces according to claim 1, wherein in the fourth step, for the multi-scale features extracted in the encoding stage, the specific content and method steps of selectively fusing features of the same scale from the two RGB and HSV color spaces using a selective kernel feature fusion module SKFF are as follows:
(1) Element-by-element addition of features of different color spaces of the same scale is performed, and then a global pooling operation is used to obtain an average value vector of each channel
C is the number of channels of the feature;
(2) Deriving compact channel characteristics from s using convolution and PReLU activation functionsCharacterization of
C is the number of channels of the feature;
(3) Feeding z into two parallel convolution layers to obtain two feature descriptors v 1 And v 2 ，v 1 ,
C is the number of channels of the feature, and then v is found separately using the self-attention mechanism 1 And v 2 Attention score s of (2) 1 Sum s 2 The specific operation is as follows:
c is the number of channels of the feature;
(4) Using the attention score s 1 Sum s 2 For input features L 1 And L 2 The specific content and method for selecting and fusing are as follows: u=s 1 ·L 1 +s 2 ·L 2 The method comprises the steps of carrying out a first treatment on the surface of the Wherein U represents the output of the SKFF module, L 1 And L 2 Two inputs representing the same scale and different color spaces, namely SKFF module, such as rgb_encoded_1 and hsv_encoded_1;
(5) The SKFF module outputs three scales of features containing rich semantic information, namely skff_scale_1, skff_scale_2, and skff_scale_4.
6. The method for enhancing an underwater image combining RGB and HSV color spaces according to claim 1, wherein in the fifth step, the fused features are decoded and reconstructed, the underwater image is gradually reconstructed from the minimum scale, and the specific content and method steps of the enhanced underwater image are as follows:
(1) Decoding the features of the skff_scale_4 scale by using a decoding block formed by residual convolution, then sending the reconstructed features into a convolution layer to stabilize gradients to obtain decoded features de_scale_4, and then up-sampling the decoded features 2 times by using a nearest neighbor interpolation method to obtain decoded features de_scale_up_2;
(2) Splicing the features of the skff_scale_2 scale to the decoded features de_scale_up_2 sampled in the step (1) according to the dimension of the channel, and repeating the step (1) to obtain the decoded features de_scale_up_1;
(3) Splicing the features of the skff_scale_1 scale to the decoded features de_scale_up_1 after up-sampling in the step (2) according to the dimension of the channel, then sending the features into a decoding block and finally obtaining the decoded features de_scale_1 in a dimension-reducing convolution layer, and finally adding the decoded features de_scale_1 with the original input color underwater image element by element to obtain the enhanced underwater image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310213964.3A CN116363001A (en) | 2023-03-08 | 2023-03-08 | Underwater image enhancement method combining RGB and HSV color spaces |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310213964.3A CN116363001A (en) | 2023-03-08 | 2023-03-08 | Underwater image enhancement method combining RGB and HSV color spaces |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116363001A true CN116363001A (en) | 2023-06-30 |
Family
ID=86918042
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310213964.3A Pending CN116363001A (en) | 2023-03-08 | 2023-03-08 | Underwater image enhancement method combining RGB and HSV color spaces |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116363001A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116993636A (en) * | 2023-07-10 | 2023-11-03 | 中国地质大学（武汉） | Image enhancement method and device for underground low-illumination deep stratum empty area |
-
2023
- 2023-03-08 CN CN202310213964.3A patent/CN116363001A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116993636A (en) * | 2023-07-10 | 2023-11-03 | 中国地质大学（武汉） | Image enhancement method and device for underground low-illumination deep stratum empty area |
| CN116993636B (en) * | 2023-07-10 | 2024-02-13 | 中国地质大学（武汉） | Image enhancement method and device for underground low-illumination deep stratum empty area |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ren et al. | Reinforced swin-convs transformer for simultaneous underwater sensing scene image enhancement and super-resolution | |
| CN112927202A (en) | Method and system for detecting Deepfake video with combination of multiple time domains and multiple characteristics | |
| CN116152120B (en) | Low-light image enhancement method and device integrating high-low frequency characteristic information | |
| CN112561799A (en) | Infrared image super-resolution reconstruction method | |
| CN113284061B (en) | Underwater image enhancement method based on gradient network | |
| CN111476744B (en) | Underwater image enhancement method based on classification and atmospheric imaging model | |
| CN114881871A (en) | Attention-fused single image rain removing method | |
| CN116188325A (en) | Image denoising method based on deep learning and image color space characteristics | |
| CN112270646B (en) | Super-resolution enhancement method based on residual dense jump network | |
| CN112801922A (en) | Color image-gray image-color image conversion method | |
| CN116363001A (en) | Underwater image enhancement method combining RGB and HSV color spaces | |
| CN114881879B (en) | Underwater image enhancement method based on brightness compensation residual error network | |
| CN116596795A (en) | Underwater image enhancement method based on semantic guidance and attention fusion | |
| CN116523794A (en) | Low-light image enhancement method based on convolutional neural network | |
| Tang et al. | AutoEnhancer: Transformer on U-Net architecture search for underwater image enhancement | |
| Liu et al. | Multi-Scale Underwater Image Enhancement in RGB and HSV Color Spaces | |
| CN115700731A (en) | Underwater image enhancement method based on dual-channel convolutional neural network | |
| CN114757863A (en) | Underwater image enhancement method integrating deep learning and traditional image enhancement technology | |
| CN113689346A (en) | Compact deep learning defogging method based on contrast learning | |
| Qi et al. | Deep Color-Corrected Multi-scale Retinex Network for Underwater Image Enhancement | |
| CN117115411A (en) | Multi-frequency double-branch underwater image enhancement method | |
| CN117078556A (en) | Water area self-adaptive underwater image enhancement method | |
| CN117351340A (en) | Underwater image enhancement algorithm based on double-color space | |
| Jin et al. | Color correction and local contrast enhancement for underwater image enhancement | |
| CN114820395B (en) | Underwater image enhancement method based on multi-field information fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |