CN116579953A - Self-supervision water surface image enhancement method and related equipment - Google Patents
Self-supervision water surface image enhancement method and related equipment Download PDFInfo
- Publication number
- CN116579953A CN116579953A CN202310775963.8A CN202310775963A CN116579953A CN 116579953 A CN116579953 A CN 116579953A CN 202310775963 A CN202310775963 A CN 202310775963A CN 116579953 A CN116579953 A CN 116579953A
- Authority
- CN
- China
- Prior art keywords
- image
- water surface
- brightness
- enhancement
- fractional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 110
- 238000000034 method Methods 0.000 title claims abstract description 93
- 238000001914 filtration Methods 0.000 claims abstract description 52
- 230000008569 process Effects 0.000 claims abstract description 14
- 238000013507 mapping Methods 0.000 claims abstract description 10
- 230000002708 enhancing effect Effects 0.000 claims description 38
- 238000005286 illumination Methods 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 12
- 230000001186 cumulative effect Effects 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 7
- 238000009499 grossing Methods 0.000 claims description 6
- 238000012805 post-processing Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 4
- 230000002146 bilateral effect Effects 0.000 claims description 2
- 230000002401 inhibitory effect Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 10
- 230000007547 defect Effects 0.000 abstract description 3
- 230000004069 differentiation Effects 0.000 description 10
- 239000003086 colorant Substances 0.000 description 4
- 230000001965 increasing effect Effects 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004321 preservation Methods 0.000 description 2
- 150000003839 salts Chemical class 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003707 image sharpening Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration by the use of histogram techniques
-
- G06T5/94—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/30—Assessment of water resources
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The application discloses a self-supervision water surface image enhancement method, a self-supervision water surface image enhancement device, electronic equipment and a computer storage medium. The application utilizes a fractional median filter and a fractional differential filter to carry out image denoising and enhancement on a water surface image, introduces a water surface brightness detail gain factor and combines with adjustment mapping to protect the contrast of a reflection position of the water surface image, adopts a guided filtering estimation image incidence component, enhances the image brightness by combining a multi-scale brightness enhancement method, introduces a clipping histogram equalization method to complete enhancement of water surface texture details of the image, and in the denoising and enhancement process, parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to characteristic information of the image, thereby overcoming the defects that the image blurring and enhancement effect are not obvious due to classical median filtering denoising and traditional fractional filtering enhancement, and solving the problem that the details and edges of the image are not clear due to low signal-noise ratio in the water surface image.
Description
Technical Field
The application relates to the technical field of unmanned ship visual perception, in particular to a self-supervision water surface image enhancement method and related equipment.
Background
Autonomous navigation of the unmanned ship is an intelligent embodiment, and the implementation process depends on the perception capability of the unmanned ship to the surrounding environment. The vision system is often used in the perception system of the unmanned ship, however, the vision function is often limited in the dark light environment, so that the perception capability of the unmanned ship to the surrounding environment is seriously influenced, and the autonomous navigation of the unmanned ship is further influenced. The prior art generally enhances the image quality of the dark-light image and reduces the image noise. Conventional algorithms for dim light images include a median filtering algorithm, a fractional order filtering algorithm, and a single-scale Retinex algorithm. The traditional dim light image enhancement algorithm does not distinguish noise points and signal points, but performs unified processing on all pixel points of an image, so that the signal points can be filtered out while denoising is performed, the edge details of the image are greatly lost, and the image is blurred. Additionally consider the characteristics of the water surface scene: reflection on the water surface can cause more reflection areas and stronger reflection brightness in the low-illumination image on the water surface, so that more difficulty is brought to image enhancement; and the water surface may cause the texture of the object surface to become coarser and blurred, especially in low-light conditions, which may adversely affect the texture-based image enhancement method. Therefore, the application provides a self-supervision water surface image enhancement method and related equipment based on the characteristics of a water surface scene, which are favorable for the reflection characteristic of the water surface, can effectively enhance the image of a camera in a water surface dim light scene, and improve the visual perception capability of an unmanned ship at night.
Disclosure of Invention
The application aims to solve the technical problem of providing a self-supervision water surface image enhancement method and related equipment, which are used for overcoming the defects that the image blurring and enhancement effect are not obvious due to classical median filtering denoising and traditional fractional order filtering enhancement, and improving the problem that the details and edges of the image are not clear enough due to lower signal-to-noise ratio (SNR) in the water surface image by effectively adjusting the details of the water surface image.
In order to solve the technical problems, the application adopts the following technical scheme:
a self-supervision water surface image enhancement method comprises the following steps: image denoising and enhancing are carried out on the water surface image by using a fractional median filter and a fractional differential filter, and parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to characteristic information of the image in the denoising and enhancing process; carrying out image preliminary enhancement on the denoised water surface image, and utilizing the water surface brightness detail gain factor and the adjustment mapping to improve the overall contrast of the image, adjust the contrast of the reflection position area of the water surface image and inhibit halation; enhancing the image brightness by using a guided filtering and multi-scale brightness enhancement method; enhancing image details by using a water surface brightness detail gain factor; enhancing the image contrast by using a clipping histogram equalization method; according to the illumination information of the original image, carrying out illumination compensation on the enhanced image; artifact removal, smoothing and normalization are performed on the enhanced image.
A self-supervision water surface image enhancement device comprises an image denoising module, an image preliminary enhancement module, an image brightness enhancement module, an image detail enhancement module, an image contrast enhancement module, an illumination compensation module and an image post-processing module; the image denoising module is used for denoising and enhancing the water surface image by using the fractional median filter and the fractional differential filter, and in the denoising and enhancing process, parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to characteristic information of the image; the image preliminary enhancement module is used for carrying out image preliminary enhancement on the water surface image processed by the image denoising module, and improving the overall contrast of the image, adjusting the contrast of a reflection position area of the water surface image and inhibiting halation by utilizing the water surface brightness detail gain factor and the adjustment mapping; the image brightness enhancement module is used for enhancing the image brightness by utilizing a guide filtering and multi-scale brightness enhancement method; the image detail enhancement module is used for enhancing image details by utilizing the water surface brightness detail gain factor; the image contrast enhancement module is used for enhancing the image contrast by utilizing a clipping histogram equalization method; the illumination compensation module is used for carrying out illumination compensation on the enhanced image according to the illumination information of the original image; the image post-processing module is used for carrying out artifact removal, smooth filtering and normalization processing on the enhanced image.
An electronic device comprising at least one processor and at least one memory communicatively coupled to the processor, wherein the memory stores program instructions that when invoked by the processor perform the self-supervising surface image enhancement method described above.
A computer storage medium storing program instructions which when executed by a processor implement the self-supervising water surface image enhancement method described above.
The beneficial technical effects of the application are as follows: the application uses a fractional median filter and a fractional differential filter to carry out image denoising and enhancement on the water surface image; introducing a water surface brightness detail gain factor and combining with adjustment mapping to protect the contrast of the reflection position of the water surface image; estimating an image incidence component by adopting guide filtering, and enhancing the image brightness by combining a multi-scale brightness enhancement method; introducing a clipping histogram equalization method to complete the enhancement of the water surface texture details of the image; in the denoising and enhancing process, parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to characteristic information of the image, so that noise points can be effectively removed, normal pixel points can be reserved, the enhancement of image edge details is more obvious, texture details are more prominent, the defects that the image blurring and enhancing effects are not obvious due to classical median filtering denoising and traditional fractional filtering enhancement are overcome, and the problem that details and edges of the image are not clear enough due to lower signal-to-noise ratio (SNR) in the water surface image is solved.
Drawings
Fig. 1 is a flow chart of the self-monitoring water surface image enhancement method of the present application.
Fig. 2 is a schematic structural diagram of the self-monitoring water surface image enhancement device of the present application.
Detailed Description
The present application will be further described with reference to the drawings and examples below in order to more clearly understand the objects, technical solutions and advantages of the present application to those skilled in the art.
As shown in fig. 1, in one embodiment of the present application, the self-monitoring water surface image enhancement method includes steps S10 to S70:
s10, denoising and enhancing the water surface image by using a fractional median filter and a fractional differential filter.
The step is used for denoising the input water surface image, and then filtering and enhancing the image so as to eliminate noise and other interference factors in the image. In the image denoising process, the correct judgment of noise points is the key of denoising. Step S10 specifically includes steps 1.1-1.4.
1.1 Calculating local variance:
for each pixel of the input water surface image, the local variance in the neighborhood thereof is calculated. Different sized neighborhood windows may be selected, such as 3x3, 5x5, etc.
1.2 Calculating fractional order parameters:
a fractional order parameter is calculated for each pixel based on the local variance. The fractional order parameter is used to adjust the intensity of the fractional order median filter. Noise may be present in areas of greater local variance, and therefore fractional order parameters should be smaller in order to more strongly remove the noise. Fractional order parameters can be calculated using the following formula:
α = 1 - exp(-k x σ 2 )
wherein alpha is a fractional order parameter, sigma 2 And k is an adjusting parameter, and can be manually set or automatically calculated according to actual conditions.
1.3 Fractional order median filtering:
for each pixel, a fractional median filter is applied. The neighborhood values of the pixels are sequenced, and then the fractional median value is calculated according to the fractional parameter alpha, and the specific calculation method is as follows:
M = M_min + α x (M_max - M_min)；
wherein M is a fractional median, and M_min and M_max are the minimum and maximum values of the ordered neighborhood values respectively. And finally, replacing the central pixel with the fractional median value to obtain the denoised image.
For edge and texture regions, fractional median filtering may result in loss of detail. To address this problem, an adaptive adjustment mechanism may be introduced for adjusting the fractional order parameters when applying fractional order median filtering. For each pixel, its local gradient is calculated, and then the fractional order parameter is adjusted according to the local gradient of the pixel. If the local gradient is high (i.e. at the edge or texture region), the alpha value is increased when computing the fractional median value to preserve detail information. The specific adjustment method can be designed according to actual conditions, for example, when Gaussian white noise is eliminated, the recommended fractional order parameter is 0.1-0.5, so that a better noise suppression effect can be provided, but image details are slightly lost; when the spiced salt noise is eliminated, the recommended fractional order parameter is 0.5-0.9, so that the image details can be well reserved, but the noise suppression effect is slightly poor; if the image contains Gaussian white noise and spiced salt noise at the same time, an adaptive fractional order filtering method can be considered to be adopted, and different types of noise can be processed by using different fractional order parameters so as to achieve the optimal filtering effect.
1.4 Fractional order differential filtering:
and enhancing the denoised image by using a fractional differential filter. In the water surface image, the image may have problems such as noise and blurring due to insufficient illumination. Fractional differentiation can better address and optimize these issues. Fractional-order differentiation (order differentiation) is an extension of the integer-order differentiation in conventional calculus, introducing the concept of real-order differentiation, which better describes non-local and non-stationary signals. The fractional differentiation can improve the quality of the water surface image by enhancing the edge information and the details of the image, so that the image is clearer and easier to observe.
Sharpening filtering based on fractional derivatives is applied: the method expands the integer order derivative in the image sharpening filter to the real order derivative, thereby improving the retention capacity of the image edge and detail and enabling the enhanced image to have more definition and contrast. The method can be realized by the following formula:
f' (x,y) = C x ( f(x+1,y) - f(x,y) ) / |x+1-y| β
in the formula, f' (x, y) represents the enhanced image pixel value, f (x, y) represents the original image pixel value, C represents a constant, β represents the order of fractional differentiation, and |x+1-y| represents the distance between the horizontal and vertical pixels.
When β is an integer, the formula is converted to the usual integer derivative form. When β is a real number, then finer edge preservation and noise removal can be achieved.
Specifically, when β takes a value smaller than 1, the formula retains edges while suppressing noise, thereby realizing enhancement of the image; when beta is larger than 1, the edges can be better enhanced and highlighted, so that the image has more stereoscopic impression and layering impression.
In the formula, the distance |x+1-y| between the horizontal and vertical pixels is introduced to ensure that the denominator is not 0, and can control the scale and range of the fractional differentiation. When the distance value is larger, the scale of fractional order differentiation is larger, and larger edges are reserved and enhanced; and when the distance value is smaller, the scale of fractional order differentiation is smaller, and details and small edges are reserved and enhanced.
The enhancement and optimization of low contrast and water surface images can be realized by applying fractional differential filtering, and the method has wide application prospect and research value.
Through the steps 1.1-1.4, the denoising and enhancing of the input water surface image can be realized, in the denoising and enhancing process, parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to the characteristic information of the image, so that noise points can be effectively removed, normal pixel points can be reserved, the enhancement of image edge details is more obvious, and the texture details are more prominent.
S20, carrying out image preliminary enhancement on the denoised water surface image, and utilizing the water surface brightness detail gain factor and the adjustment mapping to improve the overall contrast of the image, adjust the contrast of the reflection position area of the water surface image and inhibit halation.
Step S20 specifically includes steps 2.1-2.5.
2.1 Analyzing image details:
the high frequency details are extracted from the denoised and enhanced image through step S10, and may be implemented by a high pass filter such as Laplacian, sobel, prewitt, etc.
2.2 Calculating local contrast:
local contrast is calculated using the extracted high frequency details. The embodiment uses local variance or gradient amplitude to represent local contrast of the image, which is helpful to maintain contrast in the reflective position area of the water surface image, and simultaneously suppresses clipping effect and halation.
2.3 Calculating a water surface brightness detail gain factor:
the surface brightness detail gain factor may be calculated from the local contrast. The surface brightness detail gain factor may be obtained by dividing the local contrast by the global contrast, where the global contrast may be represented by the standard deviation or variance of the gray level histogram.
2.4 Enhancing image contrast:
an adjustment map (e.g., histogram equalization, contrast stretching, or logarithmic mapping, etc.) is applied to improve the overall contrast of the image. Meanwhile, a water surface brightness detail gain factor is applied to the intensity value of each pixel to adjust the contrast of the water surface image reflection position area so as to enhance the contrast in the water surface image reflection position area.
2.5 Inhibition of halation:
in order to suppress the halation phenomenon, a bilateral filter, a guided filter, or the like may be used. The filters can smooth the image area while maintaining the edge information, and effectively reduce the halation phenomenon.
S30, enhancing the image brightness by using a guided filtering and multi-scale brightness enhancement method.
Through the steps S10 and S20, denoising and preliminary enhancement of the water surface image are completed, and in step S30, the R, G colors and the B colors of the image are respectively corrected by using a multi-scale brightness enhancement method, and are synthesized, so that the brightness of the image is enhanced. By applying the multi-scale Brightness enhancement method, an initial image to be enhanced needs to be subjected to color space transformation, an image of the water surface is transformed into an HSI space model from an RGB space model, and the HSI color space is formed by describing colors by using Hue (Hue), saturation (Saturation or Chroma) and Brightness (Brightness or Brightness) from a visual system of a person.
Step S30 specifically includes steps 3.1-3.5.
3.1 Color space conversion:
the preliminary enhanced water surface image is converted from the RGB color space to the HSI color space. This process includes calculating Hue (Hue), saturation (Saturation), and Intensity (Intensity) components for each pixel. The conversion formula is as follows:
H(x, y) = acos((0.5 x [(R(x, y) - G(x, y)) + (R(x, y) - B(x, y))]) / sqrt((R(x, y) - G(x, y))^2 + (R(x, y) - B(x, y)) x (G(x, y) - B(x, y))))；
S(x, y) = 1 - 3 x min(R(x, y), G(x, y), B(x, y)) / (R(x, y) + G(x, y) + B(x, y))；
I(x, y) = (R(x, y) + G(x, y) + B(x, y)) / 3；
wherein R (x, y), G (x, y), B (x, y) represent two-dimensional matrix coordinates of R, G, B channels in RGB color space, respectively, and H (x, y), S (x, y), and I (x, y) represent hue, saturation, and intensity components in HSI color space, respectively.
3.2 Calculating the average brightness of the image:
in the HSI color space (Hue, saturation), the Intensity component (Intensity) is used to calculate the average luminance of an image. The following is the step of calculating the average luminance of an image in the HSI color space:
3.2.1 Calculating the sum of the intensity components: traversing the intensity components of the HSI image, calculating the sum of pixel values, expressed by the following formula: Σ= Σ (x, y) I (x, y), where Σ represents the sum of intensity components and (x, y) represents coordinates in the image.
3.2.2 Dividing the sum of the intensity components by the total number of pixels in the image to obtain the average brightness of the image, wherein the calculation formula is as follows: l=Σ/(width x height), where L represents average luminance, and width and height represent width and height of an image, respectively. Through the above steps, the average brightness of the image can be calculated in the HSI color space, and this value can be used for analyzing the brightness distribution of the image, performing image enhancement, and the like.
3.3 Determining reference weights for brightness enhancement:
the reference weight for brightness enhancement is a coefficient for adjusting the brightness enhancement degree of an image, and can be set according to the characteristics of the image and the enhancement target. The reference weights may be combined with other factors in processing the image to achieve the desired brightness enhancement effect. The following is the step of determining the reference weight for luminance enhancement:
3.3.1 Analyzing the brightness distribution of the image: first, the luminance distribution of an input image is analyzed by calculating statistical features such as average luminance, histogram distribution, and the like of the image. These statistics help to understand the overall brightness level and contrast condition of the image.
3.3.2 Setting a brightness enhancement target: the brightness enhancement targets include increasing overall brightness, enhancing local contrast, adjusting specific brightness intervals, etc., and the brightness enhancement targets should be selected according to image content and application requirements.
3.3.3 Calculating reference weights: based on the analyzed luminance characteristics and the set luminance enhancement target, a reference weight for luminance enhancement is calculated. The reference weight can be adjusted according to actual conditions, and can be a fixed value or dynamically adjusted according to image characteristics. For example, for an image with more dark areas, a higher reference weight may be set to enhance brightness; while for images with more bright areas, lower reference weights may be set to preserve detail.
Through the steps 3.3.1-3.3.3, the reference weight of brightness enhancement can be determined and applied to the image processing process, so that the targeted brightness enhancement can be realized, and meanwhile, the natural sense and detail information of the image can be maintained. In particular, the reference weight for brightness enhancement may be combined with other enhancement factors (e.g., contrast adjustment, color balance, etc.) to work together with brightness enhancement of the image. In this way, a desired brightness enhancement effect can be achieved while maintaining the natural feel of the image.
3.4 Stretching treatment of linear saturation component:
the saturation component in the HSI color space is linearly stretched. This can be achieved by mapping the value of the saturation component to a new range. The value of the original saturation component is multiplied by a stretch factor (greater than 1) and the result is then limited to a suitable range.
The saturation of the image reflects the color depth, the saturation S of the image is different in stretching of the water surface image obtained under different external conditions, and the image is improved to the optimal effect by adopting a linear stretching method of the saturation S, wherein the specific formula is as follows:
wherein max (R, G, B) and min (R, G, B) represent the maximum and minimum values of R, G, B for all colors, respectively, S represents the saturation which needs to be enhanced on average under low light irradiation,M v representing the mean of the image under the original low light,Srepresents the saturation of the image after linear stretching.
3.5 The image brightness enhancement is realized by applying the guide filtering to a multi-scale brightness enhancement method:
guided Filter (Guided Filter) is a linear time-varying Filter that uses a local linear model to fit the image so that the filtered output image is structurally close to the Guided image. The output of the filter is determined using a pilot image, and the filter weights are adjusted according to the distance between pixels and the gray value difference, taking into account the similarity between pixels. In the edge region, the gray value difference between pixels is larger, and the guiding filtering can automatically reduce the weight, so that the edge information is reserved.
The multi-scale brightness enhancement method is an image enhancement algorithm based on the human visual system for improving brightness and contrast of images. The guiding filtering is applied to the multi-scale brightness enhancement method, so that the method can be well adapted to edge and texture changes in the image, and the image quality is improved.
The following is the step of applying the guided filtering to the multi-scale luminance enhancement method:
3.5.1 Converting into a gray image: if the input image is color, it is first converted into a gray scale image. This helps to simplify the processing of subsequent steps.
3.5.2 Creating a filter bank: to implement the multi-scale luminance enhancement method, a set of guide filters with different radii is created. The input image I and the filter radius, eps are filter intensities, for each filter radius r, a mean I_mean, a variance I_var and a correlation Ip are calculated on the image I, the mean I_mean, the variance I_var and the correlation Ip represent statistical features of the image I itself, and p_mean and p_var represent the mean and the variance of the image I at the current radius r and reflect the statistical features of the image I at the current window, which are used for calculating linear model parameters. Wherein:
(1) the expression for calculating the mean is:
I_mean = 1/((2r+1) 2 ) x sum(sum(I))
the expression for calculating the variance is:
I_var = 1/((2r+1) 2 ) x sum(sum(I 2 )) - I_mean 2
the expression for calculating the correlation is:
Ip = 1/((2r+1) 2 ) x sum(sum(I 2 )) - I_mean 2
(2) calculating linear model parameters a and b:
a = Ip / (I_var + eps) ； b = I_mean - axp_mean
(3) calculating a filtered image q from a and b:
q = axI + b
(4) calculating a weight template W:
W = 1/(axI_var + eps)x(a - 1)x(p_var + I_var)
(5) updating the filtering template according to W and q:
filter = WxI + (1-W)xq
(6) filters are added to the filters list.
(7) Returning to the filters list, wherein each element is a filter template for the corresponding radius.
The above procedure results in linear model parameters a and b by calculating the mean, variance and correlation of each filter radius. The filtered value q and the weight template W are then calculated from a and b. And finally updating a filtering template filter according to W and q, and adding the filtering template filter into a filters list. A group of guide filtering templates with different radiuses can be obtained through filters and used for carrying out smooth filtering and edge information preservation on the image.
3.5.3 A filter bank is applied: each of the guide filters is applied to the gray scale image in turn. The logarithmically transformed image is convolved using guided filtering, which results in a series of smoothed images, each corresponding to a particular filter scale. Here, the input image serves as a guide image, which helps to preserve edge and texture information while smoothing the image. The output of the guided filtering will be the estimated incident component image.
3.5.4 Calculating the reflection component of the enhanced image: based on the image incidence component, a multi-scale brightness enhancement method is utilized to obtain a reflection component of the enhanced image. The multi-scale brightness enhancement is an image enhancement technology, mainly uses filters with different scales to enhance details in different frequency ranges of an image, thereby realizing a global brightness enhancement effect, and obtains a reflection component by selecting filters with different scales to enhance low-frequency and high-frequency information, and then calculating the difference with an original image to extract lost intermediate-frequency information. Compared with the method of directly carrying out high-pass filtering on the original image, the method based on the multi-scale thought can obtain reflection information more naturally, and the effect is more real. The process is as follows:
(1) and inputting an original image I, and performing Gaussian pyramid decomposition to obtain images Gi with different scales, wherein the images G0 comprise low-frequency information and the images G3 comprise high-frequency information. The low-frequency information reflects the overall brightness and contrast of the image and contains large-scale change information; the high-frequency information reflects the details and edge information of the image and contains small-scale change information; the intermediate frequency information of the original image is lost or not obvious due to insufficient illumination, so that the intermediate frequency information cannot be directly acquired.
(2) And selecting a filter r0 with a larger scale to filter the low-frequency image G0, and obtaining a filtering result G0_filter. This step mainly enhances the low frequency information, corresponding to the diffuse reflection component.
(3) And selecting a smaller-scale filter r3 to filter the high-frequency image G3 to obtain a filtering result G3_filter. This step mainly enhances the high frequency information, corresponding to the specular component.
(4) And combining the G0_filter and the G3_filter by using a Gaussian pyramid to obtain an intermediate result J. J contains enhanced low frequency and high frequency information.
(5) And calculating the difference between J and the original image I to obtain a difference result D. Since J loses intermediate frequency information in the original image I, the non-zero region in D corresponds to intermediate frequency information of the original image I, that is, the reflected component.
It should be noted that, we consider a low-pass filter with a size exceeding 5×5 as a larger-scale filter; a low pass filter of size 3x3 or 5x5 is considered a mid-scale filter; high-pass filters are rarely over 3×3 in size because of the need to extract high-frequency information, and high-pass filters not exceeding 3×3 in size are considered as smaller-scale filters.
3.5.5 Reconstructing an enhanced image: the estimated incident and reflected components are combined to reconstruct an enhanced image. Here, the incident component may be appropriately stretched to improve the brightness of the image. The reconstructed image may be obtained by the following formula:
Re(x, y) = S(x, y) x T(x, y)；
where Re (x, y) represents the reconstructed enhanced image, S (x, y) represents the stretched incident component, and T (x, y) represents the reflected component.
3.5.6 Inverse logarithmic transformation: the reconstructed enhanced image is inverse log transformed and converted back to the original dynamic range. The inverse logarithmic transformation can be performed using the following formula:
O(x, y) = exp(Re(x, y)) - 1；
where O (x, y) represents the final luminance enhanced image.
Through the steps 3.5.1-3.5.6, the guide filtering can be applied to a multi-scale brightness enhancement method, the guide filtering is utilized to carry out convolution operation on the water surface image, the incidence component of the image is estimated, the reflection component of the enhanced image is obtained by utilizing the multi-scale brightness enhancement method based on the incidence component of the image, and the edge and texture information is reserved while the brightness and the contrast of the image are improved. This approach is applicable to challenging image scenes of water surface, high dynamic range, etc.
S40, enhancing the image details by using the water surface brightness detail gain factor.
The water surface brightness detail gain factor is introduced in the image processing, so that the detail expression of the image can be effectively improved, and the local contrast is increased, so that the information in the image can be observed more clearly. Step S40 specifically includes steps 4.1-4.4.
4.1 Extracting a detail layer: and carrying out Gaussian filtering on the brightness-enhanced image brightness channel to obtain a smooth brightness component, and dividing the original brightness channel by the smooth brightness component to obtain a detail layer.
4.2 And (3) applying a detail gain factor to update the brightness channel: to improve the image detail, the detail layer may be multiplied by a detail gain factor (e.g., 1.5) and then multiplied by the smoothed luminance component to obtain an updated L-channel. In this way, the local contrast of the image may be increased, highlighting detailed information in the image.
4.3 Conversion back to RGB color space: the enhanced image is converted from the HSI color space back to the RGB color space and the updated luminance channel image is noted as the enhanced image.
4.4 Outputting the enhanced image: and storing or outputting and displaying the enhanced image.
Through the steps, the water surface brightness detail gain factor can be used for improving the image detail, so that the image is more visually attractive.
S50, enhancing the image contrast by using a clipping histogram equalization method.
Histogram equalization methods are often employed to provide contrast during image processing. Conventional histogram equalization methods may result in excessive amplification of noise and unimportant details when improving contrast. The application provides a clipping histogram equalization method based on a traditional histogram equalization method, which ensures that the gray values of an image are distributed more uniformly in the whole gray range by redistributing the gray values of the image, and avoids excessive amplification noise and unimportant details while improving the contrast of the image.
The method for enhancing the image contrast by using the clipping histogram equalization method specifically comprises the steps 5.1-5.6.
5.1 Calculating an original histogram: and calculating a gray level histogram of the input image, namely an original histogram. This can be achieved by counting the number of occurrences of each gray value in the image.
5.2 Calculating a clipping histogram: determining a clipping threshold value, reducing gray values exceeding the threshold value in the original histogram to the threshold value to obtain a clipping histogram, and accumulating the clipped frequency numbers to form an additional frequency number pool.
5.3 Reassigning the frequency numbers: the frequencies in the extra frequency pool are evenly distributed to the non-clipped gray values in the clipping histogram. This will make the gray value distribution in the clipping histogram more uniform.
5.4 Based on the clipping histogram and the clipping histogram after the frequency is reassigned, a cumulative histogram is calculated, the cumulative histogram representing the cumulative frequency of the gradation values of the image pixels.
5.5 Applying an equalization map: an equalization map is calculated based on the cumulative histogram, mapping the original gray value to a new gray value. This can be achieved by normalizing the cumulative histogram values to the entire gray scale range.
5.6 Updating the gray value of the image: the equalization map is applied to each pixel of the input image, replacing the original gray value with a new gray value, and image contrast enhancement is achieved.
By adding the equalization method of clipping histograms, the maximum frequency of some gray values in the histograms (clipping threshold value) is limited, the gray values of the images are redistributed, so that the gray values are more uniformly distributed in the whole gray range, the contrast and detail of the images can be improved, the dynamic range is adjusted, the contrast perception of bright areas and dark areas is enhanced, the contrast enhancement of reflection information is completed, and meanwhile, the excessive amplification of noise and unimportant details is avoided.
S60, carrying out illumination compensation on the enhanced image according to illumination information of the original image.
And carrying out illumination compensation on the enhanced image according to the illumination information of the original image. This can be achieved by multiplying the enhanced image with the illumination information of the original image. This step helps to preserve the natural light feel of the image.
And S70, artifact removal, smoothing filtering and normalization processing are carried out on the enhanced image.
The step is used for post-processing the enhanced image, and comprises the operations of removing artifacts, smoothing and filtering and the like so as to reduce noise and other visual artifacts possibly introduced in the image enhancement process, improve local contrast and keep details; and finally, inputting the image into constraint conditions, and outputting the image after normalization processing.
As shown in fig. 2, based on the self-monitoring water surface image enhancement method in the embodiment shown in fig. 1, the application provides a self-monitoring water surface image enhancement device, which comprises an image denoising module 10, an image preliminary enhancement module 20, an image brightness enhancement module 30, an image detail enhancement module 40, an image contrast enhancement module 50, an illumination compensation module 60 and an image post-processing module 70.
The image denoising module 10 is configured to denoise and enhance the water surface image by using a fractional median filter and a fractional differential filter, and in the denoising and enhancing process, parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to characteristic information of the image, that is, the step S10 in the self-monitoring water surface image enhancing method in the embodiment shown in fig. 1 is performed.
The image preliminary enhancement module 20 is configured to perform image preliminary enhancement on the water surface image processed by the image denoising module, and utilize the water surface brightness detail gain factor and the adjustment map to improve the overall contrast of the image, adjust the contrast of the reflective position area of the water surface image, and suppress halation, that is, to perform step S20 in the self-monitoring water surface image enhancement method in the embodiment shown in fig. 1.
The image brightness enhancement module 30 is configured to enhance the image brightness by using the guided filtering and multi-scale brightness enhancement method, i.e. to perform step S30 in the self-supervised water surface image enhancement method in the embodiment shown in fig. 1.
The image detail enhancement module 40 is configured to enhance the image detail by using the water brightness detail gain factor, that is, to perform step S40 in the self-supervised water image enhancement method in the embodiment shown in fig. 1.
The image contrast enhancement module 50 is configured to enhance the image contrast by using a clipping histogram equalization method, i.e. is configured to perform step S50 in the self-supervised water surface image enhancement method in the embodiment shown in fig. 1.
The illumination compensation module 60 is configured to perform illumination compensation on the enhanced image according to the illumination information of the original image, that is, to perform step S60 in the self-supervised water surface image enhancement method in the embodiment shown in fig. 1.
The image post-processing module 70 is configured to perform artifact removal, smoothing filtering and normalization processing on the enhanced image, that is, to perform step S70 in the self-supervised water surface image enhancement method in the embodiment shown in fig. 1.
The present application also provides an electronic device comprising a processor and a memory communicatively coupled to the processor, wherein the memory is configured to store various types of data to support operations on the electronic device, which may include program instructions for any application or method operating on the electronic device, as well as application-related data, such as program instructions for performing the self-supervised water surface image enhancement method of the embodiment of fig. 1, image parameter configuration data, etc. The Memory may be implemented by any type or combination of one or more volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM for short), electrically erasable programmable Read-Only Memory (EEPROM for short), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM for short), programmable Read-Only Memory (Programmable Read-Only Memory, PROM for short), read-Only Memory (ROM for short), magnetic Memory, flash Memory, magnetic disk, or optical disk. The processor may employ one or more processors for controlling the overall operation of the electronic device by invoking execution of the program instructions stored in the memory to perform the steps of the self-supervising surface image enhancement method in the embodiment shown in fig. 1.
The application also provides a computer storage medium storing program instructions which when executed by a processor implement the self-supervising water surface image enhancement method. The computer storage medium may be a memory including program instructions as described above, which are executable by a processor to perform the steps of the self-supervised water surface image enhancement method of the embodiment shown in fig. 1.
The foregoing is merely a preferred embodiment of the present application and is not intended to limit the present application in any way. Various equivalent changes and modifications can be made by those skilled in the art based on the above embodiments, and all equivalent changes or modifications made within the scope of the claims shall fall within the scope of the present application.
Claims (10)
1. The self-supervision water surface image enhancement method is characterized by comprising the following steps of:
s10, denoising and enhancing the water surface image by using a fractional median filter and a fractional differential filter, wherein parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to characteristic information of the image in the denoising and enhancing process;
s20, carrying out preliminary enhancement on the water surface image processed in the step S10, and utilizing a water surface brightness detail gain factor and an adjustment map to improve the overall contrast of the image, adjust the contrast of a reflection position area of the water surface image and inhibit halation;
s30, enhancing the image brightness by using a guide filtering and multi-scale brightness enhancement method;
s40, enhancing image details by utilizing a water surface brightness detail gain factor;
s50, enhancing the image contrast by using a clipping histogram equalization method;
s60, carrying out illumination compensation on the enhanced image according to illumination information of the original image;
and S70, artifact removal, smoothing filtering and normalization processing are carried out on the enhanced image.
2. The self-supervised water surface image enhancement method according to claim 1, wherein said step S10 further comprises:
s11, calculating local variance in the neighborhood of each pixel of the water surface image;
s12, calculating a fractional order parameter of each pixel according to the local variance;
s13, adjusting the fractional order parameters according to the local gradient of the pixel;
s14, calculating a fractional median value of each pixel according to the fractional parameters, and replacing a central pixel with the fractional median value to obtain a denoised image;
s15, enhancing the denoised image by using a fractional differential filter.
3. The self-supervised water surface image enhancement method according to claim 1, wherein said step S20 further comprises:
s21, extracting high-frequency details from the water surface image processed in the step S10;
s22, calculating local contrast by using the extracted high-frequency details;
s23, calculating a detail gain factor of the water surface brightness according to the local contrast;
s24, the overall contrast of the image is improved by applying adjustment mapping, and the contrast of the reflective position area of the water surface image is adjusted by utilizing the detail gain factor of the brightness of the water surface;
s25, utilizing a bilateral filtering or guided filtering method to restrain halation.
4. The self-supervised water surface image enhancement method as set forth in claim 1, wherein said step S30 further includes:
s31, converting the primarily enhanced water surface image from an RGB color space to an HSI color space;
s32, linearly stretching the saturation component in the HSI color space;
s33, performing convolution operation on the water surface image by adopting guide filtering, and estimating an image incidence component;
s34, obtaining a reflection component of the enhanced image by utilizing a multi-scale brightness enhancement method based on the image incidence component;
s35, combining the estimated incident component and the reflection component, and reconstructing an enhanced image;
s36, performing inverse logarithmic transformation on the reconstructed enhanced image, and converting the reconstructed enhanced image back to the original dynamic range to obtain a final brightness enhanced image.
5. The self-monitoring water surface image enhancement method according to claim 4, wherein said step S40 further comprises:
s41, performing Gaussian filtering on the brightness-enhanced image brightness channel to obtain a smooth brightness component, and dividing the original brightness channel by the smooth brightness component to obtain a detail layer;
s42, multiplying the detail layer by a water surface brightness detail gain factor, and multiplying the detail layer by the smoothed brightness component to obtain an updated brightness channel;
s43, converting the image with updated brightness channels from the HSI color space to the RGB color space.
6. The self-supervised water surface image enhancement method according to claim 1, wherein said step S50 further comprises:
s51, calculating a gray level histogram of the input image through counting the occurrence times of each gray level value in the image to obtain an original histogram;
s52, determining a clipping threshold, reducing gray values exceeding the clipping threshold in the original histogram to the clipping threshold to obtain a clipping histogram, and accumulating the clipped frequency numbers to form an additional frequency number pool;
s53, uniformly distributing the frequency numbers in the additional frequency number pool to the gray values which are not cut in the cutting histogram;
s54, calculating a cumulative histogram based on the clipping histogram and the clipping histogram after the frequency is reassigned, wherein the cumulative histogram represents the cumulative frequency of gray values of the image pixels;
s55, calculating an equalization map according to the cumulative histogram;
s56, applying an equalization map to each pixel of the input image, replacing the original gray value of each pixel with a new gray value.
7. The self-monitoring water surface image enhancement method according to claim 1, wherein said step S60 implements illumination compensation of the enhanced image by multiplying the enhanced image with illumination information of the original image.
8. A self-supervising surface image enhancement device, comprising:
the image denoising module is used for denoising and enhancing the water surface image by utilizing the fractional median filter and the fractional differential filter, and in the denoising and enhancing process, the parameters of the fractional median filter and the fractional differential filter are adaptively adjusted according to the characteristic information of the image;
the image preliminary enhancement module is used for carrying out image preliminary enhancement on the water surface image processed by the image denoising module, and improving the overall contrast of the image, adjusting the contrast of a reflection position area of the water surface image and inhibiting halation by utilizing the water surface brightness detail gain factor and the adjustment mapping;
the image brightness enhancement module is used for enhancing the image brightness by utilizing a guide filtering and multi-scale brightness enhancement method;
the image detail enhancement module is used for enhancing image details by utilizing the water surface brightness detail gain factor;
the image contrast enhancement module is used for enhancing the image contrast by utilizing a clipping histogram equalization method;
the illumination compensation module is used for carrying out illumination compensation on the enhanced image according to the illumination information of the original image;
and the image post-processing module is used for carrying out artifact removal, smooth filtering and normalization processing on the enhanced image.
9. An electronic device comprising at least one processor and at least one memory communicatively coupled to the processor, wherein the memory stores program instructions that when invoked by the processor for execution implement the self-supervising water surface image enhancement method according to any one of claims 1 to 7.
10. A computer storage medium storing program instructions which, when executed by a processor, implement the self-supervising water surface image enhancement method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310775963.8A CN116579953A (en) | 2023-06-28 | 2023-06-28 | Self-supervision water surface image enhancement method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310775963.8A CN116579953A (en) | 2023-06-28 | 2023-06-28 | Self-supervision water surface image enhancement method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116579953A true CN116579953A (en) | 2023-08-11 |
Family
ID=87536019
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310775963.8A Pending CN116579953A (en) | 2023-06-28 | 2023-06-28 | Self-supervision water surface image enhancement method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116579953A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116894793A (en) * | 2023-09-08 | 2023-10-17 | 南京道成网络科技有限公司 | Method and device for enhancing image quality |
-
2023
- 2023-06-28 CN CN202310775963.8A patent/CN116579953A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116894793A (en) * | 2023-09-08 | 2023-10-17 | 南京道成网络科技有限公司 | Method and device for enhancing image quality |
| CN116894793B (en) * | 2023-09-08 | 2023-11-28 | 南京道成网络科技有限公司 | Method and device for enhancing image quality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liang et al. | Single underwater image enhancement by attenuation map guided color correction and detail preserved dehazing | |
| Liu et al. | Efficient single image dehazing and denoising: An efficient multi-scale correlated wavelet approach | |
| CN107527332B (en) | Low-illumination image color retention enhancement method based on improved Retinex | |
| US11127122B2 (en) | Image enhancement method and system | |
| Bai et al. | Underwater image enhancement based on global and local equalization of histogram and dual-image multi-scale fusion | |
| KR101161471B1 (en) | Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing | |
| US10929967B2 (en) | Method and device for image processing | |
| US20040042676A1 (en) | Method and apparatus for illumination compensation of digital images | |
| US20090220169A1 (en) | Image enhancement | |
| JP4456819B2 (en) | Digital image sharpening device | |
| CN107292834B (en) | Infrared image detail enhancement method | |
| CN116579953A (en) | Self-supervision water surface image enhancement method and related equipment | |
| Li et al. | Single underwater image enhancement using integrated variational model | |
| CN111127377A (en) | Weak light enhancement method based on multi-image fusion Retinex | |
| Kansal et al. | Fusion-based image de-fogging using dual tree complex wavelet transform | |
| Singh et al. | Image enhancement by adaptive power-law transformations | |
| CN113344810A (en) | Image enhancement method based on dynamic data distribution | |
| CN111325680B (en) | Image brightening method capable of inhibiting local overexposure | |
| Jasper et al. | Natural image enhancement using a biogeography based optimization enhanced with blended migration operator | |
| Mortezaie et al. | An adaptive block based un-sharp masking for image quality enhancement | |
| Dasgupta | Comparative analysis of non-blind deblurring methods for noisy blurred images | |
| CN113487505A (en) | Infrared image mixed noise reduction method based on noise identification | |
| Elhefnawy et al. | Effective visibility restoration and enhancement of air polluted images with high information fidelity | |
| CN110807748A (en) | New tone mapping image enhancement method based on high dynamic range | |
| Pardhi et al. | Contrast Enhancement Using Adaptive Threshold Based Dynamic Range Adjustment In Luv Colour Space |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |