WO2021000302A1

WO2021000302A1 - Image dehazing method and system based on superpixel segmentation, and storage medium and electronic device

Info

Publication number: WO2021000302A1
Application number: PCT/CN2019/094616
Authority: WO
Inventors: 储颖; 游为麟; 罗国星; 朱泽轩
Original assignee: 深圳大学
Priority date: 2019-07-03
Filing date: 2019-07-03
Publication date: 2021-01-07

Abstract

Disclosed are an image dehazing method and system based on superpixel segmentation, and a storage medium and an electronic device, wherein same can achieve a great dehazing effect on a real haze image and a synthetic haze image data set. The method comprises: acquiring a global atmospheric light value corresponding to a haze image, and segmenting the haze image through superpixel segmentation to acquire a superpixel set corresponding to the haze image (S1); acquiring, by means of a preset cost function and on the basis of the superpixel set, an initial transmittance graph corresponding to the haze image (S2); performing refinement processing on the initial transmittance graph to acquire a target transmittance graph (S3); and acquiring, according to the target transmittance graph and the global atmospheric light value, a clear image corresponding to the haze image (S4).

Description

Image defogging method, system, storage medium and electronic equipment based on super pixel segmentation

Technical field

The present invention relates to the technical field of image processing, and more specifically, to an image defogging method, system, storage medium and electronic equipment based on super pixel segmentation.

Background technique

At present, image defogging methods can be divided into two categories: the first category is defogging methods based on image enhancement, and the second category is defogging methods based on physical models.

The defogging method based on image enhancement does not consider the principle of image degradation, does not establish a complex physical model, and directly uses conventional image enhancement techniques to improve image contrast, color saturation, and clarity, and its defogging effect is limited.

The basic idea of the defogging method based on the physical model is to establish a physical model of image degradation under haze weather and restore the optical process of image degradation. The reverse method is used to compensate the information loss in the degradation process, so as to obtain a clear image. Compared with defogging methods based on image enhancement, this type of method can retain more valuable information in the image, and the defogging results are more realistic and natural. The current mainstream model is a physical model based on atmospheric scattering. In the application of defogging algorithms, it can be divided into several categories based on prior knowledge and based on machine learning. It specifically includes:

(1) Contrast prior:

Compared with a non-fog image, the contrast of a foggy image is lower. You can defog by maximizing the contrast of the image. This type of method stipulates that the contrast of an image is represented by the gradient of the image, that is, the more obvious the edge of the image, the higher the contrast. The formula is expressed as follows:

Among them, C _edges (I) represents the sum of the image gradient information, x represents the pixel point in the image, and I _c (x) represents the gradient value at point x.

However, this kind of method is prone to oversaturate the image, and halo is prone to appear at the edge of the scene with sudden depth changes.

(2) Dark channel prior

For most outdoor images without fog, the minimum value of the color channel is taken for each pixel, and the resulting image is called the dark channel. For outdoor images without fog, the dark channel pixel value is very low, and the more severe the haze, the higher the dark channel pixel value.

Among them, J _dark (x) represents the dark channel image composed of the minimum value of the three color channels of the image.

In a fog-free image, the dark channel value approaches zero. Therefore, by calculating the expression of the dark channel in the atmospheric scattering model and setting it to zero, a dehazing image is obtained.

The disadvantage of the dark channel a priori dehazing is that when the haze image contains a sky area, the image will be distorted in the sky area after dehazing.

(3) Color prior

The image surface shadow and the atmospheric transfer function are statistically uncorrelated in the local area of the image. It expresses the clear image J(x) in the atmospheric scattering model as the product of the transmittance map and the surface reflection coefficient R, and then decomposes R and the clear image I(x) into components R _A and I in the direction parallel to the atmospheric light. _a (x), the vertical resolution of the atmosphere in the direction of the light components R 'and I _R', transmittance FIG follows:

Among them, R _A is the surface reflection coefficient, R'represents the residual vector perpendicular to the atmospheric light, and I _A (x) is the component parallel to the atmospheric light, and the Markov random field is used to restore the transmittance map t(x).

This kind of method is based on the atmospheric scattering model, which can solve the depth map. However, since the haze image needs to have rich color information, it is not applicable to the dense fog image.

(4) Dehazing method based on linear regression and color attenuation prior. Since the color saturation of a clear image is similar to the brightness, the color saturation of the haze image will decrease and the brightness will increase. Therefore, saturation and brightness are used to estimate the concentration of haze, and because the attenuation rate of incident light is related to the depth of the scene, it uses a linear model of color saturation and brightness to predict the depth of the scene, which is expressed as follows:

d(x)=θ ₀ +θ ₁ k(x)+θ ₂ c(x)+ε(x)

Among them, k represents brightness, c represents color saturation, ε represents random error, d represents scene depth, and θ ₁ , θ ₂ and θ ₃ are the parameters of the linear regression model.

After that, the global atmospheric light value A is estimated according to the depth of the scene and combined with the atmospheric scattering model to derive the original image. This method is easy to fail when the haze density is large and the color characteristics are not obvious, and it is only suitable for the situation where the color saturation of the background object is high or the haze degree is low.

(5) Dehazing method based on cost function.

The process of image dehazing algorithm based on cost function is as follows:

First, cut the image into blocks, and assume that the transmittance maps within the blocks are consistent.

After that, design a cost function about image contrast and information entropy, and solve the intra-block transmittance map by minimizing the cost function. The formula is as follows:

L=L _contrast +λL _info

Among them, L represents the overall cost function in the block, L _contrast represents the cost function of the intra-block contrast, _Linfro represents the cost function of the information entropy in the block, and λ represents the weight of the information entropy cost function in the overall cost function.

After that, the intra-block contrast and information entropy are maximized by minimizing the cost function, so as to achieve the overall image defogging effect.

In the existing solutions, the cost function-based image defogging method directly divides the image to calculate the optimal transmittance map value in each rectangular block. The transmittance map value is shared in the rectangular block, but the depth of the scene in the rectangular block may be inconsistent. Therefore, this method may cause errors in the estimation of the transmittance map.

In summary, the performance of existing image defogging methods still has a large room for improvement, and it is necessary to improve.

Summary of the invention

The technical problem to be solved by the present invention is to provide an image defogging method, system, storage medium, and electronic device based on superpixel segmentation in view of the above-mentioned prior art defects of the prior art.

The technical solution adopted by the present invention to solve its technical problems is to construct an image defogging method based on super pixel segmentation, including:

S1. Obtain the global atmospheric light value corresponding to the haze image, and segment the haze image by super pixel segmentation to obtain a super pixel set corresponding to the haze image;

S2. Obtain an initial transmittance map corresponding to the haze image through a preset cost function based on the super pixel set;

S3. Refine the initial transmittance map to obtain a target transmittance map;

S4. Obtain a clear image corresponding to the haze image according to the target transmittance map and the global atmospheric light value.

Preferably, in the step S1, the obtaining the global atmospheric light value corresponding to the haze image includes:

S111. Divide the haze image into four regions.

S112. Obtain the difference between the average value of the pixels and the standard deviation of each area, and obtain the area corresponding to the maximum difference.

S113: Determine whether the area is smaller than a preset value, if not, perform step S114, if yes, perform step S115;

S114: Divide the area into four areas, and execute step S112;

S115. Obtain the average value of pixels in the area to set it as the global atmospheric light value corresponding to the haze image; and/or

In the step S1, the segmenting the haze image through super pixel segmentation to obtain the super pixel set corresponding to the haze image includes: segmenting the haze image through SLIC super pixel segmentation; and/or

In the step S2, the obtaining an initial transmittance map corresponding to the haze image through a preset cost function based on the super pixel set includes:

S21: Acquire a first cost function corresponding to the contrast of the super pixel set, and a second cost function corresponding to the information entropy of the super pixel set;

S22: Obtain a third cost function corresponding to the super pixel set based on the first cost function and the second cost function;

S23. Perform iteration based on the third cost function to obtain the transmittance map corresponding to the minimum value of the third cost function as the initial transmittance map corresponding to the haze image; and/or

In the step S3, the performing refinement processing on the initial transmittance map to obtain the target transmittance map includes: performing refinement processing on the initial transmittance map based on guided filtering; and/or

In the step S4, the obtaining a clear image corresponding to the haze image according to the transmittance map and the global atmospheric light value includes: obtaining a clear image corresponding to the haze image by using the following formula,

Among them, I(x) is the haze image, J(x) is the clear image, t is the target transmittance map, and A is the global atmospheric light value.

Preferably, the first cost function is a contrast cost function, and the contrast cost function satisfies the following formula:

Where x is the position of the pixel, c∈{r,g,b} is a certain color channel of the pixel x, D is the superpixel area corresponding to any superpixel set, and J _c (x) is the clear image The pixel value in the c channel, N _x is the number of pixels in the super pixel area;

Is the average value of J _c (x) of the super pixel area, N _x represents the number of pixels in any super pixel set, I _c (x) represents the pixel value of pixel x in color channel c,

Is the average value of I _c (x) in the haze image area;

The second cost function is an information entropy cost function, and the information entropy cost function satisfies the following formula:

Among them, min{0, J _c (p)}, max{0, J _c (p)-255} represent the overflow value of pixel underflow and overflow respectively, and h _c (i) represents the histogram of the input pixel Take the value, α _c and β _c represent the pixel value that is truncated;

The third cost function satisfies the following formula:

L=L _contrast +λ _D L _info ,

Among them, L _contrast represents the contrast cost function, L _info represents the information entropy cost function, and λ _D is the weight parameter that coordinates the contrast loss and the information entropy loss.

Preferably, the value of λ _D is 6.

Preferably, the segmentation of the haze image by SLIC superpixel segmentation includes:

S121: Perform color space conversion on the haze image to obtain a CIELab color space, and obtain an initial center point of the haze image according to a preset size value of the superpixel set;

S122: Perform five-dimensional clustering on the pixel points of the haze image and the CIELab color space based on the initial center point to obtain an initial superpixel set;

S123: Obtain a gradient value of a pixel point of the initial super pixel set, and correct the initial center point corresponding to the minimum gradient value to obtain a corrected initial center point;

S124. Perform five-dimensional clustering on the pixel points of the haze image and the CIELab color space based on the corrected initial center point to obtain a corrected initial superpixel set, and count once to determine the current And whether the previous count times meet the pre-designed value, if not, execute the step S123; if yes, execute the step S125;

S125. Use the corrected initial super pixel set as a super pixel set corresponding to the haze image.

Preferably,

The preset size value of the super pixel set satisfies the requirement of greater than 300 pixels and less than 1500 pixels; and/or

The preset count value is 10.

Preferably, the preset size value of the super pixel set is 900 pixels.

The present invention also constructs an image defogging system based on super pixel segmentation, including:

The first processing unit is configured to obtain the global atmospheric light value corresponding to the haze image;

A segmentation unit, configured to segment the haze image through super pixel segmentation to obtain a super pixel set corresponding to the haze image;

A second processing unit, configured to obtain a transmittance map corresponding to the haze image through a preset cost function based on the super pixel set;

A third processing unit, configured to refine the initial transmittance map to obtain a target transmittance map;

The fourth processing unit is configured to obtain a clear image corresponding to the haze image according to the target transmittance map and the global atmospheric light value.

The present invention also constructs a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method for image defogging based on superpixel segmentation as described in any one of the above is realized.

The present invention also constructs an electronic device, including a memory and a processor;

The memory is used to store computer programs;

The processor is configured to execute the computer program to implement the image defogging method based on superpixel segmentation as described in any one of the above.

An image defogging method, system, storage medium, and electronic device based on superpixel segmentation implemented in the present invention have the following beneficial effects: both real haze images and synthetic haze image data sets have good defogging effects.

Description of the drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments. In the accompanying drawings:

FIG. 1 is a program flowchart of an embodiment of an image defogging method based on superpixel segmentation of the present invention;

2 is a program flowchart of another embodiment of the image defogging method based on super pixel segmentation of the present invention;

Figure 3 is an example diagram of global atmospheric light value estimation results;

4 is a program flowchart of another embodiment of the image defogging method based on super pixel segmentation of the present invention;

Figure 5 is a schematic diagram of the comparison of super pixel areas;

Fig. 6 is a schematic diagram of a clustering search area of pixels;

Figures 7-10 are schematic diagrams of iterative SLIC superpixel segmentation algorithm;

FIG. 11 is a program flowchart of another embodiment of an image defogging method based on superpixel segmentation according to the present invention;

Figure 12 is a schematic diagram of pixel cut-off;

Figure 13 shows the image defogging effect under different λ _D ;

Figures 14-15 are schematic diagrams showing the comparison of transmittance diagrams of different defogging methods;

Figures 16-17 are schematic diagrams showing the comparison of defogging effects of haze images with different defogging methods.

Detailed ways

In order to have a clearer understanding of the technical features, objectives and effects of the present invention, the specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

As shown in FIG. 1, in the first embodiment of the image defogging method based on superpixel segmentation of the present invention, the method includes:

S1. Obtain the global atmospheric light value corresponding to the haze image, and segment the haze image by superpixel segmentation to obtain the superpixel set corresponding to the haze image; S2, obtain the haze through the preset cost function based on the superpixel set The initial transmittance map corresponding to the image; S3, the initial transmittance map is refined to obtain the target transmittance map; S4, the clear image corresponding to the haze image is obtained according to the target transmittance map and the global atmospheric light value.

Specifically, the atmospheric scattering model, which is widely used in the field of image processing under haze weather, describes the atmospheric scattering process and the principle of ambient light attenuation. The detailed technical principle of the atmospheric scattering model under a single light source. This model divides the light reaching the imaging device into two parts: one is directly attenuating the light, and the reflected light of the scene is scattered by particles in the air during the process of propagating to the imaging device The effect is that the incident light attenuation occurs, which is called direct light attenuation; the other part is that atmospheric light directly acts on the suspended particles in the air, is received by the imaging device after being scattered, and overlaps on the target image, which is called additional scattered light. Usually these two parts of light exist, the lower the haze degree, the higher the proportion of the direct attenuation light in the image; the higher the haze degree, the higher the proportion of additional scattered light in the image. On the basis of the above, in the field of image defogging, refer to the following formula according to the relationship between the foggy image and the defogging image.

I(x)=J(x)t(x)+A(1-t(x)) (1)

Among them, I(x) is the haze image, A is the global atmospheric light value, t(x) is the transmittance map, and J(x) is the clear image. On the basis of the above, first obtain the global atmospheric light value corresponding to the haze image, and then obtain the transmittance map corresponding to the haze image. The process of obtaining the perspective ratio corresponding to the haze image is first through pixel clustering to obtain compact, Approximate set of superpixels. At the same time, the cost function is defined, and the optimal solution of the transmittance map in the super pixel set is obtained by minimizing the cost function, which is the initial transmittance map of the haze image. The design of the cost function can be based on a variety of image characteristics, such as contrast, information entropy, saturation and other characteristics. Since the direct use of the transmittance map acquired based on the cost function will produce blockiness, it is necessary to refine the acquired initial transmittance map so that the transmittance map can obtain a texture close to the haze image to obtain the target transmission corresponding to the haze image Rate graph.

Further, as shown in FIG. 2, in step S1, obtaining the global atmospheric light value corresponding to the haze image includes:

S111. Divide the haze image into four regions;

S114: Divide the area into four areas, and execute step S112;

S115. Obtain an average value of pixels in the area to set the global atmospheric light value corresponding to the haze image;

Specifically, the traditional method for estimating the global atmospheric light value is to take a small number of pixels with the largest brightness in the picture, and use the average value of each channel of these pixels as the global atmospheric light value. However, if artificial light sources such as car lights and street lights appear in the picture, the global atmospheric light value may be estimated incorrectly. In order to improve the estimation accuracy of global atmospheric light value, this paper adopts a quadtree estimation method based on contrast and brightness. The specific process is to evenly divide the haze image into four regions. After that, subtract the standard deviation from the average value of each area pixel to ensure that the area with the largest value has the largest average brightness and the smallest contrast. Finally, loop this process until the pixels in the area are less than the preset value. At this time, the average brightness of each channel in the area is the global atmospheric light estimate. Figure 3 shows an example graph of the global atmospheric light estimation results. By selecting the average brightness minus the area block with the largest contrast value, the maximum value of each channel brightness in the white highlight area is finally selected as the global atmospheric light value of this haze image.

Optionally, in step S1, segmenting the haze image through super pixel segmentation to obtain a super pixel set corresponding to the haze image includes: segmenting the haze image through SLIC super pixel segmentation; specifically, SLIC super pixel Segmentation is an image segmentation algorithm with simple thinking and easy implementation. The algorithm has fast execution speed and can maintain the contour of the object relatively completely. The generated super pixel blocks are uniformly distributed, compact in structure, similar in features, and easy to convert with pixel-based methods. In other embodiments, other image segmentation algorithms can also produce similar segmentation effects.

Further, as shown in Figure 4, segmenting the haze image through SLIC superpixel segmentation includes:

S121. Perform color space conversion on the haze image to obtain the CIELab color space, and obtain the initial center point of the haze image according to the preset size value of the superpixel set; specifically, although the image can be directly SLIC in the RGB space or the CIELab space Super pixel segmentation. But the image segmentation effect in CIELab space is better than in other color spaces, and the segmented super pixel blocks are more detailed. The number of center points is related to the average size of the superpixel set. The method of calculating the center point is to divide the total number of pixels of the image by the size of the super pixel set and round up. The initial size of the super pixel set needs to be predefined. The initial center point is defined as the seed point, and the number of center points is the same as the number of super pixel sets and remains constant during iterations. SLIC super pixel segmentation will uniformly distribute seed points in the image according to the predefined super pixel set size. The preset size of the super pixel set should not be too large, and the depth of the scene in the super pixel set should be consistent. In the process of super pixel segmentation, you can define the average size of the super pixel set, that is, the average number of pixels in the super pixel. The selection principle is to keep the scene depth in the super pixel set consistent. Figure 5 shows the effect of the average superpixel set size on the segmentation results. Where (a) is the original haze image, (b) is the super pixel set size of 1500, (c) is the super pixel set size of 900, and (d) is the super pixel set size of 300. When the size of the super pixel set is 1500 pixels, because the value is too large, the consistency of the depth of field in some areas cannot be guaranteed. When the set size is set to 900, it can basically ensure the same depth of field in the super pixel set. When the parameter is set to 300, the total number of super pixel sets is too large, which will increase the amount of calculation. In this embodiment, the super pixel set size is set to 900.

S122. Perform five-dimensional clustering of the pixels of the haze image by coordinates and CIELab color space based on the initial center point to obtain the initial superpixel set; specifically, the SLIC superpixel segmentation is a clustering algorithm that specifies each pixel of the picture The (x, y) coordinate value and (L, a, b) color value form a five-dimensional vector [x, y, L, a, b], and the similarity of two pixels is measured by the vector distance between them . As shown in FIG. 6 for the cluster search area of pixels, the search range of each pixel is a 2S×2S area, where S is the distance between the initial seed points. This local clustering strategy can accelerate convergence and maintain the connectivity of regional blocks.

S123. Obtain the gradient values of the pixels of the initial superpixel set, and correct the initial center point and the minimum gradient value to obtain the corrected initial center point; specifically, calculate the gradient values of all pixels in each superpixel set, Move the center point to the smallest gradient in the neighborhood. This can prevent the center point from falling on the boundary with a larger gradient, which will affect the subsequent clustering effect.

S124. Perform five-dimensional clustering on the pixel points of the haze image according to the coordinates and CIELab color space based on the corrected initial center point to obtain the corrected initial superpixel set, and count it once to determine whether the current and past count times are Meet the pre-designed value, if not, go to step S123; if yes, go to step S125;

S125. Use the corrected initial super pixel set as the super pixel set corresponding to the haze image. Specifically, after the position of the new center point is confirmed, five-dimensional clustering is performed on the pixel coordinates of the haze image and the CIELab color space based on the corrected initial center point to obtain a corrected initial superpixel set for iteration. When the iteration exceeds a certain number of times, the pre-designed value, the image superpixel segmentation result will no longer change. After 10 iterations of the SLIC superpixel segmentation method, the segmentation results usually start to become stable. Figures 7 to 10 show the segmentation results of 4 iterations, 6, 8, and 10 iterations respectively. It can be seen from Figures 7 and 8 that after 6 iterations, the TV tower area in the picture is cut corresponding to the box mark area in the figure. Come out; Figure 9 and Figure 10 show that the segmentation results of 8 iterations and 10 iterations do not change much.

In this embodiment, the upper limit of the number of iterations of SLIC superpixel segmentation is set to 10 times.

Optionally, as shown in FIG. 11, in step S2, obtaining the initial transmittance map corresponding to the haze image through a preset cost function based on the super pixel set includes:

S21: Obtain a first cost function corresponding to the contrast of the super pixel set, and a second cost function corresponding to the information entropy of the super pixel set;

S23: Perform iteration based on the third cost function to obtain the transmittance map corresponding to the minimum value of the third cost function as the initial transmittance map corresponding to the haze image;

Specifically, after completing the superpixel segmentation image, it is necessary to estimate the transmittance map in the superpixel set. In the calculation process of the transmittance map, the design of the cost function can be based on a variety of image characteristics, such as contrast, information entropy, saturation and other characteristics. In this embodiment, contrast and information entropy are selected. In order to improve the information entropy and contrast of the defogging image, in this embodiment, the third cost function is used to balance the information loss and contrast loss in the super pixel set at the same time.

Further, the first cost function is a contrast cost function, and the contrast cost function satisfies the following formula:

Is the average value of J _c (x) in the super pixel area, N _x represents the number of pixels in any super pixel set, and I _c (x) represents the pixel value of pixel x in color channel c,

Is the average value of I _c (x) in the haze image area;

The second cost function is the information entropy cost function, and the information entropy cost function satisfies the following formula:

The third cost function satisfies the following formula:

L=L _contrast +λ _D L _info (4)

Specifically, after the image is divided, the scene depth in the super pixel set is the same, and the transmittance map is also the same. The formula (1) is modified to obtain the clear image corresponding to the super pixel set as:

Among them, A is the global atmospheric light value, I(x) is the haze image, and t is the target transmittance map.

In this embodiment, the mean square error contrast C _MSE can be used to evaluate the contrast in the image area block to be restored. The formula is as follows:

Among them, c∈{r,g,b} represents a certain color channel of pixel x. J _c (x) represents the pixel value of the pixel point x of the clear image in the block of the color channel c.

Is the average value of J _c (x) in the block, and N represents the number of pixels in the block. Substitute formula (5) into (6) to obtain:

Among them, I _c (x) represents the pixel value of pixel x in color channel c,

Is the average value of I _c (x) in the haze image area.

It can be seen from formula (7) that the contrast C _MSE is a decreasing function of the transmittance t, that is, the smaller the transmittance t, the higher the contrast. Therefore, the contrast loss function L _contrast can be defined as in formula (2). It can be seen that the smaller the contrast cost function L _contrast , the greater the image contrast. Therefore, the contrast within the superpixel set can be maximized by minimizing the contrast cost function.

At the same time, after determining the global atmospheric light value A and the intra-superpixel transmittance t, the mapping between the input pixel value I _c (x) and the output pixel value J _c (x) can be obtained according to formula (5). As shown in Figure 12, if the input pixel value is in the range of [α, β], the output value mapping interval is guaranteed to be [0, 255]. The effective range [α, β] of the input pixel value is determined by the transmittance map t. When all the pixel values of the super pixel set belong to [α, β], the output pixel value is distributed in [0,255], and the image contrast is high; if the pixel value in the super pixel set does not belong to [α, β], the output pixel value exceeds gray Valid area of degree value [0,255]. In this case, the output pixel values beyond the legal area are cut off, information loss occurs (as shown in the black area in FIG. 12), and the image information entropy decreases. The analysis shows that the larger the transmittance graph t, the smaller the area of the black area, and the greater the length of the [α, β] interval. That is: the larger the transmittance map t, the fewer pixels to be truncated in the image to be restored, the less information is lost, and the higher the information entropy. Therefore: the information entropy cost function L _info can be defined as in formula (3), by minimizing the information entropy cost function L _info , the information entropy loss in the super pixel set can be minimized.

After obtaining the contrast cost function and the information entropy cost function, an improved cost function related to the contrast cost function and the information entropy cost function is set to improve the image contrast and reduce the information loss. It satisfies the formula (4). A larger value of λ _D can reduce the information entropy loss. When λ _{D is} infinite, no information entropy loss occurs. At this time:

Among them, _Ac represents the global atmospheric light value in channel c, D represents the super pixel area, and I _c (x) represents the pixel value of pixel x in color channel c.

Combine formulas (8) and (9):

It can be seen from formula (10) that the smaller the transmittance graph t, the smaller the contrast loss function L _contrast , that is, the higher the contrast. Therefore, under the condition that the loss of information entropy is acceptable, the value of the transmittance map should be:

In summary, the constraint conditions of formula (10) are the same as those in the dark channel prior. Formula (11) constrains the gray value overflow that may occur after the pixels are defogged, which can be regarded as a priori method for the dark channel Supplement. In this paper, the improved algorithm is named the superpixel cost function algorithm. The experimental results show that this method can more effectively estimate the transmittance map t. At the same time, by adjusting the value of the parameter λ _D , a better balance can be achieved between increasing the contrast and reducing the loss of information entropy.

The weight parameter λ _D proposed in formula (4) has the meaning of weighing the importance of improving contrast and reducing information entropy. According to the defogging effect of different values of λ _{D in} Figure 13, (a) is the original haze image, (b) is the defogging effect of λ _D =3, (c) is the defogging effect of λ _D =6, (d) The defogging effect of λ _D =10 can be obtained: When λ _D =3, the contrast of the image after defogging is improved, but because more information is cut off, the pixels in the picture are too dark and too bright More points. When λ _D =10, the unnatural pixels caused by the loss of information are reduced, but the contrast is too low to completely remove the fog. When λ _D =6, a balance is struck between improving image contrast and suppressing information loss. Therefore, λ _{D is} set to 6 in the image defogging algorithm of super pixel cost function.

Optionally, in step S3, performing refinement processing on the initial transmittance map to obtain the target transmittance map includes: performing refinement processing on the initial transmittance map based on guided filtering; specifically, a dehazing method based on an atmospheric scattering model Among them, many methods, such as the dark channel prior method, the color attenuation prior method, etc., have insufficiently refined transmittance maps. The commonly used transmittance map refinement methods include soft matting method and guided filtering method. In this article, the original transmittance map obtained by the cost function defogging method also needs to be refined. If it is used directly, it will cause blocking. In view of the good refinement effect and high speed of the guided filtering method, the guided filtering is used in this example to refine the transmittance map. The specific process is to filter the input image with the guidance image, so that the input image retains the original features while obtaining the texture of the guidance image. If the output image is t, the relationship between the guidance image I(x) and the output image is as follows:

Among them, I(x) represents the guidance image, W _xy (I(x)) represents the weight used in the weighted average operation determined by the guidance image, t(x) is the input image, and β is the offset. The guided image for guided filtering may be the input image itself. When the input image is used as a guide image, the role of the guide filter is degenerated into an edge preservation filter. The input image here is the initial transmittance map of the haze image, and the output image is the target transmittance map of the haze image. Both the cost function defogging method and the SLIC superpixel cost function defogging method use guided filtering to refine the transmittance map. The transmittance maps of different defogging methods have different edge retention effects. A better defogging method, the texture of the transmittance map is closer to the original image. Figure 14 and Figure 15 show the comparison of the traditional cost function image defogging method and the superpixel cost function image defogging method. Among them (a) is the haze image, (b) cost function, (c) cost function refinement, (d) is the super pixel method, (e) is the super pixel method refinement, (f) is the cost function detail, ( g) is the super pixel detail. (e) is the defogging effect picture refined by the super pixel method through guided filtering.

Compare Figure 14 (b) and Figure 14 (c), and enlarge the details to Figure 15 (f) and Figure 15 (g). It can be seen that the edge of the first row of leaves in the box, the second row of hillsides and At the third row of haystacks, the transmittance map of the SLIC superpixel cost function is more detailed than the transmittance map of the traditional cost function defogging method, and the extracted texture is closer to the haze image.

Optionally, in step S4, obtaining a clear image corresponding to the haze image according to the transmittance map and the global atmospheric light value includes: obtaining a clear image corresponding to the haze image by using the following formula:

Among them, A is the global atmospheric light value, I(x) is the haze image, and t is the target transmittance.

As shown in Figure 16 and Figure 17, (a) is the haze image, (b) is the defogging image of the histogram equalization method, and (c) is the defogging image of the Retinex method. It can be seen that both methods exist Partially distorted, and the image color is not coordinated. (d) is the defogging image of the dark channel prior defogging method, in which the sky part is over-enhanced. (e) is the defogging image of the cost function, (f) is the defogging image of the superpixel cost function algorithm, both of which are more natural, but the image brightness of the superpixel cost function defogging algorithm is higher than the cost function .

In order to objectively compare the effect of defogging, a typical objective image quality evaluation algorithm is used to compare the effects of different defogging methods. Including: structure similarity, peak signal-to-noise ratio, gray-scale variance, Laplacian gradient and entropy function five methods. Table 1 and Table 2 respectively show the objective evaluation results of image quality of different defogging algorithms on the HID2018 haze image database and NYU synthetic haze image database. The super pixel cost function corresponds to the image defogging method based on super pixel of the present invention. Table 1 shows the comparison of objective image quality evaluation results of different defogging algorithms on the HID2018 dataset, and Table 2 shows the comparison of objective evaluation results of different defogging algorithms on the NYU synthetic haze image data set.

Table 1

Table 2

In the image quality evaluation algorithm, the structural similarity and the peak signal-to-noise ratio belong to the full-reference image quality evaluation method, which requires fog-free images for comparison. Therefore, the NYU synthetic haze image database was selected for testing. The database contains original clear images and synthetic haze images. The gray variance (SMD), Laplacian gradient function and entropy function are non-reference image quality evaluation methods, which can be directly tested using real haze images in the HID2018 database. It can be seen from Table 1 that on the HID2018 data set, when SMD and Laplacian gradient function indicators are used, the performance of the superpixel-based image defogging method of the present invention is slightly lower than that of the cost function defogging method; and when Entropy is used, The performance of the superpixel-based image defogging method of the present invention is better than other methods. On the NYU data set, the superpixel-based image defogging method of the present invention has the best performance. In summary, the superpixel-based image defogging method of the present invention has a good defogging effect on both the real haze image and the synthetic haze image data set.

In addition, the image defogging system based on superpixel segmentation of the present invention includes:

The first processing unit is used to obtain the global atmospheric light value corresponding to the haze image;

The segmentation unit is used to segment the haze image through super pixel segmentation to obtain the super pixel set corresponding to the haze image;

The second processing unit is configured to obtain a transmittance map corresponding to the haze image through a preset cost function based on the super pixel set;

The third processing unit is used to refine the initial transmittance map to obtain the target transmittance map;

The fourth processing unit is used to obtain a clear image corresponding to the haze image according to the target transmittance map and the global atmospheric light value.

Specifically, the specific coordination operation process between the units of the image defogging system based on superpixel segmentation can refer to the above-mentioned image defogging method based on superpixel segmentation, which will not be repeated here.

In addition, an electronic device of the present invention includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to implement any of the above-mentioned methods for image defogging based on superpixel segmentation. Specifically, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, when the computer program can be downloaded and installed by an electronic device and executed, it executes the above-mentioned functions defined in the method of the embodiment of the present invention. The electronic device in the present invention can be a terminal such as a notebook, a desktop computer, a tablet computer, a smart phone, etc., or a server.

In addition, a computer storage medium of the present invention has a computer program stored thereon, and when the computer program is executed by a processor, any one of the above methods for image defogging based on superpixel segmentation is realized. Specifically, it should be noted that the above-mentioned computer-readable medium of the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium. The computer-readable signal medium may send, propagate or transmit the program for use by or in combination with the instruction execution system, apparatus, or device . The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.

It is understandable that the above examples only express the preferred embodiments of the present invention, and the description is more specific and detailed, but it should not be construed as a limitation on the scope of the present invention; it should be pointed out that for those of ordinary skill in the art In other words, without departing from the concept of the present invention, the above technical features can be freely combined, and several modifications and improvements can be made. These all belong to the scope of protection of the present invention; therefore, everything that follows the scope of the claims of the present invention All equivalent changes and modifications shall fall within the scope of the claims of the present invention.

Claims

An image defogging method based on superpixel segmentation, which is characterized in that it includes:

S1. Obtain the global atmospheric light value corresponding to the haze image, and segment the haze image by super pixel segmentation to obtain a super pixel set corresponding to the haze image;

S2. Obtain an initial transmittance map corresponding to the haze image through a preset cost function based on the super pixel set;

S3. Refine the initial transmittance map to obtain a target transmittance map;

S4. Obtain a clear image corresponding to the haze image according to the target transmittance map and the global atmospheric light value.
The image defogging method based on superpixel segmentation according to claim 1, wherein:

In the step S1, the obtaining the global atmospheric light value corresponding to the haze image includes:

S111. Divide the haze image into four regions.

S112. Obtain the difference between the average value of the pixels and the standard deviation of each area, and obtain the area corresponding to the maximum difference.

S113: Determine whether the area is smaller than a preset value, if not, perform step S114, if yes, perform step S115;

S114: Divide the area into four areas, and execute step S112;

S115. Obtain the average value of pixels in the area to set it as the global atmospheric light value corresponding to the haze image; and/or

In the step S1, the segmenting the haze image through super pixel segmentation to obtain the super pixel set corresponding to the haze image includes: segmenting the haze image through SLIC super pixel segmentation; and/or

In the step S2, the obtaining an initial transmittance map corresponding to the haze image through a preset cost function based on the super pixel set includes:

S21: Acquire a first cost function corresponding to the contrast of the super pixel set, and a second cost function corresponding to the information entropy of the super pixel set;

S22: Obtain a third cost function corresponding to the super pixel set based on the first cost function and the second cost function;

S23. Perform iteration based on the third cost function to obtain the transmittance map corresponding to the minimum value of the third cost function as the initial transmittance map corresponding to the haze image; and/or

In the step S3, the performing refinement processing on the initial transmittance map to obtain the target transmittance map includes: performing refinement processing on the initial transmittance map based on guided filtering; and/or

In the step S4, the obtaining a clear image corresponding to the haze image according to the target transmittance map and the global atmospheric light value includes: obtaining a clear image corresponding to the haze image by using the following formula,

Among them, I(x) is the haze image, J(x) is the clear image, t is the target transmittance map, and A is the global atmospheric light value.
The method for image defogging based on superpixel segmentation according to claim 2, wherein the first cost function is a contrast cost function, and the contrast cost function satisfies the following formula:

Where x is the position of the pixel, c∈{r,g,b} is a certain color channel of the pixel x, D is the superpixel area corresponding to any superpixel set, and J c (x) is the clear image The pixel value in the c channel, N x is the number of pixels in the super pixel area;
Is the average value of J c (x) of the super pixel area, N x represents the number of pixels in any super pixel set, I c (x) represents the pixel value of pixel x in color channel c,
Is the average value of I c (x) in the haze image area;

The second cost function is an information entropy cost function, and the information entropy cost function satisfies the following formula:

Among them, min{0, J c (p)}, max{0, J c (p)-255} represent the overflow value of pixel underflow and overflow respectively, and h c (i) represents the histogram of the input pixel Take the value, α c and β c represent the pixel value that is truncated;

The third cost function satisfies the following formula:

L=L contrast +λ D L info ,

Among them, L contrast represents the contrast cost function, L info represents the information entropy cost function, and λ D is the weight parameter that coordinates the contrast loss and the information entropy loss.
The image defogging method based on super pixel segmentation according to claim 3, wherein the value of λ D is 6.
The method for image defogging based on superpixel segmentation according to claim 2, wherein said segmenting the haze image by SLIC superpixel segmentation comprises:

S121: Perform color space conversion on the haze image to obtain a CIELab color space, and obtain an initial center point of the haze image according to a preset size value of the superpixel set;

S122: Perform five-dimensional clustering on the pixel points of the haze image and the CIELab color space based on the initial center point to obtain an initial superpixel set;

S123: Obtain a gradient value of a pixel point of the initial super pixel set, and correct the initial center point corresponding to the minimum gradient value to obtain a corrected initial center point;

S124. Perform five-dimensional clustering on the pixel points of the haze image and the CIELab color space based on the corrected initial center point to obtain a corrected initial superpixel set, and count once to determine the current And whether the previous count times meet the pre-designed value, if not, execute the step S123; if yes, execute the step S125;

S125. Use the corrected initial super pixel set as a super pixel set corresponding to the haze image.
The image defogging method based on superpixel segmentation according to claim 5, characterized in that,

The preset size value of the super pixel set satisfies greater than 300 pixels and less than 1500 pixels;

The preset count value is 10.
The image defogging method based on super pixel segmentation according to claim 6, wherein the preset size value of the super pixel set is 900 pixels.
An image defogging system based on superpixel segmentation, which is characterized in that it includes:

The first processing unit is configured to obtain the global atmospheric light value corresponding to the haze image;

A segmentation unit, configured to segment the haze image through super pixel segmentation to obtain a super pixel set corresponding to the haze image;

A second processing unit, configured to obtain a transmittance map corresponding to the haze image through a preset cost function based on the super pixel set;

A third processing unit, configured to refine the initial transmittance map to obtain a target transmittance map;

The fourth processing unit is configured to obtain a clear image corresponding to the haze image according to the target transmittance map and the global atmospheric light value.
A computer storage medium having a computer program stored thereon, wherein the computer program implements the image defogging method based on superpixel segmentation according to any one of claims 1 to 7 when the computer program is executed by a processor.
An electronic device, characterized in that it includes a memory and a processor;

The memory is used to store computer programs;

The processor is configured to execute the computer program to implement the image defogging method based on superpixel segmentation according to any one of claims 1 to 7.