CN103500440A

CN103500440A - Method for eliminating cloud and haze of atmospheric degraded image

Info

Publication number: CN103500440A
Application number: CN201310454978.0A
Authority: CN
Inventors: 南栋; 毕笃彦; 王晨; 查宇飞; 何林远
Original assignee: Air Force Engineering University of PLA
Current assignee: Air Force Engineering University of PLA
Priority date: 2013-09-27
Filing date: 2013-09-27
Publication date: 2014-01-08

Abstract

The invention discloses a method for removing clouds and fog in atmospheric degradation images, comprising the following steps: acquisition of dark channel images and median filtering, adaptive decomposition and acquisition of atmospheric light images; refinement of atmospheric transfer functions; and visual correction of color gamut. The atmospheric degradation images processed by the present invention have different levels of illumination, contrast, and dynamic range problems. The dark channel theoretical model is adopted, and the median filter of the dark channel image, the adaptive decomposition and acquisition of the atmospheric light image and the visual correction of the color gamut are integrated into the image. De-cloud and fog are carried out to make up for the deficiencies in the prior art. In addition, the invention is easy to operate and has good application prospects.

Description

A Method for Removing Clouds and Fog in Images of Atmospheric Degradation

技术领域technical field

本发明属于计算机与图像处理技术领域，尤其涉及一种去除大气退化图像云雾的方法。The invention belongs to the technical field of computer and image processing, and in particular relates to a method for removing clouds and fog in images of atmospheric degradation.

背景技术Background technique

可见光成像下的大气退化图像去云雾技术是计算机图像处理中、高阶段提高图像质量的基础，它旨在从云雾和极低照度等大气退化现象下恢复出图像的原始信息，图像信息的有效恢复对复杂气象条件下的道路安全监控、汽车安全驾驶、光学武器作战效能的提升、敌我战场的有效监视发挥着重大作用^[1]，然而，现有的基于物理模型的可见光成像系统受到大气条件的严重影响，在云雾等大气退化现象下，大气中悬浮的大量微小水滴、气溶胶的衰减和散射作用使得大气能见度、亮度严重下降，致使可见光成像传感器采集到的图像质量降低，极大地影响和限制了复杂气象环境下户外可见光成像系统的功能^[2][3]，既而引起了国内外许多研究者的浓厚兴趣，近年来已经成为图像处理计算机技术领域备受关注的前沿热点。Atmospheric degradation image cloud removal technology under visible light imaging is the basis for improving image quality in the middle and advanced stages of computer image processing. It plays an important role in road safety monitoring under complex meteorological conditions, safe driving of vehicles, improvement of optical weapon combat effectiveness, and effective surveillance of enemy and enemy battlefields ^[1] . However, the existing visible light imaging system based on physical models is affected by atmospheric conditions Serious impact. Under atmospheric degradation phenomena such as clouds and fog, the attenuation and scattering of a large number of tiny water droplets and aerosols suspended in the atmosphere seriously reduce the visibility and brightness of the atmosphere, resulting in a decrease in the quality of images collected by visible light imaging sensors, which greatly affects and limits The function of the outdoor visible light imaging system under complex meteorological environment ^[2][3] has aroused the strong interest of many researchers at home and abroad, and has become a frontier hotspot in the field of image processing computer technology in recent years.

依据算法特点，针对大气退化现象的图像去云雾技术主要形成了两个方向：基于图像增强的方法和基于图像复原的方法，基于图像增强的方法只能相对的提升大气退化现象下图像的质量，并不能实现真正意义上的去云雾，基于图像复原的方法依赖于图像退化模型的建立，通过对大气衰减和环境光照进行建模，并依据强有力的假设和先验信息将模型求解的非适定性转换为适定性，从而实现参数分析和去云雾图像的获得，该研究方向从CVPR2008开始至今是国内外研究的前沿和热点，尤其以CVPR2009何恺明等所提出的暗原色(DarkChannel Prior,DCP)为代表，该方法具备物理有效性，但是当场景目标在很大的区域和大气光本质上很相似时，并且没有阴影投影到物体上时，它的物理模型就会无效，本发明从图像复原角度出发，旨在从降质模型中恢复出原始高质量图像，在对降质过程进行适应性物理建模的同时，引入色彩空间校正机制，使得颜色表现更加丰富，从而改善图像的视觉效果，为了使之具有广泛的适用性，在处理时要使图像亮度、对比度和颜色分别通过模型与图像的内在底层因素相联系，这样就不用人为的设定参数，并且使复原结果更加符合图像的本质特性和人眼视觉感知，总之，基于图像复原的去云雾方法很多，关键是寻找到其病态退化物理模型的一个最优解法，从而实现图像局域特征和全局特征的契合。According to the characteristics of the algorithm, the image declouding technology for atmospheric degradation phenomena mainly forms two directions: the method based on image enhancement and the method based on image restoration. The method based on image enhancement can only relatively improve the quality of the image under the phenomenon of atmospheric degradation. It cannot achieve the true declouding. The method based on image restoration relies on the establishment of an image degradation model. By modeling atmospheric attenuation and ambient light, and based on strong assumptions and prior information, the model is not suitable for solving the problem. Qualitative transformation into well-posedness, so as to realize parameter analysis and obtain cloud-free images. This research direction has been the frontier and hotspot of domestic and foreign research since CVPR2008, especially the Dark Channel Prior (DCP) proposed by CVPR2009 He Yuming et al. It means that this method has physical validity, but when the scene object is similar in nature to the atmospheric light in a large area, and when there is no shadow projected onto the object, its physical model will be invalid. The present invention from the perspective of image restoration Starting from the original high-quality image, it aims to restore the original high-quality image from the degraded model. While performing adaptive physical modeling on the degraded process, a color space correction mechanism is introduced to make the color expression richer, thereby improving the visual effect of the image. To make it widely applicable, the brightness, contrast and color of the image should be connected with the underlying factors of the image through the model respectively during processing, so that there is no need to artificially set parameters, and the restoration result is more in line with the essential characteristics of the image And human visual perception, in short, there are many cloud removal methods based on image restoration, the key is to find an optimal solution to its pathological degradation physical model, so as to achieve the fit of image local features and global features.

何恺明等所提出的暗原色理论(Dark Channel Prior，DCP)认为，清晰的图像上除天空区域外，在RGB颜色通道中至少有一个通道存在很低的强度值，在云雾图像上，暗原色的强度值主要由大气光组成,该方法直接使用暗原色来估计传输图,并使用图像修补的方法对传输图进行了平滑操作,使用修补后的传输图能够恢复出清晰的图像,并从中获得雾天图像的深度图^[5]，具体的实现方案如下：The Dark Channel Prior (DCP) theory proposed by He Yuming et al. believes that on a clear image, except for the sky area, there is at least one channel in the RGB color channel with a very low intensity value. On the cloud image, the dark channel prior The intensity value is mainly composed of atmospheric light. This method directly uses the dark channel to estimate the transmission map, and uses the image inpainting method to smooth the transmission map. Using the patched transmission map can restore a clear image and obtain fog from it. The depth map of the sky image ^[5] , the specific implementation scheme is as follows:

首先假设大气光A是给定的，进一步假定在一个局部区域的大气传输函数恒定不变，对McCarney的大气散射模型使用取最小运算符，并同除以A，得到：First assume that the atmospheric light A is given, and further assume that the atmospheric transfer function in a local area is constant, use the minimum operator for McCarney’s atmospheric scattering model, and divide it by A to get:

$\underset{y the y &Element; &Element; Ω Ω}{min min} ((\frac{{I I}^{C C} ((y the y))}{{A A}^{C C}})) = = \overset{~ ~}{t t} ((x x)) \underset{y the y &Element; &Element; Ω Ω}{min min} ((\frac{{J J}^{C C} ((y the y))}{{A A}^{C C}})) + + ((11 - - \overset{~ ~}{t t} ((x x))))$

三个颜色通道中使用最小运算，有：The minimum operation is used in the three color channels, there are:

$\underset{C C}{min min} ((\underset{y the y &Element; &Element; Ω Ω}{min min} ((\frac{{I I}^{C C} ((y the y))}{{A A}^{C C}})))) = = \overset{~ ~}{t t} ((x x)) \underset{C C}{min min} ((\underset{y the y &Element; &Element; Ω Ω}{min min} ((\frac{{J J}^{C C} ((y the y))}{{A A}^{C C}})))) + + ((11 - - \overset{~ ~}{t t} ((x x))))$

根据暗原色先验的规律，无雾自然图像的暗原色项J^dark应该接近于0：According to the law of the dark channel prior, the dark channel item J ^dark of the fog-free natural image should be close to 0:

${J J}^{dark dark} ((x x)) = = \underset{C C}{min min} ((\underset{y the y &Element; &Element; Ω Ω}{min min} (({J J}^{C C} ((y the y)))))) = = 00$

由于A^C总为正数，导出：Since A ^C is always a positive number, derive:

$\underset{C C}{min min} ((\underset{y the y &Element; &Element; Ω Ω}{min min} ((\frac{{J J}^{C C} ((y the y))}{{A A}^{C C}})))) = = 00$

因此，可简单地估算出透射率t：Therefore, the transmittance t can be estimated simply:

$\overset{~ ~}{t t} ((x x)) = = 11 - - \underset{C C}{min min} ((\underset{y the y &Element; &Element; Ω Ω}{min min} ((\frac{{I I}^{C C} ((y the y))}{{A A}^{C C}}))))$

如果彻底地移除雾的存在，图像会看起来不真实，并且深度感会丢失，所以可以通过在上式中引入一个常数ω(0<ω≤1)，保留一部分覆盖遥远景物的雾：If the existence of fog is completely removed, the image will look unreal and the sense of depth will be lost, so a constant ω (0<ω≤1) can be introduced in the above formula to retain a part of the fog covering the distant scene:

$\overset{~ ~}{t t} ((x x)) = = 11 - - \underset{C C}{w w min min} ((\underset{y the y &Element; &Element; Ω Ω}{min min} ((\frac{{I I}^{C C} ((y the y))}{{A A}^{C C}}))))$

由上式估计出透射率是粗略的，为了提高精度，应用softmatting算法来完善透射率分布函数，记经完善后的透射率函数为t(x)，通过求解下面的式子得到：The transmittance estimated from the above formula is rough. In order to improve the accuracy, the softmatting algorithm is used to improve the transmittance distribution function. The perfected transmittance function is t(x), which is obtained by solving the following formula:

$((L L + + λU λ U)) t t = = λ λ \overset{~ ~}{t t}$

λ是一个修正参数，L是Levin提出的拉普拉斯修正矩阵，由下式计算去雾后的图像J(x)：λ is a correction parameter, L is the Laplace correction matrix proposed by Levin, and the image J(x) after dehazing is calculated by the following formula:

$J J ((x x)) = = \frac{I I ((x x)) - - A A}{max max ((t t ((x x)),, {t t}_{00}))} + + A A$

大气光A的估计方法为：先取暗原色J^dark中0.1%亮度最大的像素，然后取这些像素对应在原图中的最大值作为A的值，分别对存在轻度和浓重云雾现象的两幅图像进行处理。The method of estimating atmospheric light A is: first take the pixels with the highest brightness of 0.1% in the dark primary color J ^dark , and then take the maximum value of these pixels corresponding to the original image as the value of A, respectively for the two images with mild and heavy clouds and fog to process.

该方案建立在暗原色先验假设之上，但在现实场景中当目标在很大的区域和大气光本质上很相似或云雾比较深时，就很难满足该假设，会使的得到的暗通道图像边缘粗糙、层次不分明；子快大小影响去除云雾质量；大气光照不能反映真实景象，并且不具备局部特性；导致估计的大气传输函数局部失真严重；恢复结果出现色彩失真，影响了图像的视觉效果；整体复原结果较暗，颜色不够鲜明。This scheme is based on the prior assumption of the dark primary color, but in real scenes, when the target is similar in nature to the atmospheric light in a large area or the clouds and fog are relatively deep, it is difficult to satisfy the assumption, which will make the obtained dark The channel image has rough edges and unclear layers; the size of the sub-block affects the quality of cloud removal; atmospheric illumination cannot reflect the real scene, and does not have local characteristics; resulting in serious local distortion of the estimated atmospheric transfer function; color distortion in the restoration result affects the image quality. Visual effects; the overall restoration result is dark and the colors are not bright enough.

发明内容Contents of the invention

本发明实施例的目的在于提供一种去除大气退化图像云雾的方法，旨在解决现有技术方案存在的当目标在很大的区域和大气光本质上很相似或云雾比较深时，就很难满足该假设，会使的得到的暗通道图像边缘粗糙、层次不分明；子快大小影响去除云雾质量；大气光照不能反映真实景象，并且不具备局部特性；导致估计的大气传输函数局部失真严重；恢复结果出现色彩失真，影响了图像的视觉效果；整体复原结果较暗，颜色不够鲜明的问题。The purpose of the embodiments of the present invention is to provide a method for removing clouds and fog in images of atmospheric degradation, aiming at solving the problems existing in existing technical solutions when the target is similar in nature to the atmospheric light in a large area or the clouds and fog are relatively deep. Satisfying this assumption will make the dark channel image obtained have rough edges and unclear layers; the size of the sub-fast affects the quality of cloud removal; atmospheric illumination cannot reflect the real scene, and does not have local characteristics; resulting in serious local distortion of the estimated atmospheric transfer function; The restoration result has color distortion, which affects the visual effect of the image; the overall restoration result is dark and the color is not bright enough.

本发明实施例是这样实现的，一种去除大气退化图像云雾的方法，所述去除大气退化图像云雾的方法包括以下步骤：The embodiment of the present invention is achieved in this way, a method for removing clouds and fog in images of atmospheric degradation, the method for removing clouds and fog in images of atmospheric degradation includes the following steps:

暗通道图像的获取及中值滤波、大气光图像的自适应分解获取;Acquisition of dark channel images, median filtering, and adaptive decomposition of atmospheric light images;

大气传输函数的细化;Refinement of the atmospheric transfer function;

色彩域的视觉校正。Visual correction of color gamut.

进一步，暗通道图像生成采用图像自适应分块处理思想，在暗通道先验理论基础上获取原始图像所对应的暗通道图像I_D(x,y)，并使用中值滤波平滑处理，可用如下公式定义生成的暗通道图像I_D(x,y)：Further, the dark channel image generation adopts the image adaptive block processing idea, and obtains the dark channel image _ID (x, y) corresponding to the original image on the basis of the dark channel prior theory, and uses the median filter smoothing process, which can be used as follows The formula defines the generated dark channel image _ID (x,y):

${I I}_{D D.} ((x x,, y the y)) = = \underset{c c &Element; &Element; {{r r,, g g,, b b}}}{min min} ((\underset{Ω Ω}{min min} (({I I}_{c c} ((x x,, y the y))))))$

其中，I_c(x,y)为原始图像的指定通道；Ω为自适应的图像子块，是以(x,y)为中心的一块正方形区域，为了防止深度跳变所产生的粗糙现象，避免在结果中出现棋盘格效应和光晕痕迹，需对暗通道图像进行中值滤波；Among them, I _c (x, y) is the specified channel of the original image; Ω is the adaptive image sub-block, which is a square area centered on (x, y). In order to prevent the rough phenomenon caused by the depth jump, To avoid checkerboard effects and halo traces in the results, it is necessary to perform median filtering on the dark channel image;

进一步，大气光图像获取采用自适应二维经验模式分解，在进行5次分解之后，获取原始图像所对应的大气光图像A(x,y)，并通过低通滤波来均匀局域大气光。Furthermore, the atmospheric light image is obtained by adaptive two-dimensional empirical mode decomposition. After five times of decomposition, the atmospheric light image A(x, y) corresponding to the original image is obtained, and the local atmospheric light is uniformed by low-pass filtering.

进一步，自适应二维经验模式分解，具体实施步骤如下：Further, the adaptive two-dimensional empirical mode decomposition, the specific implementation steps are as follows:

步骤一，通过数学形态学中的测地算子识别图像中的区域极值点；Step 1, identify the regional extreme points in the image through the geodesic operator in mathematical morphology;

步骤二，采用优化了的径向基函数插值法构造包络曲面，对极大值点和极小值点分别进行插值运算，运算后得到极大值点包络曲面和极小值点包络曲面，将两曲面数据求平均得到均值包络曲面数据，因此，图像空间中的上下包络求解问题就转化成三维曲面的离散数据点插值重建问题，求解包络平面可表示为:Step 2: Use the optimized radial basis function interpolation method to construct the envelope surface, perform interpolation operations on the maximum point and the minimum point respectively, and obtain the maximum point envelope surface and the minimum point envelope after the operation surface, average the two surface data to obtain the mean envelope surface data, therefore, the problem of solving the upper and lower envelopes in the image space is transformed into the problem of interpolation and reconstruction of discrete data points on the three-dimensional surface, and the solution to the envelope plane can be expressed as:

$s the s ((x x)) = = {c c}_{00} + + {c c}_{11} x x + + {c c}_{22} y the y + + {Σ Σ}_{i i = = 11}^{N N} {λ λ}_{i i} φ φ ((| | | | x x - - {x x}_{i i} | | | |))$

其中，s(x)为插值节点，c₀、c₁、c₂、λ_i为多项式系数和径向基函数组合系数，‖·‖为欧几里德范数，φ(·)为径向基函数，通过求解N+3元一次的大型线性方程组，解得c₀、c₁、c₂、λ_i的值，进而代入所有点的坐标值得到整个插值曲面，对φ(·)的选取有thin-plate,multiquadratics等方法，改进φ(·)的选取为：Among them, s(x) is an interpolation node, c ₀ , c ₁ , c ₂ , and λ _i are polynomial coefficients and radial basis function combination coefficients, ‖·‖ is Euclidean norm, φ(·) is radial The basis function, by solving a large-scale linear equation system of N+3 elements, the values of c ₀ , c ₁ , c ₂ , and λ _i are obtained, and then the coordinate values of all points are substituted to obtain the entire interpolation surface. For φ(·) There are thin-plate, multiquadratics and other methods to choose, and the selection of improved φ( ) is:

$φ φ ((R R)) = = \{\begin{matrix} {((11 - - R R))}^{P P} PR PR,, & R R < < 11 \\ 00,, & R R > > 11 \end{matrix}$

其中，R=‖x-x_i‖为欧式距离，P是一个常数系数；Among them, R=‖xx _i ‖ is the Euclidean distance, and P is a constant coefficient;

步骤三，用原曲面减去均值包络曲面；Step 3, subtract the mean envelope surface from the original surface;

步骤四，判断是否满足终止条件，因为二维内蕴模函数过零点的数目是无法统计的，所以二维经验模式分解时可以用二维内蕴模函数的约束条件作为筛选过程的停止条件，也可以用高效的Cauchy-type收敛条件：Step 4, judging whether the termination condition is satisfied, because the number of zero-crossing points of the two-dimensional intrinsic modulus function cannot be counted, so the constraint condition of the two-dimensional intrinsic modulus function can be used as the stopping condition of the screening process when decomposing the two-dimensional empirical model. An efficient Cauchy-type convergence condition can also be used:

$SD SD = = {Σ Σ}_{x x}^{M m} {Σ Σ}_{y the y}^{N N} \frac{{| | {f f}_{k k - - 11} ((x x,, y the y)) - - {f f}_{k k} ((x x,, y the y)) | |}^{22}}{{f f}_{k k - - 11}^{22} ((x x,, y the y))}$

其中，f_k(x,y)是第k层自适应二维经验模式分解时图像上(x,y)点的像素值，并可使SD为0.2到0.3之间的数；Among them, f _k (x, y) is the pixel value of point (x, y) on the image when the k-th layer adaptive two-dimensional empirical mode is decomposed, and SD can be a number between 0.2 and 0.3;

重复步骤(一)～(三)，直到满足给定的终止条件，得到第1层二维内蕴模函数图像bimf₁(x,y)，用原图像减去bimf₁(x,y)得到第1层残差图像residue1，对residue1重复步骤(一)～(四)，依次得到图像的N层二维内蕴模函数图像和第N层残差图像，在上述过程中，极值点求解、平面插值和筛选的停止条件是本算法的核心，一般是进行5次分解，就可以得到纯净的大气光图像，所以最终的结果表示为：Repeat steps (1) to (3) until the given termination conditions are satisfied, and the first layer two-dimensional intrinsic modulus image bimf ₁ (x, y) is obtained, and the original image is subtracted from bimf ₁ (x, y) to obtain The first layer residual image residue1, repeat steps (1) to (4) for residue1, and sequentially obtain the N-layer two-dimensional intrinsic modulus image of the image and the N-th layer residual image. In the above process, the extreme points are solved , Plane interpolation and filtering stop conditions are the core of this algorithm. Generally, a pure atmospheric light image can be obtained by performing five decompositions, so the final result is expressed as:

${r r}_{55} ((x x,, y the y)) = = f f ((x x,, y the y)) - - {Σ Σ}_{k k = = 11}^{55} {bimf bimf}_{k k} ((x x,, y the y))$

其中，bimf_k(x,y)是第k层二维内蕴模函数图像，r₅(x,y)是经过5层分解后的趋势图像，实际中的一些大气退化图像，其大气光照很难满足图像的局域平滑性，为了克服这一缺点，提高算法的鲁棒性，可以采用低通滤波平滑处理大气光照图像中的低频照度信息。Among them, bimf _k (x, y) is the two-dimensional intrinsic modulus image of the k-th layer, and r ₅ (x, y) is the trend image after five layers of decomposition. Some atmospheric degradation images in practice have very low atmospheric illumination. It is difficult to satisfy the local smoothness of the image. In order to overcome this shortcoming and improve the robustness of the algorithm, low-pass filtering can be used to smooth the low-frequency illuminance information in the atmospheric illumination image.

进一步，大气传输函数细化采用最暗通道处理思想，通过下面的公式：Further, the refinement of the atmospheric transfer function adopts the darkest channel processing idea, through the following formula:

${t t}_{11} ((x x,, y the y)) = = 11 - - w w \underset{Ω Ω}{min min} ((\frac{{I I}_{D D.} ((x x,, y the y))}{A A ((x x,, y the y))}))$

获取粗略的大气传输函数t₁，其中Ω为自适应的图像子块，w是为了保持复原后图像的真实度引入的参数，通常在0到1之间取值，实验中取0.95；Obtain a rough atmospheric transfer function t ₁ , where Ω is an adaptive image sub-block, and w is a parameter introduced to maintain the authenticity of the restored image, usually taking a value between 0 and 1, and taking 0.95 in the experiment;

采用softmatting算法，对粗略的大气传输函数t₁进行拉普拉斯矩阵修正，通过最优化下面的能量方程来细化t₁：Using the softmatting algorithm, the rough atmospheric transfer function t ₁ is modified by the Laplace matrix, and t ₁ is refined by optimizing the following energy equation:

E(t)=t^TLt+λ(t-t₁)^TU(t-t₁)E(t)=t ^T Lt+λ(tt ₁ ) ^T U(tt ₁ )

获取细化的大气传输函数t，其中λ为归一化参数，U为与图像同等大小的单位矩阵，L为拉普拉斯修正矩阵，对上述能量方程进行稀疏线性表示：Obtain the refined atmospheric transfer function t, where λ is the normalization parameter, U is the unit matrix with the same size as the image, and L is the Laplace correction matrix, and perform a sparse linear representation of the above energy equation:

(L+λU)t=λt₁ (L+λU)t=λt ₁

其中，λ为一个很小的数值，实验中设定为10^-4；拉普拉斯修正矩阵L某一像素点(i,j)可表示为：Among them, λ is a very small value, which is set to 10 ^-4 in the experiment; a certain pixel point (i, j) of the Laplace correction matrix L can be expressed as:

$\underset{k k | | ((i i,, j j)) &Element; &Element; {w w}_{k k}}{Σ Σ} {{{δ δ}_{ij ij} - - \frac{11}{| | {w w}_{k k} | |} [[11 + + {(({I I}_{i i} - - {μ μ}_{k k}))}^{T T} {(({Σ Σ}_{k k} + + \frac{ϵ ϵ}{| | {w w}_{k k} | |} {U u}_{33}))}^{- - 11} (({I I}_{j j} - - {μ μ}_{k k}))]]}}$

其中，δ_ij是克罗内克函数；μ_k和Σ_k是窗口w_k中像素点的均值和协方差矩阵；|w_k|是窗口w_k中像素点的数量；ε是归一化参数；U₃是3乘3的单位矩阵；在进行运算时，已经将图像矩阵按列向量进行展开，转换为一维的向量，I_i和I_j表示一维向量中下标为i和j像素点的值；Among them, δ _ij is the Kronecker function; μ _k and Σ _k are the mean and covariance matrix of pixels in window w _k ; |w _k | is the number of pixels in window w _k ; ε is the normalization parameter ; U ₃ is a unit matrix of 3 by 3; when performing operations, the image matrix has been expanded by column vectors and converted into one-dimensional vectors, and I _i and I _j represent the subscript i and j pixels in the one-dimensional vector point value;

进一步，去云雾图像色彩域校正具体操作为，对图像的R，G，B三原色通道分别进行去云雾运算：Further, the specific operation of the color gamut correction of the cloud-removing image is to perform the cloud-removing operation on the R, G, and B three primary color channels of the image respectively:

${J J}_{c c} ((x x,, y the y)) = = \frac{{I I}_{c c} ((x x,, y the y)) - - A A ((x x,, y the y))}{max max ((t t ((x x,, y the y)),, {t t}_{00}))} + + A A ((x x,, y the y)),, c c &Element; &Element; {{r r,, g g,, b b}}$

其中，I_c(x,y)为原始图像的指定通道，t₀为设定的大气耗散函数下界值，通常取0.1，RGB三分量之间有很强的相关性，基于ImagineMacmillan库统计了四组视觉效果良好的图像，求出了RGB三个分量相关系数矩阵，在其图像融合算法中作为目标值，任意两个分量X和Y的相关系数定义为：Among them, I _c (x, y) is the specified channel of the original image, t ₀ is the lower bound value of the set atmospheric dissipation function, usually 0.1, and there is a strong correlation between the three components of RGB, based on the statistics of the ImagineMacmillan library Four groups of images with good visual effects, the three-component correlation coefficient matrix of RGB is obtained, which is used as the target value in the image fusion algorithm, and the correlation coefficient of any two components X and Y is defined as:

$r r = = \frac{{Σ Σ}_{m m} {Σ Σ}_{n no} (({X x}_{mn mn} - - \overset{&OverBar; &OverBar;}{X x})) (({Y Y}_{mn mn} - - \overset{&OverBar; &OverBar;}{Y Y}))}{\sqrt{{Σ Σ}_{m m} {Σ Σ}_{n no} {(({X x}_{mn mn} - - \overset{&OverBar; &OverBar;}{X x}))}^{22} {Σ Σ}_{m m} {Σ Σ}_{n no} {(({Y Y}_{mn mn} - - \overset{&OverBar; &OverBar;}{Y Y}))}^{22}}}$

其中：

为X，Y均值，显然-1≤r≤1，因此颜色校正存在一种思路：寻求一个线性变换，将图像的RGB分量进行变换。in:

is the mean value of X and Y, obviously -1≤r≤1, so there is an idea for color correction: seek a linear transformation to transform the RGB components of the image.

进一步，对去除云雾的图像J(x,y)中任一像素点m_xy=[m_R,m_G,m_B]^T，进行如下的RGB色彩域校正，Further, for any pixel point m _xy =[m _R ,m _G ,m _B ] ^T in the cloud-removed image J(x,y), perform the following RGB color gamut correction,

$\{\begin{matrix} {n no}_{xy xy} = = {B B}^{T T} {m m}_{xy xy} \\ {C C}_{n no} = = {B B}^{T T} {C C}_{m m} B B \end{matrix}$

其中，n_xy为校正后像素点向量，B为色彩域校正矩阵，C_m、C_n分别为校正前后图像色彩通道间协方差矩阵，为了求解色彩校正矩阵B，可先对正定矩阵C_m、C_n进行cholesky分解，得到C_m=Q_m ^TQ_m，C_n=Q_n ^TQ_n，易求得B=Q_m ^-1Q_n，设分量间相关系数构成的相关系数矩阵为R_n，易知R_n与C_n存在以下关系：Among them, n _xy is the pixel vector after correction, B is the color gamut correction matrix, C _m , C _n are the covariance matrices between image color channels before and after correction, respectively, in order to solve the color correction matrix B, the positive definite matrices C _m , Cholesky decomposes C _n to get C _m = Q _m ^T Q _m , C _n = Q _n ^T Q _n , easy to obtain B = Q _m ^-1 Q _n , let the correlation coefficient matrix formed by the correlation coefficient between components be R _n , it is easy to know that there is the following relationship between R _n and C _n :

${C C}_{n no} = = [\begin{matrix} {σ σ}_{n no 11} & 00 & 00 \\ 00 & {σ σ}_{n no 22} & 00 \\ 00 & 00 & {σ σ}_{n no 33} \end{matrix}] {R R}_{n no} {[\begin{matrix} {σ σ}_{n no 11} & 00 & 00 \\ 00 & {σ σ}_{n no 22} & 00 \\ 00 & 00 & {σ σ}_{n no 33} \end{matrix}]}^{T T}$

对于待校正图像的C_m可通过

求得，其中

或者也可以根据上式中关系求得；The C _m of the image to be corrected can be obtained by

obtain, among them

Or it can also be obtained according to the relationship in the above formula;

进一步，去云雾图像色彩域校正采用均值加减方差作为另外两个分量间相关系数矩阵与R_n分别求出校正后的三幅图像，之后求平均得到最后的校正图像。Further, the color gamut correction of the cloud-removing image uses the mean value plus and minus the variance as the correlation coefficient matrix and R _n between the other two components to obtain the three corrected images, and then calculate the average to obtain the final corrected image.

本发明提供的去除大气退化图像云雾的方法，通过采用暗通道理论模型，并且融入暗通道图像的中值滤波、大气光图像的自适应分解获取和色彩域的视觉校正来进行去云雾，经中值滤波处理后的暗通道图像边界跳变细腻、区域内部平滑，经自适应分解获取的大气光图像整体变化平滑、局部有其各自光照特征，经过softmatting细化处理的大气传输函数细节明显、层次感好，最后经RGB色彩域校正的图像亮度适中、色彩饱和、细节清晰、信息丰富适合人眼评定。此外，本发明处理流程较为合理，处理的各个环节不可或缺，充分说明本发明在图像动态范围压缩、细节凸显以及恢复色彩信息上的能力。The method for removing clouds and fog in atmospheric degradation images provided by the present invention uses the dark channel theoretical model and integrates the median filtering of dark channel images, the adaptive decomposition and acquisition of atmospheric light images and the visual correction of color gamut to remove clouds and fog. After value filtering, the dark channel image has fine boundary transitions and smooth interior areas. The atmospheric light image obtained by adaptive decomposition has smooth overall changes and local lighting characteristics. The atmospheric transfer function processed by softmatting has obvious details and layers. The sensory perception is good, and finally the RGB color gamut-corrected image has moderate brightness, saturated color, clear details, and rich information, which is suitable for human eye evaluation. In addition, the processing flow of the present invention is relatively reasonable, and each link of the processing is indispensable, which fully demonstrates the capabilities of the present invention in image dynamic range compression, detail highlighting, and color information recovery.

附图说明Description of drawings

图1是本发明实施例提供的去除大气退化图像云雾的方法的流程图；Fig. 1 is a flowchart of a method for removing clouds and fog in an image of atmospheric degradation provided by an embodiment of the present invention;

图2是本发明实施例提供的ABEMD的一层分解流程示意图。Fig. 2 is a schematic diagram of a one-layer decomposition flow chart of ABEMD provided by the embodiment of the present invention.

具体实施方式Detailed ways

为了使本发明的目的、技术方案及优点更加清楚明白，以下结合实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

图1示出了本发明提供的去除大气退化图像云雾的方法流程。为了便于说明，仅仅示出了与本发明相关的部分。FIG. 1 shows the flow of the method for removing clouds and fog in an image of atmospheric degradation provided by the present invention. For ease of illustration, only the parts relevant to the present invention are shown.

本发明的去除大气退化图像云雾的方法，该去除大气退化图像云雾的方法包括以下步骤：The method for removing the cloud and fog of the atmospheric degradation image of the present invention, the method for removing the cloud and fog of the atmospheric degradation image comprises the following steps:

大气传输函数的细化;Refinement of the atmospheric transfer function;

色彩域的视觉校正。Visual correction of color gamut.

作为本发明实施例的一优化方案，暗通道图像生成采用图像自适应分块处理思想，在暗通道先验理论基础上获取原始图像所对应的暗通道图像I_D(x,y)，并使用中值滤波平滑处理，可用如下公式定义生成的暗通道图像I_D(x,y)：As an optimization scheme of the embodiment of the present invention, the dark channel image generation adopts the idea of image adaptive block processing, obtains the dark channel image _ID (x, y) corresponding to the original image on the basis of the dark channel prior theory, and uses For median filter smoothing, the generated dark channel image _ID (x,y) can be defined by the following formula:

作为本发明实施例的一优化方案，大气光图像获取采用自适应二维经验模式分解，在进行5次分解之后，获取原始图像所对应的大气光图像A(x,y)，并通过低通滤波来均匀局域大气光。As an optimization scheme of the embodiment of the present invention, the atmospheric light image acquisition adopts self-adaptive two-dimensional empirical mode decomposition. After five decompositions, the atmospheric light image A(x, y) corresponding to the original image is obtained, and the low-pass Filter to even out local atmospheric light.

作为本发明实施例的一优化方案，自适应二维经验模式分解，具体实施步骤如下：As an optimization scheme of the embodiment of the present invention, the adaptive two-dimensional empirical mode decomposition, the specific implementation steps are as follows:

作为本发明实施例的一优化方案，

As an optimization scheme of the embodiment of the present invention,

E(t)=t^TLt+λ(t-t₁)^TU(t-t₁)E(t)=t ^T Lt+λ(tt ₁ ) ^T U(tt ₁ )

(L+λU)t=λt₁ (L+λU)t=λt ₁

作为本发明实施例的一优化方案，去云雾图像色彩域校正具体操作为，对图像的R，G，B三原色通道分别进行去云雾运算：As an optimization scheme of the embodiment of the present invention, the specific operation of the color gamut correction of the cloud-removing image is to perform the cloud-removing operation on the R, G, and B three primary color channels of the image respectively:

其中：

作为本发明实施例的一优化方案，对去除云雾的图像J(x,y)中任一像素点m_xy=[m_R,m_G,m_B]^T，进行如下的RGB色彩域校正，As an optimization scheme of the embodiment of the present invention, for any pixel point m _xy =[m _R , m _G , m _B ] ^T in the cloud-removed image J(x, y), the following RGB color gamut correction is performed,

对于待校正图像的C_m可通过

求得，其中

obtain, among them

Or it can also be obtained according to the relationship in the above formula;

作为本发明实施例的一优化方案，去云雾图像色彩域校正采用均值加减方差作为另外两个分量间相关系数矩阵与R_n分别求出校正后的三幅图像，之后求平均得到最后的校正图像。As an optimization scheme of the embodiment of the present invention, the color gamut correction of the cloud and fog image uses the mean value plus and minus the variance as the correlation coefficient matrix and R _n between the other two components to obtain the three corrected images respectively, and then calculate the average to obtain the final correction image.

下面结合附图及具体实施例对本发明的应用原理作进一步描述。The application principle of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

如图1所示，本发明实施例的去除大气退化图像云雾的方法包括以下步骤：As shown in Figure 1, the method for removing clouds and fog in an image of atmospheric degradation in an embodiment of the present invention includes the following steps:

S101：暗通道图像的获取及中值滤波、大气光图像的自适应分解获取;S101: Acquisition of dark channel images, median filtering, and adaptive decomposition acquisition of atmospheric light images;

S102：大气传输函数的细化;S102: refinement of the atmospheric transfer function;

S103:色彩域的视觉校正。S103: Visual correction of color gamut.

本发明实施例的具体步骤为：The concrete steps of the embodiment of the present invention are:

第一步，暗通道图像生成：采用图像自适应分块处理思想，在暗通道先验理论基础上获取原始图像所对应的暗通道图像I_D(x,y)，并使用中值滤波平滑处理，The first step, dark channel image generation: adopt the idea of image adaptive block processing, obtain the dark channel image _ID (x, y) corresponding to the original image on the basis of the dark channel prior theory, and use the median filter smoothing process ,

本发明中所用到的物理模型是McCarney的大气散射模型，该模型被广泛的应用于图像处理计算机领域，根据大气光在大气退化现象下传输的物理特性，可采用如下公式描述：The physical model used in the present invention is the atmospheric scattering model of McCarney, and this model is widely used in image processing computer field, according to the physical characteristic that atmospheric light transmits under atmospheric degeneration phenomenon, can adopt following formula to describe:

I(x,y)=t(x,y)J(x,y)+(1-t(x,y))AI(x,y)=t(x,y)J(x,y)+(1-t(x,y))A

其中，I(x,y)为产生的大气退化图像，t(x,y)为大气传输函数，A为大气光强度，J(x,y)代表图像的本来面貌，上述公式表明了大气退化现象的成因，包含了图像对比度和色彩的变化，本发明的目的就是利用上述公式，以及已知的参数或假设来求解得到J(x,y)，但该式求解的未知数个数大于列出的方程数，因此，本发明以何恺明等提出的暗原色先验作为求解的基础约束条件，从而有效的求出本发明模型的解；Among them, I(x,y) is the generated atmospheric degradation image, t(x,y) is the atmospheric transfer function, A is the atmospheric light intensity, J(x,y) represents the original appearance of the image, the above formula shows that the atmospheric degradation The cause of formation of phenomenon includes the change of image contrast and color, and the purpose of the present invention is exactly to utilize above-mentioned formula, and known parameter or assumption to solve and obtain J (x, y), but the number of unknowns that this formula solves is greater than listed The number of equations, therefore, the present invention uses the dark channel prior proposed by He Yuming etc. as the basic constraint condition for solving, thereby effectively finding out the solution of the model of the present invention;

暗原色先验是在户外大气退化图像统计学基础上提出的，是一种局部的暗物体概念（Dark Object Subtraction,DOS），该理论认为，在绝大多数的非天空局部区域里，某一像素总会有至少一个颜色通道具有极低的数值，即产生一幅区域光强度极小值很小的图像，可用如下公式定义生成的暗通道图像I_D(x,y)：The dark channel prior is proposed on the basis of the statistics of outdoor atmospheric degradation images. It is a local dark object concept (Dark Object Subtraction, DOS). A pixel will always have at least one color channel with a very low value, that is, an image with a small local light intensity minimum value can be defined by the following formula to define the generated dark channel image _ID (x,y):

其中，I_c(x,y)为原始图像的指定通道；Ω为自适应的图像子块，是以(x,y)为中心的一块正方形区域，自适应分块是本发明中重要的一步，直接影响着后期处理质量，子快的取值较小时，大气传输函数的细节较多，层次感鲜明，但是平滑度不足，会导致局部对比度失真；反之，能够有效减少局部对比度失真，但得到的大气传输函数图像过于单一，致使图像的细节和层次感不够明显，不能有效区分近景和远景图像，针对这种情况，为了达到在失真率和细节之间的平衡，选取图像行和列的4%的最大值作为子块的大小，这样也就避免了统一子块对大小不同图像在去云雾时的影像，此外，为了防止深度跳变所产生的粗糙现象，避免在结果中出现棋盘格效应和光晕痕迹，需对暗通道图像进行中值滤波；Among them, I _c (x, y) is the specified channel of the original image; Ω is an adaptive image sub-block, which is a square area centered on (x, y), and adaptive block is an important step in the present invention , which directly affects the quality of post-processing. When the value of the sub-fast is small, the details of the atmospheric transfer function are more and the layering is clear, but the smoothness is insufficient, which will lead to local contrast distortion; on the contrary, it can effectively reduce the local contrast distortion, but get The image of the atmospheric transfer function is too single, resulting in the details and layering of the image are not obvious enough, and it is impossible to effectively distinguish the close-range and distant images. In this case, in order to achieve a balance between the distortion rate and the details, select 4 of the image row and column The maximum value of % is used as the size of the sub-block, which avoids the image of the uniform sub-block for images of different sizes when removing clouds and fog. In addition, in order to prevent the rough phenomenon caused by the depth jump, avoid the checkerboard effect in the result and halo traces, it is necessary to perform median filtering on the dark channel image;

第二步，大气光图像获取：采用自适应二维经验模式分解，在进行5次分解之后，获取原始图像所对应的大气光图像A(x,y)，并通过低通滤波来均匀局域大气光，The second step, atmospheric light image acquisition: adopt adaptive two-dimensional empirical mode decomposition, after five times of decomposition, obtain the atmospheric light image A(x, y) corresponding to the original image, and use low-pass filtering to uniform local area atmospheric light,

二维经验模式分解（Bidimensional Empirical Mode Decomposition,BEMD）是在一维经验模式分解基础上提出的一个多尺度结构的新方法，在时域上将一个二维的图像信号分解成不同尺度的二维内蕴模函数(Bidimensional Intrinsic Mode Functions,BIMF)和剩余趋势图像(residue)，它们是包含不同频率的图层，是一种在时域范围内的频率分析方法，经过本发明自适应BEMD的图像，分解得到多个不同尺度的BIMF图像和趋势图像，它们包含着图像的不同频率特性，其中：BIMF图像包含着图像中的高频部分，体现着图像中的细节信息，当其包含的尺度达到5个时，它所含有的细节信息达到99.99%；剩余的趋势图像则对应着图像的大气光照，可将得到的趋势图像作为大气光图像的估计值；Bidimensional Empirical Mode Decomposition (BEMD) is a new method of multi-scale structure proposed on the basis of one-dimensional empirical mode decomposition, which decomposes a two-dimensional image signal into two-dimensional images of different scales in the time domain. Bidimensional Intrinsic Mode Functions (Bidimensional Intrinsic Mode Functions, BIMF) and residual trend image (residue), they are layers that include different frequencies, are a kind of frequency analysis method in the time domain range, through the image of self-adaptive BEMD of the present invention , decompose to obtain multiple BIMF images and trend images of different scales, which contain different frequency characteristics of the image, among which: BIMF images contain high-frequency parts in the image, reflecting the detailed information in the image, when the scale it contains reaches When there are 5, the detail information it contains reaches 99.99%; the remaining trend images correspond to the atmospheric light of the image, and the obtained trend image can be used as the estimated value of the atmospheric light image;

用这种方法进行大气光图像估计时，因为自适应BEMD根据图像特自身征选配尺度，使之对细节的提取趋于最大，就可以完全的去除掉图像中的细节信息，即高频分量；而图像噪声也表现为高频分量，所以得到的照度分量能十分纯净的反映出大气光信息，在后续的处理中就不会因为大气光图像的误差，导致图像中的天空、偏白色物体、水面等大面积明亮区域，在处理时产生色彩失真现象，本发明中提出的自适应算法框架如图2所示，具体实施步骤如下：When using this method to estimate the atmospheric light image, because the adaptive BEMD selects the scale according to the image characteristics, so that the extraction of details tends to be maximized, the detailed information in the image, that is, the high-frequency component, can be completely removed. ; and the image noise also manifests as high-frequency components, so the obtained illuminance components can reflect the atmospheric light information very purely, and in the subsequent processing, the sky and white objects in the image will not be caused by the error of the atmospheric light image. , water surface and other large-area bright areas, color distortion occurs during processing, the adaptive algorithm framework proposed in the present invention is shown in Figure 2, and the specific implementation steps are as follows:

步骤二，采用优化了的径向基函数插值法构造包络曲面，对极大值点和极小值点分别进行插值运算，运算后得到极大值点包络曲面和极小值点包络曲面，将两曲面数据求平均得到均值包络曲面数据，因此，图像空间中的上下包络求解问题就转化成三维曲面的离散数据点插值重建问题，求解包络平面可表示为:Step 2: Use the optimized radial basis function interpolation method to construct the envelope surface, perform interpolation operations on the maximum point and minimum point point respectively, and obtain the maximum value point envelope surface and the minimum value point envelope after the operation surface, average the two surface data to obtain the mean envelope surface data, therefore, the problem of solving the upper and lower envelopes in the image space is transformed into the problem of interpolation and reconstruction of discrete data points on the three-dimensional surface, and the solution to the envelope plane can be expressed as:

其中，s(x)为插值节点，c₀、c₁、c₂、λ_i为多项式系数和径向基函数组合系数，‖·‖为欧几里德范数，φ(·)为径向基函数，通过求解N+3元一次的大型线性方程组，解得c₀、c₁、c₂、λ_i的值，进而代入所有点的坐标值得到整个插值曲面，对φ(·)的选取有thin-plate,multiquadratics等方法，改进φ(·)的选取为：Among them, s(x) is an interpolation node, c ₀ , c ₁ , c ₂ , and λ _i are polynomial coefficients and radial basis function combination coefficients, ‖·‖ is Euclidean norm, φ(·) is radial The basis function, by solving a large-scale linear equation system of N+3 elements, the values of c ₀ , c ₁ , c ₂ , and λ _i are obtained, and then the coordinate values of all points are substituted to obtain the entire interpolation surface. For φ(·) There are thin-plate, multiquadratics and other methods to choose, and the selection of improved φ(·) is:

步骤四，判断是否满足终止条件，因为BIMF过零点的数目是无法统计的，所以二维经验模式分解时可以用BIMF的约束条件作为筛选过程的停止条件，也可以用高效的Cauchy-type收敛条件：Step 4: Determine whether the termination condition is satisfied, because the number of BIMF zero-crossing points cannot be counted, so the constraint condition of BIMF can be used as the stopping condition of the screening process when decomposing the two-dimensional empirical model, or the efficient Cauchy-type convergence condition can be used :

其中，f_k(x,y)是第k层自适应BEMD分解时图像上(x,y)点的像素值，并可使SD为0.2到0.3之间的数；Among them, f _k (x, y) is the pixel value of point (x, y) on the image when the k-th layer adaptive BEMD is decomposed, and SD can be set to a number between 0.2 and 0.3;

重复步骤(一)～(三)，直到满足给定的终止条件，得到第1层BIMF图像bimf₁(x,y)，用原图像减去bimf₁(x,y)得到第1层残差图像residue1，对residue1重复步骤(一)～(四)，依次得到图像的N层BIMF图像和第N层残差图像，在上述过程中，极值点求解、平面插值和筛选的停止条件是本算法的核心，一般是进行5次分解，就可以得到纯净的大气光图像，所以最终的结果表示为：Repeat steps (1) to (3) until the given termination conditions are met, and the first layer BIMF image bimf ₁ (x, y) is obtained, and the first layer residual is obtained by subtracting bimf ₁ (x, y) from the original image For image residue1, repeat steps (1) to (4) for residue1 to obtain the N-layer BIMF image and the N-th layer residual image in sequence. The core of the algorithm is generally to perform 5 decompositions to obtain a pure atmospheric light image, so the final result is expressed as:

其中，bimf_k(x,y)是第k层BIMF图像，r₅(x,y)是经过5层分解后的趋势图像(大气光图像)，实际中的一些大气退化图像，其大气光照很难满足图像的局域平滑性，为了克服这一缺点，提高算法的鲁棒性，可以采用低通滤波平滑处理大气光照图像中的低频照度信息；Among them, bimf _k (x, y) is the k-th layer BIMF image, r ₅ (x, y) is the trend image (atmospheric light image) after 5 layers of decomposition, some atmospheric degradation images in practice, the atmospheric illumination is very It is difficult to satisfy the local smoothness of the image. In order to overcome this shortcoming and improve the robustness of the algorithm, low-pass filtering can be used to smooth the low-frequency illuminance information in the atmospheric illumination image;

第三步，大气传输函数细化：采用最暗通道处理思想，通过下面的公式：The third step is the refinement of the atmospheric transfer function: using the darkest channel processing idea, through the following formula:

采用Levin提出的softmatting算法，对粗略的大气传输函数t₁进行拉普拉斯矩阵修正，通过最优化下面的能量方程来细化t₁：Using the softmatting algorithm proposed by Levin, the rough atmospheric transfer function t ₁ is modified by the Laplace matrix, and t ₁ is refined by optimizing the following energy equation:

E(t)=t^TLt+λ(t-t₁)^TU(t-t₁)E(t)=t ^T Lt+λ(tt ₁ ) ^T U(tt ₁ )

获取细化的大气传输函数t，其中λ为归一化参数，U为与图像同等大小的单位矩阵，L为拉普拉斯修正矩阵，对上述能量方程进行稀疏线性表示：Obtain the refined atmospheric transfer function t, where λ is the normalization parameter, U is the identity matrix with the same size as the image, and L is the Laplace correction matrix, and perform a sparse linear representation of the above energy equation:

(L+λU)t=λt₁ (L+λU)t=λt ₁

其中，λ为一个很小的数值，实验中设定为10^-4；拉普拉斯修正矩阵L某一像素点(i,j)可表示为^[12]：Among them, λ is a very small value, which is set to 10 ^-4 in the experiment; a certain pixel point (i, j) of the Laplace correction matrix L can be expressed as ^[12] :

第四步，去云雾图像色彩域校正：对图像的R，G，B三原色通道分别进行去云雾运算：The fourth step is to correct the color gamut of the cloudy image: perform the cloudy operation on the R, G, and B three primary color channels of the image respectively:

其中，I_c(x,y)为原始图像的指定通道，t₀为设定的大气耗散函数下界值，通常取0.1，然而处理过的图像较难恢复到人眼满意的舒适度，因此，本发明引入了RGB色彩域校正：人类视觉颜色感知的研究和统计结果表明，大量人眼颜色感知较好的图像，其RGB三分量之间有很强的相关性；本发明基于Imagine Macmillan库统计了四组视觉效果良好的图像，求出了RGB三个分量相关系数矩阵，在其图像融合算法中作为目标值，分量间相关系数及统计方差见表1，Among them, I _c (x, y) is the specified channel of the original image, and t ₀ is the lower limit value of the set atmospheric dissipation function, which is usually 0.1. However, it is difficult to restore the processed image to the comfort level of the human eye, so , the present invention introduces RGB color gamut correction: the research and statistical results of human visual color perception show that a large number of images with better color perception of human eyes have a strong correlation between the three components of RGB; the present invention is based on the Imagine Macmillan library Four groups of images with good visual effects were counted, and the three-component correlation coefficient matrix of RGB was obtained, which was used as the target value in the image fusion algorithm. The correlation coefficients and statistical variances between components are shown in Table 1.

表1经过统计的彩色图像分量间相关系数Table 1 Statistical correlation coefficients between color image components

表中任意两个分量X和Y的相关系数定义为：The correlation coefficient of any two components X and Y in the table is defined as:

其中：

为X，Y均值，显然-1≤r≤1，因此颜色校正存在一种思路：寻求一个线性变换，将图像的RGB分量进行变换，使其分量间相关系数矩阵符合表1，这样便可以使输出颜色较符合人眼视觉，in:

is the mean value of X and Y, obviously -1≤r≤1, so there is an idea for color correction: seek a linear transformation, transform the RGB components of the image, and make the correlation coefficient matrix between the components conform to Table 1, so that the The output color is more in line with human vision,

对去除云雾的图像J(x,y)中任一像素点m_xy=[m_R,m_G,m_B]^T，进行如下的RGB色彩域校正^[13]，For any pixel point m _xy =[m _R ,m _G ,m _B ] ^T in the cloud-removed image J(x,y), perform the following RGB color gamut correction ^[13] ,

对于待校正图像的C_m可通过

求得(其中

)；或者也可以根据上式中关系求得，文献[13]中通过并设σ_n1=σ_n2=σ_n3来求出上式中的σ_nk(k=1,2,3)，以保证运算前后功率相等；The C _m of the image to be corrected can be obtained by

obtain (where

); or it can also be obtained according to the relationship in the above formula. In the literature [13], the And set σ _n1 =σ _n2 =σ _n3 to find σ _nk (k=1,2,3) in the above formula to ensure that the power before and after the operation is equal;

实际上色彩在校正时需要保持原有各个分量功率比例关系，这样才不至于出现色偏，所以求此标准差时，本发明中并不平均分配，而是采用原有图像各分量比例分配标准差值，同时由于目标分量间相关系数存在方差，这在实验中很重要，为了更加符合统计值，本步骤采用均值加减方差作为另外两个分量间相关系数矩阵与R_n分别求出校正后的三幅图像，之后求平均得到最后的校正图像。In fact, the color needs to maintain the original power ratio of each component when correcting, so that there will be no color shift. Therefore, when calculating the standard deviation, the present invention does not distribute it evenly, but uses the ratio distribution standard of the original image components. At the same time, due to the variance of the correlation coefficient between the target components, this is very important in the experiment. In order to be more in line with the statistical value, this step uses the mean value plus or minus the variance as the correlation coefficient matrix between the other two components and R _n to obtain the corrected The three images are then averaged to obtain the final corrected image.

以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims

1. a method for removing atmospheric degradation image cloud and fog, is characterized in that, the method for described removal atmospheric degradation image cloud and fog comprises the following steps:

Acquisition of dark channel images, median filtering, and adaptive decomposition of atmospheric light images;

Refinement of the atmospheric transfer function;

Visual correction of color gamut.

2. The method for removing clouds and fog in atmospheric degradation images according to claim 1, wherein the dark channel image generation adopts the idea of image adaptive block processing, and obtains the corresponding dark channel of the original image on the basis of the dark channel prior theory The image _ID (x, y) is smoothed using a median filter, and the generated dark channel image _ID (x, y) can be defined by the following formula:

{I I}_{D D.} ((x x,, y the y)) = = \underset{c c &Element; &Element; {{r r,, g g,, b b}}}{min min} ((\underset{Ω Ω}{min min} (({I I}_{c c} ((x x,, y the y))))))

Among them, I _c (x, y) is the specified channel of the original image; Ω is the adaptive image sub-block, which is a square area centered on (x, y). In order to prevent the rough phenomenon caused by the depth jump, To avoid checkerboard effects and halo traces in the results, it is necessary to perform median filtering on the dark channel image;

3. the method for removing atmospheric degradation image clouds and fog as claimed in claim 1, is characterized in that, atmospheric light image acquisition adopts self-adaptive two-dimensional empirical pattern decomposition, after carrying out 5 decompositions, obtains the corresponding atmospheric light image of original image A(x,y), and uniform local atmospheric light through low-pass filtering.

4. the method for removing atmospheric degradation image clouds and fog as claimed in claim 3, is characterized in that, self-adaptive two-dimensional experience mode is decomposed, and concrete implementation steps are as follows:

Step 1, identify the regional extreme points in the image through the geodesic operator in mathematical morphology;

Step 2: Use the optimized radial basis function interpolation method to construct the envelope surface, perform interpolation operations on the maximum point and minimum point point respectively, and obtain the maximum value point envelope surface and the minimum value point envelope after the operation surface, average the two surface data to obtain the mean envelope surface data, therefore, the problem of solving the upper and lower envelopes in the image space is transformed into the problem of interpolation and reconstruction of discrete data points on the three-dimensional surface, and the solution to the envelope plane can be expressed as:

s the s ((x x)) = = {c c}_{00} + + {c c}_{11} x x + + {c c}_{22} y the y + + {Σ Σ}_{i i = = 11}^{N N} {λ λ}_{i i} φ φ ((| | | | x x - - {x x}_{i i} | | | |))

Among them, s(x) is an interpolation node, c ₀ , c ₁ , c ₂ , and λ _i are polynomial coefficients and radial basis function combination coefficients, ‖·‖ is Euclidean norm, φ(·) is radial The basis function, by solving a large-scale linear equation system of N+3 elements, the values of c ₀ , c ₁ , c ₂ , and λ _i are obtained, and then the coordinate values of all points are substituted to obtain the entire interpolation surface. For φ(·) There are thin-plate, multiquadratics and other methods to choose, and the selection of improved φ( ) is:

φ φ ((R R)) = = \{\begin{matrix} {((11 - - R R))}^{P P} PR PR,, & R R < < 11 \\ 00,, & R R > > 11 \end{matrix}

Among them, R=‖xx _i ‖ is the Euclidean distance, and P is a constant coefficient;

Step 3, subtract the mean envelope surface from the original surface;

Step 4, judging whether the termination condition is satisfied, because the number of zero-crossing points of the two-dimensional intrinsic modulus function cannot be counted, so the constraint condition of the two-dimensional intrinsic modulus function can be used as the stopping condition of the screening process when decomposing the two-dimensional empirical model. An efficient Cauchy-type convergence condition can also be used:

SD SD = = {Σ Σ}_{x x}^{M m} {Σ Σ}_{y the y}^{N N} \frac{{| | {f f}_{k k - - 11} ((x x,, y the y)) - - {f f}_{k k} ((x x,, y the y)) | |}^{22}}{{f f}_{k k - - 11}^{22} ((x x,, y the y))}

Among them, f _k (x, y) is the pixel value of point (x, y) on the image when the k-th layer adaptive two-dimensional empirical mode is decomposed, and SD can be a number between 0.2 and 0.3;

Repeat steps (1) to (3) until the given termination conditions are satisfied, and the first layer two-dimensional intrinsic modulus image bimf ₁ (x, y) is obtained, and the original image is subtracted from bimf ₁ (x, y) to obtain The first layer residual image residue1, repeat steps (1) to (4) for residue1, and sequentially obtain the N-layer two-dimensional intrinsic modulus image of the image and the N-th layer residual image. In the above process, the extreme points are solved , Plane interpolation and filtering stop conditions are the core of this algorithm. Generally, a pure atmospheric light image can be obtained by performing five decompositions, so the final result is expressed as:

{r r}_{55} ((x x,, y the y)) = = f f ((x x,, y the y)) - - {Σ Σ}_{k k = = 11}^{55} {bimf bimf}_{k k} ((x x,, y the y))

Among them, bimf _k (x, y) is the two-dimensional intrinsic modulus image of the k-th layer, and r ₅ (x, y) is the trend image after five layers of decomposition. Some atmospheric degradation images in practice have very low atmospheric illumination. It is difficult to satisfy the local smoothness of the image. In order to overcome this shortcoming and improve the robustness of the algorithm, low-pass filtering can be used to smooth the low-frequency illuminance information in the atmospheric illumination image.

5. the method for removing atmospheric degradation image clouds and fog as claimed in claim 1, is characterized in that, the refinement of atmospheric transfer function adopts the darkest channel processing thought, by following formula:

{t t}_{11} ((x x,, y the y)) = = 11 - - w w \underset{Ω Ω}{min min} ((\frac{{I I}_{D D.} ((x x,, y the y))}{A A ((x x,, y the y))}))

Obtain a rough atmospheric transfer function t ₁ , where Ω is an adaptive image sub-block, and w is a parameter introduced to maintain the authenticity of the restored image, usually taking a value between 0 and 1, and taking 0.95 in the experiment;

Using the softmatting algorithm, the rough atmospheric transfer function t ₁ is modified by the Laplace matrix, and t ₁ is refined by optimizing the following energy equation:

E(t)=t ^T Lt+λ(tt ₁ ) ^T U(tt ₁ )

Obtain the refined atmospheric transfer function t, where λ is the normalization parameter, U is the identity matrix with the same size as the image, and L is the Laplace correction matrix, and perform a sparse linear representation of the above energy equation:

(L+λU)t=λt ₁

Among them, λ is a very small value, which is set to 10 ^-4 in the experiment; a certain pixel point (i, j) of the Laplace correction matrix L can be expressed as:

\underset{k k | | ((i i,, j j)) &Element; &Element; {w w}_{k k}}{Σ Σ} {{{δ δ}_{ij ij} - - \frac{11}{| | {w w}_{k k} | |} [[11 + + {(({I I}_{i i} - - {μ μ}_{k k}))}^{T T} {(({Σ Σ}_{k k} + + \frac{ϵ ϵ}{| | {w w}_{k k} | |} {U u}_{33}))}^{- - 11} (({I I}_{j j} - - {μ μ}_{k k}))]]}}

Among them, δ _ij is the Kronecker function; μ _k and Σ _k are the mean and covariance matrix of pixels in window w _k ; |w _k | is the number of pixels in window w _k ; ε is the normalization parameter ; U ₃ is a unit matrix of 3 by 3; when performing operations, the image matrix has been expanded by column vectors and converted into one-dimensional vectors, and I _i and I _j represent the subscript i and j pixels in the one-dimensional vector point value;

6. the method for removing atmospheric degradation image cloud and fog as claimed in claim 1, is characterized in that, the specific operation of removing cloud and fog image color gamut correction is, to the R of image, G, B three primary color channels carry out cloud and fog operation respectively:

{J J}_{c c} ((x x,, y the y)) = = \frac{{I I}_{c c} ((x x,, y the y)) - - A A ((x x,, y the y))}{max max ((t t ((x x,, y the y)),, {t t}_{00}))} + + A A ((x x,, y the y)),, c c &Element; &Element; {{r r,, g g,, b b}}

Among them, I _c (x, y) is the specified channel of the original image, t ₀ is the lower bound value of the set atmospheric dissipation function, usually 0.1, and there is a strong correlation between the three components of RGB, based on the statistics of the ImagineMacmillan library Four groups of images with good visual effects, the three-component correlation coefficient matrix of RGB is obtained, which is used as the target value in the image fusion algorithm, and the correlation coefficient of any two components X and Y is defined as:

r r = = \frac{{Σ Σ}_{m m} {Σ Σ}_{n no} (({X x}_{mn mn} - - \overset{&OverBar; &OverBar;}{X x})) (({Y Y}_{mn mn} - - \overset{&OverBar; &OverBar;}{Y Y}))}{\sqrt{{Σ Σ}_{m m} {Σ Σ}_{n no} {(({X x}_{mn mn} - - \overset{&OverBar; &OverBar;}{X x}))}^{22} {Σ Σ}_{m m} {Σ Σ}_{n no} {(({Y Y}_{mn mn} - - \overset{&OverBar; &OverBar;}{Y Y}))}^{22}}}

in:

7. The method for removing cloud and fog in atmospheric degradation image as claimed in claim 6, characterized in that, any pixel point m _xy =[m _R , m _G , m _B ] in the image J (x, y) for removing cloud and fog ^T , perform the following RGB color gamut correction,

\{\begin{matrix} {n no}_{xy xy} = = {B B}^{T T} {m m}_{xy xy} \\ {C C}_{n no} = = {B B}^{T T} {C C}_{m m} B B \end{matrix}

Among them, n _xy is the pixel vector after correction, B is the color gamut correction matrix, C _m , C _n are the covariance matrices between image color channels before and after correction, respectively, in order to solve the color correction matrix B, the positive definite matrices C _m , Cholesky decomposes C _n to get C _m = Q _m ^T Q _m , C _n = Q _n ^T Q _n , easy to obtain B = Q _m ^-1 Q _n , let the correlation coefficient matrix formed by the correlation coefficient between components be R _n , it is easy to know that there is the following relationship between R _n and C _n :

{C C}_{n no} = = [\begin{matrix} {σ σ}_{n no 11} & 00 & 00 \\ 00 & {σ σ}_{n no 22} & 00 \\ 00 & 00 & {σ σ}_{n no 33} \end{matrix}] {R R}_{n no} {[\begin{matrix} {σ σ}_{n no 11} & 00 & 00 \\ 00 & {σ σ}_{n no 22} & 00 \\ 00 & 00 & {σ σ}_{n no 33} \end{matrix}]}^{T T}

The C _m of the image to be corrected can be obtained by

obtain, among them

Or it can also be obtained according to the relationship in the above formula;

8. the method for removing atmospheric degradation image cloud and fog as claimed in claim 6, is characterized in that, removes cloud and fog image color gamut correction and adopts mean value plus and minus variance as correlation coefficient matrix and _R between other two components to obtain corrected respectively Three images are then averaged to obtain the final corrected image.