CN106846260A - Video defogging method in a kind of computer - Google Patents

Abstract

The invention discloses video defogging method in a kind of computer, its step includes：Step 1, the edge amplitude Density Estimator comentropy based on consecutive frame difference extracts key frame of video；Step 2, representative pixel points are extracted based on key frame marginal information amplitude information entropy；Step 3, based on Cross Correlation Matching algorithm and generalized procrustes analysis registering key frame of video pixel-by-pixel；Step 4, the spreading rate based on color Estimation of Mean atmosphere illumination intensity and based on two-dimensional nucleus regression optimization key frame of video；Step 5, the spreading rate of video non-key frame is estimated based on Catmull-Rom cubic splines；Step 6, according to atmospherical scattering model, using the model parameter estimated, the haze sequence of video images after output recovery.

Description

Video defogging method in computer

Technical Field

The invention belongs to the technical field of computer image processing, and particularly relates to a video defogging method in a computer.

Background

The fields of security systems, urban traffic, military technology, criminal investigation, navigation, meteorology, astronomy and the like often need to acquire haze video image sequences from outdoor monitoring. The poor weather such as haze makes the image color that video monitoring was caught dim, the contrast ratio becomes low, and the serious degradation of image quality has directly influenced the visual effect of image, seriously influences their application. The influence of haze weather is removed from the video, and the restoration of the color and the contrast of the image has important research significance and practical significance.

There are several defogging methods for videos: on the basis of a depth of field distribution model, the Wangxiaitong of the navy Dalian naval vessel academy acquires the reference depth of field and the sky brightness of an image by using a dark primary color idea, and then realizes the defogging treatment of the video at sea by using a relative depth of field method. The Wangxiaitong also combines the traditional two-dimensional empirical mode decomposition to provide a construction method of a high-frequency auxiliary signal, the high-frequency auxiliary signal is added into an original video image signal, the obtained auxiliary signal of the intrinsic mode component is decomposed, and the frequency component of the original signal, which is closest to the auxiliary signal, is obtained, so that the problem of the defogging of the maritime video with deficient local or global extreme points is solved. Hautiere et al have proposed video defogging studies based on-board camera systems for vehicles whose visible light range is very sensitive to the atmospheric environment. Lilongli provides a defogging algorithm for a lossy compressed video image, and the obvious characteristic that the image can be divided into high-frequency sub-bands and low-frequency sub-bands through wavelet transformation is utilized to help find out the irregular regions, so that the transmissivity of the irregular regions is processed, and the phenomenon of uneven color is eliminated after the image is restored through a dark primary color prior algorithm.

Existing video defogging methods can be divided into two categories: the basic idea of the first category of methods is to calculate the propagation rate, which is then used to perform a defogging process on each frame of the video. The requirement of using a reference image is too strict, so that the method is difficult to realize in practical application. The second method is to separate the background image and the foreground image, to be clarified by the relative defogging algorithm, and to fuse the two results to obtain the final defogged video. The foreground detection method comprises the following steps: (1) background subtraction is a method of constructing a background model image of a scene and detecting a moving area according to the difference between the current image and the background model image, and the method is mainly applied to the detection of the movement of a target under the condition that a camera is fixed. However, information detected by the background subtraction method is generally incomplete, and a ghost phenomenon is easily generated. (2) The time difference method is used for extracting a motion region by calculating two or three adjacent frames of images in a continuous haze video image sequence by adopting a pixel difference method and thresholding the result. However, it is difficult to extract all the related feature pixels, and voids are easily generated inside the motion entity. (3) The optical flow method is based on the basic principle that each pixel in an image is endowed with a velocity vector to form an image motion field, the projection relation of the velocity vector and the motion field can be used for calculating the one-to-one corresponding relation between points on the image and real objects at a certain moment of motion, and the image is dynamically analyzed according to the change characteristics of the vector.

Most of the existing video defogging methods mainly focus on processing each frame image of the video independently, and do not utilize redundant information among each frame of the video to reduce the calculation amount, so that the problem of low speed exists. In addition, in the video defogging process, the space-time consistency of the video is not considered, and the continuity and the smoothness of video frames and frames cannot be maintained, so that the problem of flicker is caused.

Disclosure of Invention

In order to solve the technical problem, the invention provides a video defogging method based on two-dimensional kernel regression and a Catmull-Rom cubic spline in a computer. The method is characterized in that the video defogging method is realized by utilizing the technologies of edge amplitude kernel density estimation, information entropy, generalized alignment algorithm, two-dimensional kernel regression, Catmull-Rom and the like. The method comprises the following specific steps:

step 1, inputting a haze video image sequence, and extracting video key frames based on an information entropy estimated by edge information amplitude kernel density of adjacent frame difference;

step 2, extracting representative pixel points of the key frame based on the edge amplitude information entropy;

step 3, registering video key frames pixel by pixel based on a cross-correlation matching algorithm and a generalized alignment algorithm;

step 4, estimating the atmospheric illumination intensity and the two-dimensional kernel regression optimization video key frame propagation rate based on the color mean value;

step 5, estimating the transmission rate of the non-key frames of the video based on a Catmull-Rom cubic spline;

and 6, outputting the recovered haze video image sequence according to the atmospheric scattering model.

The step 1 of the invention comprises the following steps:

step 1-1, inputting a haze video image sequence, and calculating the difference of brightness channels of two adjacent frames of the haze video image sequence: suppose that the haze video image sequence has N frames, respectively f₁,f₂,…,f_N，f_NExpressing the sequence of the N frame of haze video images, converting each frame of image from an RGB color space into a YUV (Y expresses brightness and chroma), and calculating the absolute value of the difference of brightness Y channels of the adjacent two frames of YUV spaces by the following formula:

wherein,respectively representing the luminance Y channel of the N-1 th frame and the luminance Y channel of the N-th frame, d_N-1Representing the absolute value of the difference between the luminance Y channel of the N-1 th frame and the luminance Y channel of the N-th frame, the luminance Y channel of all N frames having a size M₁×M₂Wherein M is₁And M₂Respectively counting the number of rows and columns of the video frame;

step 1-2, calculating the edge amplitude information of the absolute value of the differences between all luminance Y channels, d_rExpressing the absolute value of the difference between the r-th luminance Y channel, r is more than or equal to 1 and less than or equal to N-1, and calculating d by the following formula_rEdge amplitude information e of the ith row and jth column of pixels_r(i,j)：

Wherein i is more than or equal to 1 and less than or equal to M₁，1≤j≤M₂，Representing the horizontal direction edge of the pixel of the ith row and the jth column, is calculated by the following formula

The vertical direction edge of the pixel of the ith row and the jth column is expressed by the following formula

Step 1-3, calculating d_rKernel density function of edge information amplitude of (1): for any edge amplitude e, the kernel density function p (e) is:

wherein,gaussian kernel function of variable u being a one-dimensional kernel functionThe smoothing parameter s is calculated by the following formula:

where σ represents the set of edge amplitudes e_r＝{e_r(1,1),e_r(1,2),…,e_r(M₁,M₂) Standard deviation of, edge magnitude bins are calculated according to step 1-2, e_r(M₁,M₂) Denotes the M th₁Line M₂Edge magnitude of the column.

Step 1-4, extracting a key frame of the haze video image sequence based on the kernel density function information entropy of the edge information amplitude: at the edge amplitude set e_rIs sampled n in equal steps between the minimum and maximum values of₁(invention n)₁100) data, labeled respectively:

c₁＝min(e_r)，c₂＝c₁+Δ，c₃＝c₁+2Δ,…,c_n＝c₁+(n₁-1)Δ，

wherein the step lengthc_nDenotes the nth data, n₁The kernel density function corresponding to each data is respectively Denotes c_nCorresponding kernel density function, calculating d by the following formula_rInformation entropy H of_r：

If d is_rIs greater than a threshold value H₀，H₀If the value is 0.1, the r frame f in the haze video image sequence_rAre key frames.

The step 2 of the invention comprises:

let the r frame f in the haze video image sequence_rFor key frames, for edge amplitude set { e }_r(1,1),e_r(1,2),…,e_r(M₁,M₂) Sorting the elements of the first n according to descending order₂With greater edge amplitude corresponding to positionThe pixel points of (2) are taken as representative pixel points, whereinRespectively representing the edge amplitude value in the r-th frame as the n-th frame₂The abscissa and ordinate of a large pixel point.

The step 3 of the invention comprises:

two adjacent key frames in the haze video image sequence are respectively set asAndr₁,r₂respectively represent the serial numbers of adjacent key frame videos, and r is more than or equal to 1₁≤r₂≤N，Andthe coordinates of the representative pixel points are respectivelyAndusing a cross-correlation matching algorithm, respectivelyAndselecting a window by taking the representative pixel point as the center,as a reference frame, the frame is,as a frame to be registered, with a reference frameEach representative pixel point is taken as a reference point, and in a frame to be registeredAnd searching a window with the maximum correlation coefficient as a matching point. Wherein the correlation coefficient is calculated as follows:

suppose thatThe center coordinate isOf the representative pixel point of (1) is composed of all the luminances in the windowThe vector is Y₁，The center coordinate isThe vector formed by all the luminances in the representative pixel point window is Y₂Then Y is₁And Y₂The correlation coefficient of (d) defines:

whereinAndrespectively represent vector Y₁Mean sum vector Y of₂Is measured.

And selecting a representative pixel point pair with a corresponding relation according to a cross-correlation matching algorithm, and calculating translation, rotation and scaling transformation parameters of a reference frame and a frame to be registered by using a generalized alignment algorithm to realize pixel-by-pixel registration between two key frames.

Step 4 of the invention comprises the following steps:

step 4-1, establishing an atmospheric scattering illumination model of the video key frame: the key frame atmosphere scattering illumination model of the haze video image sequence is as follows:

I_c(p)＝t(p)J_c(p)+(1-t(p))A_c，

wherein, J_c(p) and I_c(p) respectively representing the c channel value of the haze-free video key frame pixel point p and the c channel value of the haze video key frame pixel point p, c ∈ { R, G, B }, R, G and B respectively representing three colors of red, green and blue, and t (p) ∈ [0,1]Representing the atmospheric light transmittance, A, at a pixel point p_cRepresenting the atmospheric illumination intensity of the c channel;

step 4-2, estimating the atmospheric illumination intensity of the video key frame based on the color mean vector: the magnitude of the atmospheric illumination intensity is set to be a constant 240, and the average value is normalized by the red, green and blue color channels of the key frameAs the direction of the intensity of the atmospheric illumination, whereinRespectively representing the red channel mean value, the green channel mean value and the blue channel mean value of the key frame, and then the estimated values of the atmospheric illumination intensity of the red, green and blue channels are:

wherein A is_R、A_G、A_BRespectively representing an estimated value of the atmospheric illumination intensity of the red channel, an estimated value of the atmospheric illumination intensity of the green channel and an estimated value of the atmospheric illumination intensity of the blue channel;

4-3, optimizing the propagation rate based on two-dimensional kernel regression: preliminarily estimating the atmospheric illumination propagation rate t (p) of a pixel point p according to an optimal contrast method proposed by Jin-Han Kim:

and (3) optimizing the propagation rate by adopting a two-dimensional kernel regression method:

assuming that the propagation rate estimated value of the ith row and the jth column pixel point is t_i,jThe propagation rate t under the local window with radius w at the point is calculated by the following formula_i,jTwo-dimensional kernel regression function of

WhereinComputing a two-dimensional kernel regression function for a two-dimensional Gaussian kernel functionIn the formula (2), the denominator part is calculated as:

the molecular fraction was calculated as:

the step 5 of the invention comprises:

for two adjacent key frames in haze video image sequenceAndis provided withThe previous key frame of Next key frame ofIs composed ofRegistering the 4 neighboring key framesAndestimating and optimizingAndpropagation rate ofEstimation of position based on Catmull-Rom cubic splineAndnon-key frames in betweenThe propagation rate of the corresponding pixel inComprises the following steps:

wherein r is₁<k<r₂-r₁K is an integer, and the parameter x represents two key framesAndthe k-th non-key frame in between,

the step 6 of the invention comprises: and recovering and outputting the haze video image sequence according to the following formula:

has the advantages that: the invention has the advantages that the video defogging method based on the two-dimensional kernel regression and the Catmull-Rom cubic spline is provided, the video key frames are extracted based on the edge amplitude kernel density information entropy of the difference of adjacent frames, the low gray contrast of the haze video image sequence can be overcome, and the key frames in the haze video image sequence can be extracted quickly and accurately. The atmospheric illumination intensity estimation method based on the color mean value can better reflect the illumination effect. The video key frame propagation rate optimization based on the two-dimensional kernel regression avoids the problem of incoherent recovered images caused by abrupt change of the propagation rate. The propagation rate estimation method based on the Catmull-Rom cubic spline for the non-key frames of the video avoids the problems that the propagation rate needs to be estimated again every frame and the video is in a flash state after being recovered. The innovation points of the invention comprise:

(1) the method provides the entropy extraction of the video key frames based on the edge amplitude kernel density information of the difference of adjacent frames. The gray contrast of the haze image is low, the edge information can well reflect the content transformation of the haze video image sequence, and the key frames in the haze video image sequence can be extracted quickly and accurately.

(2) And the atmospheric illumination intensity estimation based on the color mean value and the video key frame propagation rate optimization based on two-dimensional kernel regression are provided. The direction of atmospheric illumination in the haze image is important, and a scalar is adopted in a common defogging method, so that the direction in which the illumination cannot be emitted is reflected in the recovered image. The invention provides that the illumination effect can be better embodied by taking the average value of different color channels as the direction of the atmospheric illumination intensity. Two-dimensional kernel regression optimization is adopted for the propagation rate of the video key frame, so that the problem of incoherent recovery images caused by abrupt change of the propagation rate is avoided.

(3) Video non-key frame propagation rate estimation based on Catmull-Rom cubic spline is provided. The propagation rate of the non-key frame is estimated by using the estimated propagation rate of the key frame, so that the problems that the propagation rate needs to be estimated again every frame and the flash exists after the video is recovered are solved.

Drawings

The foregoing and other advantages of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.

FIG. 1 is a flow chart of the present invention.

Detailed Description

The method comprises the steps of extracting key frames of haze videos, extracting representative pixel points in the key frames, matching the key frames pixel by pixel, estimating atmospheric illumination intensity, optimizing key frame propagation rate, estimating non-key frame propagation rate and the like, and achieves defogging of the haze images of the videos rapidly, wherein the specific working flow is shown in fig. 1.

Step 1, extracting video key frames based on information entropy estimated by edge information amplitude kernel density of adjacent frame difference;

(1-1) calculating the difference between the luminance channels of two adjacent frames. Suppose that the sequence of haze video images collected from monitoring has N frames, respectively f₁,f₂,…,f_NEach frame of image is converted from RGB color space to YUV space. Calculating the absolute value of the difference of the luminance Y channels of the YUV space of the difference between two adjacent frames:

wherein Respectively representing the luminance Y channels of the 1 st, 2 nd, 3 rd 3 … th, N-1 th, N frames, with the size M₁×M₂。

(1-2) calculating the absolute value d of the difference between luminance Y channels₁,d₂,…,d_N-1The edge information amplitude of (1).

d_rR is more than or equal to 1 and less than or equal to N-1, and i is more than or equal to 1 and less than or equal to M₁) And j is not less than 1 and not more than M₂) Edge information amplitude of position:

wherein,the edge in the horizontal direction is shown,

the edge in the vertical direction is shown,

(1-3) calculating the absolute value d of the difference between luminance Y channels_r(1. ltoreq. r. ltoreq.N-1) kernel density function of edge amplitudes. For any one edgeAmplitude e, whose kernel density function is:

whereinFor one-dimensional kernel functions, the invention uses Gaussian kernel functionsThe calculation formula of the smoothing parameter s is as follows:

where σ represents the set of edge amplitudes e_r＝{e_r(1,1),e_r(1,2),…,e_r(M₁,M₂) Standard deviation of.

(1-4) extracting key frames based on the information entropy of the edge amplitude kernel density function:

at the edge amplitude set e_rIs sampled n in equal steps between the minimum and maximum values of₁Data, labeled respectively as:

c₁＝min(e_r)，c₂＝c₁+Δ，c₃＝c₁+2Δ,…,c_n＝c₁+(n₁-1)Δ (7)

wherein the step sizeTheir corresponding kernel density functions are respectivelyThen d_r(r is more than or equal to 1 and less than or equal to N-1) the information entropy is as follows:

if d is_r(r is more than or equal to 1 and less than or equal to N-1) information entropy is more than threshold value H₀(the value of the invention is 0.1), the r frame f in the haze video image sequence_rAre key frames.

Step 2, extracting representative pixel points of the key frame based on the edge amplitude information entropy;

let the r frame f in the haze video image sequence_rFor key frames, for edge amplitude set { e }_r(1,1),e_r(1,2),…,e_r(M₁,M₂) Sorting the elements of the first n according to descending order₂Of corresponding position of edge amplitudeThe pixel points of (2) are taken as representative pixel points.

Step 3, registering video key frames pixel by pixel based on a cross-correlation matching algorithm and a generalized alignment algorithm;

two adjacent key frames in the haze video image sequence are respectively set asAnd(r₁,r₂respectively representing the sequence numbers of the videos), and their representative pixel points are respectivelyAndusing a cross-correlation matching algorithm, respectively(reference frame) andand (3) selecting a window by taking the representative pixel point of the frame to be registered as the center, and searching the window with the maximum correlation coefficient in the frame to be registered as a matching point by taking each representative pixel point of the reference frame as a reference point. And selecting a representative pixel point pair with a corresponding relation according to a cross-correlation matching algorithm, and calculating translation, rotation and scaling transformation parameters of a reference frame and a frame to be registered by using a generalized alignment algorithm to realize pixel-by-pixel registration between two key frames.

Step 4, estimating the atmospheric illumination intensity based on the color mean value and optimizing the video key frame propagation rate based on two-dimensional kernel regression;

(4-1) establishing an atmospheric scattering illumination model of the video key frame;

the key frame atmosphere scattering illumination model of the haze video image sequence is as follows:

I_c(p)＝t(p)J_c(p)+(1-t(p))A_c(9)

wherein J_c(p) and I_c(p) c channel values of pixel points p of the original video key frame and the observed video key frame are respectively represented, c ∈ { R, G, B } represents three colors of red, green and blue, t (p) ∈ [0,1]Representing the atmospheric light transmittance, A, at a pixel point p_cRepresents the atmospheric illumination intensity of the c ∈ { R, G, B } channel.

(4-2) estimating atmospheric illumination intensity of the video key frame based on the color mean vector;

the magnitude of the atmospheric illumination intensity is set at a constant 240. Average value normalized by three color channels of red, green and blue of key frameAs the direction of the intensity of the atmospheric illumination, whereinThree of red, green and blue representing key framesIndividual color channel means. Then the estimated values of the atmospheric illumination intensities of the red, green and blue channels are:

(4-3) two-dimensional kernel regression-based propagation rate optimization:

initially estimating the propagation rate of the pixel point p based on the optimal contrast method proposed by Jin-Han Kim

In this method, the propagation rate between different targets is abrupt, and the blocking effect exists. The depth distance between adjacent pixels in the video keyframe is gradual, and the propagation rate should also be gradual. Therefore, the invention provides a two-dimensional kernel regression method for optimizing the coarse estimation of the propagation rate of the formula (11);

assuming that the propagation rate estimated value of the pixel point of the ith row and the jth column is t_i,jThen the propagation rate t is at the local window with radius w at that point_i,jThe two-dimensional kernel regression smoothing of (a) is:

whereinIs a two-dimensional gaussian kernel function. The denominator part is calculated as:

the molecular fraction was calculated as:

step 5, estimating the transmission rate of the non-key frames of the video based on a Catmull-Rom cubic spline;

two adjacent key frames in the haze video image sequence are respectively set asAnd(1≤r₁≤r₂≤N)，the previous key frame ofThe next key frame ofRegistering the 4 neighboring key frames using the method of step 3Andthe method of step 4 is used to estimate the corresponding optimized propagation rates in the 4 frames asEstimation of position based on Catmull-Rom cubic splineAndnon-key frames in between(r₁<k∈Z<r₂-r₁) The propagation rate of the corresponding pixel inIs composed of

Wherein the parametersRepresenting the kth non-key frame between two key frames.

And 6, outputting the recovered haze video image sequence by using the estimated model parameters according to the atmospheric scattering model.

Estimating atmospheric illumination of the haze video image sequence according to a formula (10) in the step (4-2), optimizing the transmission rate of the video key frame according to the step (4-3), estimating the transmission rate of the video non-key frame according to the step (5), and recovering and outputting the haze video image sequence according to the following formula

The present invention provides a method for defogging a video image in a computer, and a plurality of methods and ways for implementing the same, and the above description is only a preferred embodiment of the present invention, it should be noted that, for those skilled in the art, a plurality of modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention. All the components not specified in the present embodiment can be realized by the prior art.

Claims (7)

1. A method for removing fog from video in a computer is characterized by comprising the following steps:

step 1, inputting a haze video image sequence, and extracting video key frames based on an information entropy estimated by edge information amplitude kernel density of adjacent frame difference;

step 2, extracting representative pixel points of the key frame based on the edge amplitude information entropy;

step 3, registering video key frames pixel by pixel based on a cross-correlation matching algorithm and a generalized alignment algorithm;

step 4, estimating the atmospheric illumination intensity and the two-dimensional kernel regression optimization video key frame propagation rate based on the color mean value;

step 5, estimating the transmission rate of the non-key frames of the video based on a Catmull-Rom cubic spline;

and 6, outputting the recovered haze video image sequence according to the atmospheric scattering model.

2. The method of claim 1, wherein step 1 comprises the steps of:

step 1-1, inputting a haze video image sequence, and calculating the difference of brightness channels of two adjacent frames of the haze video image sequence: suppose that the haze video image sequence has N frames, respectively f₁,f₂,…,f_N，f_NExpressing an N frame haze video image sequence, converting each frame image from an RGB color space to a YUV space, and calculating the absolute value of the difference of brightness Y channels of two adjacent frames of YUV spaces by the following formula:

$d_{N-1}=|f_{N-1}^Y-f_N^Y|,$

wherein,respectively representing the luminance Y channel of the N-1 th frame and the luminance Y channel of the N-th frame, d_N-1Representing the absolute value of the difference between the luminance Y channel of the N-1 th frame and the luminance Y channel of the N-th frame, the luminance Y channel of all N frames having a size M₁×M₂Wherein M is₁And M₂Respectively counting the number of rows and columns of the video frame;

step 1-2, calculating the edge amplitude information of the absolute value of the differences between all luminance Y channels, d_rExpressing the absolute value of the difference between the r-th luminance Y channel, r is more than or equal to 1 and less than or equal to N-1, and calculating d by the following formula_rEdge amplitude information e of the ith row and jth column of pixels_r(i,j)：

$e_r(i,j)=\sqrt{{\left(e_r^h\right(i,j\left)\right)}^2+{\left(e_r^v\right(i,j\left)\right)}^2},$

Wherein i is more than or equal to 1 and less than or equal to M₁，1≤j≤M₂，Representing the horizontal direction edge of the pixel of the ith row and the jth column, is calculated by the following formula

$\begin{array}{c}{e}_r^h(i,j)=d_r(i+1,j-1)+2d_r(i,j+1)+d_r(i+1,j+1)\\ -d_r(i-1,j-1)-2d_r(i-1,j)-d_r(i-1,j+1)\end{array},$

The vertical direction edge of the pixel of the ith row and the jth column is expressed by the following formula

$e_r^v(i,j)=d_r(i-1,j+1)+2d_r(i,j+1)+d_r(i+1,j+1)-d_r(i-1,j-1)-2d_r(i,j-1)-d_r(i+1,j-1);$

Step 1-3, calculating d_rKernel density function of edge information amplitude of (1): for any edge amplitude e, the kernel density function p (e) is:

$p\left(e\right)=\frac{1}{M_1{M}_{2}s}\underset{i=1}{\overset{M_1}{\&Sigma;}}\underset{j=1}{\overset{M_2}{\&Sigma;}}{K}_s^1\left(\frac{e-e_r(i,j)}{s}\right),$

wherein,gaussian kernel function of variable u being a one-dimensional kernel functionThe smoothing parameter s is calculated by the following formula:

$s={\left(\frac{4}{3M_1{M}_2}\right)}^{1/5}\mathrm{\&sigma;},$

where σ represents the set of edge amplitudes e_r＝{e_r(1,1),e_r(1,2),…,e_r(M₁,M₂) Standard deviation of, edge magnitude bins are calculated according to step 1-2, e_r(M₁,M₂) Denotes the M th₁Line M₂Edge amplitude of the column;

step 1-4, extracting a key frame of the haze video image sequence based on the kernel density function information entropy of the edge information amplitude: at the edge amplitude set e_rIs sampled n in equal steps between the minimum and maximum values of₁Data, labeled respectively as:

c₁＝min(e_r)，c₂＝c₁+Δ，c₃＝c₁+2Δ,…,c_n＝c₁+(n₁-1)Δ，

wherein the step lengthn₁The kernel density function corresponding to each datum is p (c)₁),p(c₂),…,p(c_n1)，p(c_n1) Denotes c_nCorresponding kernel density function, calculating d by the following formula_rInformation entropy H of_r：

$H_r=-\underset{r_1=1}{\overset{n_1}{\&Sigma;}}{c}_{r_1}log\left(p\right(c_{r_1}\left)\right),$

If d is_rIs greater than a threshold value H₀，H₀If the value is 0.1, the r frame f in the haze video image sequence_rAre key frames.

3. The method of claim 2, wherein step 2 comprises:

let the r frame f in the haze video image sequence_rFor key frames, for edge amplitude set { e }_r(1,1),e_r(1,2),…,e_r(M₁,M₂) Sorting the elements of the first n according to descending order₂With greater edge amplitude corresponding to positionThe pixel points of (2) are taken as representative pixel points, whereinRespectively representing the edge amplitude value in the r-th frame as the n-th frame₂The abscissa and ordinate of a large pixel point.

4. The method of claim 3, wherein step 3 comprises:

two adjacent key frames in the haze video image sequence are respectively set asAndr₁,r₂respectively represent the serial numbers of adjacent key frame videos, and r is more than or equal to 1₁≤r₂≤N，Andthe coordinates of the representative pixel points are respectivelyAndusing a cross-correlation matching algorithm, respectivelyAndselecting a window by taking the representative pixel point as the center,as a reference frame, the frame is,as a frame to be registered, with a reference frameEach representative pixel point is taken as a reference point, and in a frame to be registeredSearching a window with the maximum correlation coefficient as a matching point, wherein the correlation coefficient is calculated as follows:

suppose thatThe center coordinate isThe vector formed by all the luminances in the window of the representative pixel point is Y₁，The center coordinate isThe vector formed by all the luminances in the representative pixel point window is Y₂Then Y is₁And Y₂The correlation coefficient of (d) defines:

${\mathrm{\&rho;}}_1=\frac{\mathrm{\&Sigma;}(Y_1-{\stackrel{\&OverBar;}{Y}}_1)(Y_2-{\stackrel{\&OverBar;}{Y}}_2)}{\sqrt{{(Y_1-{\stackrel{\&OverBar;}{Y}}_1)}^2}\sqrt{{(Y_2-{\stackrel{\&OverBar;}{Y}}_2)}^2}},$

whereinAndrespectively represent vector Y₁Mean sum vector Y of₂The mean value of (a);

and selecting a representative pixel point pair with a corresponding relation according to a cross-correlation matching algorithm, and calculating translation, rotation and scaling transformation parameters of a reference frame and a frame to be registered by using a generalized alignment algorithm to realize pixel-by-pixel registration between two key frames.

5. The method of claim 4, wherein step 4 comprises the steps of:

step 4-1, establishing an atmospheric scattering illumination model of the video key frame: the key frame atmosphere scattering illumination model of the haze video image sequence is as follows:

I_c(p)＝t(p)J_c(p)+(1-t(p))A_c，

wherein, J_c(p) and I_c(p) respectively representing the c channel value of the haze-free video key frame pixel point p and the c channel value of the haze video key frame pixel point p, c ∈ { R, G, B }, R, G and B respectively representing three colors of red, green and blue, and t (p) ∈ [0,1]Representing the atmospheric light transmittance, A, at a pixel point p_cRepresenting the atmospheric illumination intensity of the c channel;

step 4-2, estimating the atmospheric illumination intensity of the video key frame based on the color mean vector: the magnitude of the atmospheric illumination intensity is set to be a constant 240, and the average value is normalized by the red, green and blue color channels of the key frameAs the direction of the intensity of the atmospheric illumination, whereinRespectively representing the red channel mean value, the green channel mean value and the blue channel mean value of the key frame, and then the atmospheric illumination intensity estimated values of the red channel, the green channel and the blue channel are as follows:

$A_R=240\frac{{\stackrel{\&OverBar;}{I}}_R}{{\stackrel{\&OverBar;}{I}}_R+{\stackrel{\&OverBar;}{I}}_G+{\stackrel{\&OverBar;}{I}}_B},A_G=240\frac{{\stackrel{\&OverBar;}{I}}_G}{{\stackrel{\&OverBar;}{I}}_R+{\stackrel{\&OverBar;}{I}}_G+{\stackrel{\&OverBar;}{I}}_B},A_B=240\frac{{\stackrel{\&OverBar;}{I}}_B}{{\stackrel{\&OverBar;}{I}}_R+{\stackrel{\&OverBar;}{I}}_G+{\stackrel{\&OverBar;}{I}}_B},$

wherein A is_R、A_G、A_BRespectively representing the estimated value of the atmospheric illumination intensity of the red channel and the atmosphere of the green channelAn estimated value of the illumination intensity and an estimated value of the atmospheric illumination intensity of the blue channel;

4-3, optimizing the propagation rate based on two-dimensional kernel regression: preliminarily estimating the atmospheric illumination propagation rate t (p) of a pixel point p according to an optimal contrast method proposed by Jin-Han Kim:

$t\left(p\right)=\max \left\{\underset{c\&Element;\{R,G,B\}}{\min }\underset{p\&Element;B}{\min }\right\{\frac{I_c\left(p\right)-A_c}{-A_c}\},\underset{c\&Element;\{R,G,B\}}{\max }\underset{p\&Element;B}{\max }\{\frac{I_c\left(p\right)-A_c}{255-A_c}\left\}\right\},$

and (3) optimizing the propagation rate by adopting a two-dimensional kernel regression method:

assuming that the propagation rate estimated value of the ith row and the jth column pixel point is t_i,jThe propagation rate t under the local window with radius w at the point is calculated by the following formula_i,jTwo-dimensional kernel regression function of

$t_{i,j}^{*}=\frac{\underset{k_1=-w}{\overset{w}{\&Sigma;}}\underset{k_2=-w}{\overset{w}{\&Sigma;}}{K}_h^2\left(\right(i+k_1,j+k_2),(i,j))t_{i+k_1,j+k_2}}{\underset{k_1=-w}{\overset{w}{\&Sigma;}}\underset{k_2=-w}{\overset{w}{\&Sigma;}}{K}_h^2\left(\right(i+k_1,j+k_2),(i,j))},$

WhereinComputing a two-dimensional kernel regression function for a two-dimensional Gaussian kernel functionIn the formula (2), the denominator part is calculated as:

$\begin{array}{c}\underset{k_1=-r}{\overset{r}{\&Sigma;}}\underset{k_2=-r}{\overset{r}{\&Sigma;}}{K}_h^2\left(\right(i+k_1,j+k_2),(i,j))\\ =\underset{k_1=-w}{\overset{w}{\&Sigma;}}\underset{k_2=-w}{\overset{w}{\&Sigma;}}\frac{1}{h_2}{K}_h^1\left(\frac{i+k_1-i}{h}\right)K_h^1\left(\frac{j+k_2-j}{h}\right)\\ =\underset{k_1=-w}{\overset{w}{\&Sigma;}}\underset{k_2=-w}{\overset{w}{\&Sigma;}}\frac{1}{h_2}{K}_h^1\left(\frac{k_1}{h}\right)K_h^1\left(\frac{k_2}{h}\right)\end{array},$

the molecular fraction was calculated as:

$\begin{array}{c}\underset{k_1=-w}{\overset{w}{\&Sigma;}}\underset{k_2=-w}{\overset{w}{\&Sigma;}}{K}_h^2\left(\right(i+k_1,j+k_2),(i,j))t_{i+k_1,j+k_2}\\ =\underset{k_1=-w}{\overset{w}{\&Sigma;}}\underset{k_2=-w}{\overset{w}{\&Sigma;}}\frac{1}{h_2}{K}_h^1\left(\frac{k_1}{h}\right).K_h^1\left(\frac{k_2}{h}\right)t_{i+k_1,j+k_2}\end{array}.$

6. the method of claim 5, wherein step 5 comprises:

for two adjacent key frames in haze video image sequenceAndis provided withThe previous key frame of The next key frame ofRegistering the 4 neighboring key framesAndestimating and optimizingAndpropagation rate ofEstimation of position based on Catmull-Rom cubic splineAndnon-key frames in betweenThe propagation rate of the corresponding pixel inComprises the following steps:

$t_{r_0+k}=\frac{1}{2}{\left[\begin{array}{c}{x}^3\\ x^2\\ x\\ 1\end{array}\right]}^T\left[\begin{array}{cccc}-1& 3& -3& 1\\ 2& -5& 4& -1\\ -1& 0& 1& 0\\ 0& 2& 0& 0\end{array}\right]\left[\begin{array}{c}{t}_{r_3}^{*}\\ t_{r_2}^{*}\\ t_{r_1}^{*}\\ t_{r_0}^{*}\end{array}\right],$

wherein r is₁<k<r₂-r₁K is an integer and parameterx represents two key framesAndthe k-th non-key frame in between,

7. the method of claim 6, wherein step 6 comprises: and recovering and outputting the haze video image sequence according to the following formula:

$J_c\left(p\right)=\frac{I_c\left(p\right)-A_c}{t\left(p\right)}+A_c.$