CN108093175A - A kind of adaptive defogging method of real-time high-definition video and device - Google Patents

Abstract

The invention relates to a real-time high-definition video adaptive defogging method and device. The device includes a video image acquisition module, a video image defogging processing module, a communication interface module, a video image encoding and display module; the video image acquisition module realizes video signal processing. Acquisition and decoding, the video image defogging processing module realizes the defogging processing of the video signal, the communication interface module realizes the command exchange between the device and the host computer, and the video image coding and display module realizes the video coding output and display. This method is based on the characteristics of large amount of high-definition image data, and adopts reasonable assumptions to propose a method for estimating the transmittance and atmospheric light intensity with a small amount of calculation; it greatly reduces the amount of calculation while improving the effect of defogging, ensuring The real-time performance of the dehazing algorithm makes the image smoother and more natural. The invention has a simple structure and is easy to realize, and can adaptively select the defogging intensity according to the concentration of fog and haze, and is more intelligent and humanized.

Description

A real-time high-definition video adaptive defogging method and device

technical field

The invention relates to a real-time high-definition video adaptive defogging method and device, which can be used to verify various defogging methods including real-time high-definition video adaptive defogging algorithms; a real-time high-definition real-time high-definition video is proposed for different fog and haze conditions Video adaptive defogging algorithm, this method can realize real-time adaptive defogging function for high-definition video, can significantly improve the video collection and observation effect, belongs to the field of video/image processing and computer vision.

Background technique

With the development of computer vision system and its application in military, transportation and security monitoring and other fields, image defogging has become an important research direction of computer vision. Images collected in severe weather such as fog and haze will be seriously degraded due to atmospheric scattering, which will make the image color off-white, reduce the contrast, and make it difficult to identify object features, which not only deteriorates the visual effect, but also reduces the appreciation of the image. , It will also affect the post-processing of the image, and it will also affect the work of various systems that rely on optical imaging instruments, such as satellite remote sensing systems, aerial photography systems, outdoor monitoring and target recognition systems, etc.

Defogging devices have been widely used in practice. However, the existing defogging devices usually can only achieve real-time defogging of SD video, or cannot meet the real-time requirements at all, or cannot meet the adaptive selection of defogging intensity. In order to solve this series of problems, the present invention designs a real-time high-definition video adaptive defogging device and method. Fog function.

The real-time high-definition video adaptive defogging method still adopts the idea of calculating the image transmittance and atmospheric light intensity frame by frame, but for real-time requirements, the atmospheric light intensity and transmittance calculated by the previous frame image are used to approximate the current frame image. Atmospheric light intensity and transmittance. Utilizing the parallel computing characteristics of FPGA, by designing a parallel computing method, the running speed of the computing defogging algorithm is improved to ensure real-time performance under high-definition video;

The research on haze removal method is one of the hotspots in the field of machine vision at present, which has attracted the attention of many research institutions at home and abroad; the document "Single Image Haze Removal Using DarkChannel Prior" published in the top international conference CVPR in 2009 carried out the research on the dehaze method. After research, good practical results have been achieved. However, when this method processes foggy images with a large area of sky, distortion occurs. This is because the sky area does not satisfy the dark channel prior. At the same time, the calculated transmittance Although the soft matting method can obtain a relatively fine transmittance map, the calculation is complex and cannot meet the real-time requirements. Being able to adapt to various natural scenes, automatically select the strength of the defogging algorithm, and meet the real-time requirements are very important to realize the real-time high-definition defogging function. Therefore, it is of great significance to study the adaptive defogging method and device for real-time high-definition video.

The Chinese patent document with publication number CN 107071353 A discloses an image defogging device. This method uses the TMS320DM6437 chip as the processor, and the TVP5150 analog-to-digital converter as the decoding chip, which can only process standard-definition image data, and for fog images of different intensities, only different defogging algorithms can be selected manually, and cannot be self-adapted Select a defogging algorithm.

The Chinese patent document with publication number CN 107194894 A discloses a video defogging method and system thereof. This method adopts the defogging method based on Dark Channel Prior, which can effectively defog the surveillance video, but cannot guarantee real-time performance.

Contents of the invention

The technical problem of the present invention is to overcome the deficiencies of the prior art and provide a real-time high-definition video self-adaptive defogging method and device to solve the image quality degradation, blurred image and poor visual effect of the image extracted by the existing real-time camera system Good and other issues.

The technical solution adopted by the present invention is: an image defogging device is invented, which includes a video image acquisition module, a video image defogging processing module, a communication interface module, and a video image encoding and display module.

The video image acquisition module includes a video signal receiving device and a video decoding chip. The video signal receiving device is generally a CCD high-definition camera, and the decoding chip is a GS2971A 3G-SDI video decoding chip.

The video image defogging processing module includes a processor and an external expansion connected thereto. The processor is an XC7K325T FPGA chip, and the external expansion is a DDR3, SRAM, FLASH, communication chip, etc. connected thereto.

The communication interface module includes a communication chip and a communication protocol. The communication chip is MAX3077E, and the communication protocol is an industrially mature RS422 bus.

The video image display module includes an encoding chip and a high-definition video image display, and the encoding chip uses a GS2972 digital-to-analog converter.

The real-time high-definition video adaptive defogging method based on the device comprises the following steps:

In the first step, for a pixel x, the degraded model of the original image J(x) is:

I(x)=J(x)t(x)+A(1-t(x))

where I(x) represents the foggy image, J(x) represents the original fog-free image, t(x) represents the transmittance, and A represents the atmospheric light intensity. The degree of image degradation is related to the distance. The long-distance degraded image can be regarded as a layer of uniform fog covering different local areas on the original clear image. Therefore, the image defogging algorithm usually assumes that the local depth of field of the image is the same, that is, t( x) is obtained through local image blocks, and the atmospheric light intensity A is constant.

In the second step, the solution process of J(x) at each pixel can be simplified:

in C=A(1-t(x)).

The third step is to use the dark channel prior algorithm to calculate the dark channel image, and select the original image brightness value corresponding to the position of the pixel value with the highest brightness of 0.1% in the dark channel as the estimated value of the atmospheric light intensity A;

Where I ^dark (x) represents the dark channel image, I ^c (y) represents each channel of the color image, Ω(x) represents the image block centered on pixel x, and c represents the r, g, b three colors of the image aisle. Indicates the pixel value with the highest brightness of 0.1% in the dark channel, I(x) indicates the original input image, and both A ^c and A indicate the intensity of atmospheric light;

The fourth step, through the atmospheric light intensity A calculated above, the estimation formula of the transmittance t(x) is:

Where ω=0.95, in real life, even in sunny days, there are some particles in the air, therefore, it is necessary to retain a certain degree of fog when defogging, which can introduce a value between [0,1] Factor ω.

The fifth step is to calculate the gear selection parameter α through the transmittance t(x) obtained above,

Among them, β ₁ and β ₂ represent the transmittance threshold, α ₁ , α ₂ , α ₃ are gear control parameters, and ||t(x)|| represents the transmittance 1 norm;

The sixth step is to obtain t(x), A and α through the above calculation, and use the algorithm of over-parameter suppression in the dehazing process, specifically:

Among them, x represents the position coordinates of pixels on the image, I(x) represents the foggy image, t(x) represents the transmittance, t ₀ represents the lower limit of transmittance, J(x) represents the fog-free image to be restored, and A represents the atmosphere Light intensity, A ₀ represents the upper limit of atmospheric light intensity, and parameter α is the selection parameter for the defogging gear; when the value of the projection map t(x) is small, the value of J(x) will be too large, so that the overall image is white , in order to avoid this problem, set a threshold t ₀ , when the value of t is less than t ₀ , use t ₀ for calculation; when the value of the atmospheric light intensity A is large, it will cause the processed image to have a color cast and a large number of colors Spot, in order to avoid this problem, set a threshold A ₀ , when the value of A is greater than A ₀ , use A ₀ for calculation.

Compared with prior art, the characteristics of the present invention are:

(1) The system is simple to implement, has good real-time performance, and can adaptively defog according to the fog concentration. Compared with the traditional single Gain/off method, the defogging ability has been improved. The main reasons are as follows: (1) Using a simplified calculation method, including a reasonable estimation algorithm for atmospheric light intensity and transmittance, can improve the defogging effect at the same time , reduce the amount of computation, and ensure the real-time performance of the dehazing algorithm. (2) Adopt the method of over-parameter suppression, increase A ₀ and t ₀ to perform over-parameter suppression on the calculated atmospheric light intensity and transmittance, prevent the image from being distorted such as white, color, and color spots, and make the image smoother , more naturally.

(2) Patent CN102017000379449 proposes a video defogging method and its system, but the hardware system used is not specified, so it is impossible to judge whether real-time image defogging operation can be realized. At the same time, the patent uses various approximations for the same parameter The solution method leads to a significant drop in solution accuracy and poor defogging effect; this solution uses FPGA as the main control chip, which can give full play to the advantages of parallel computing, and uses a small amount of reasonable approximation, which not only effectively reduces the amount of calculation, but also ensures the calculation accuracy and realization As a result, the defogging method and device of the present invention can realize real-time defogging of images with a resolution of 1080P.

Description of drawings

Fig. 1 is a flow chart of a real-time high-definition video adaptive defogging device of the present invention;

Fig. 2 is a flow chart of a real-time high-definition video adaptive defogging method of the present invention;

Fig. 3 is a simulation test diagram of a real-time high-definition video adaptive defogging method and device of the present invention, wherein a is a foggy image, and b is an image after a defogging operation.

Detailed ways

The specific implementation manner of the present invention will be described in further detail below in conjunction with the accompanying drawings.

Figure 1 is a flow chart of a real-time high-definition video adaptive defogging device of the present invention, which includes a video image acquisition module, a video image defogging processing module, a communication interface module, and a video image encoding and display module.

Described video image acquisition module comprises video signal receiving device, decoding chip, and video signal receiving device is generally CCD camera, and decoding chip selects GS2971A SDI signal decoding chip for use; Concrete process is that the image data collected by CCD camera passes through decoding chip GS2971A, will The input serial signal is converted into a parallel digital image stream, and the decoding chip GS2971A is connected to the XC7K325T chip;

Described video image defogging processing module comprises processor and the external expansion that is connected with it, processor selects the XC7K325T FPGA chip of Xilinx Company for use, external expansion is SRAM, DDR3, FLASH, communication chip etc. that are connected with it; Concrete process is through The digital image code stream processed by the decoding chip GS2971A is stored in DDR3, the video data is cached, and the parameters obtained through calculation are stored in SRAM. Program, communication chip realizes communication between XC7K325T FPGA chip and external computer;

The communication interface module includes a communication chip, the communication chip selects MAX3077E, and the communication protocol selects the very mature RS422 bus in the industry;

The video image display module includes an encoding chip and a video image display, and the encoding chip is GS2972. The specific process is that the digital image code stream processed by the above modules is converted into a serial video signal after being encoded by the encoding chip GS2972, and displayed on the video image display;

The above is the whole process of a real-time high-definition video adaptive defogging device of the present invention, but the above-mentioned implementation process is not intended to limit the present invention, and those skilled in the art can do it without departing from the present invention. Corresponding improvements have been made, and the protection scope of the present invention shall be determined by the scope defined in the claims.

As shown in Figure 2, it is a flow chart of a real-time high-definition video adaptive defogging method of the present invention, and the specific implementation steps are as follows:

(1) Convert the format of the input video image, from YUV422 format to RGB format, to facilitate subsequent data processing;

(2) Use the dark channel prior algorithm to calculate the dark channel

(3) By selecting the original image brightness value corresponding to the position of the highest 0.1% pixel value in the dark channel as the estimated value of the atmospheric light intensity A;

The formula for estimating the transmittance is

(4) Calculate the gear selection parameter α through the transmittance t(x) obtained above,

Among them, β ₁ and β ₂ represent the transmittance threshold, α ₁ , α ₂ , α ₃ are gear control parameters, and ||t(x)|| represents the transmittance 1 norm;

(5) Obtain t(x), A and α through the above calculation, and use the algorithm of over-parameter suppression in the dehazing process, specifically:

Among them, x represents the position coordinates of pixels on the image, I(x) represents the foggy image, t(x) represents the transmittance, t ₀ represents the lower limit of transmittance, J(x) represents the fog-free image to be restored, and A represents the atmosphere Light intensity, A ₀ represents the upper limit of atmospheric light intensity, and parameter α is the selection parameter for the defogging gear; when the value of the projection map t(x) is small, the value of J(x) will be too large, so that the overall image is white , in order to avoid this problem, set a threshold t ₀ , when the value of t is less than t ₀ , use t ₀ for calculation; when the value of the atmospheric light intensity A is large, it will cause the processed image to have a color cast and a large number of colors Spot, in order to avoid this problem, set a threshold A ₀ , when the value of A is greater than A ₀ , use A ₀ for calculation.

As shown in Figure 3, it is a simulation test diagram of a real-time high-definition video adaptive defogging device and method of the present invention, wherein a is a real foggy image taken in the air, and b is a picture through the defogging device and defogging device of the present invention Image after fog processing.

The contents not described in detail in the description of the present invention belong to the prior art known to those skilled in the art.

The above embodiments are provided only for the purpose of describing the present invention, not to limit the scope of the present invention. The scope of the invention is defined by the appended claims. Various equivalent replacements and modifications made without departing from the spirit and principle of the present invention shall fall within the scope of the present invention.

Claims (7)

1. A real-time high-definition video adaptive defogging method is characterized in that: the realization steps are as follows: (1) The brightness value of the original image corresponding to the pixel value with the highest brightness of 0.1% in the dark channel in the collected image is used as the estimated value of the atmospheric light intensity A, and the estimation formula of the transmittance t(x) is: <mrow><mi>t</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>=</mo><mn>1</mn><mo>-</mo><mi>&amp;omega;</mi><munder><mrow><mi>m</mi><mi>i</mi><mi>n</mi></mrow><mrow><mi>y</mi><mo>&amp;Element;</mo><mi>&amp;Omega;</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow></mrow></munder><mrow><mo>(</mo><munder><mrow><mi>m</mi><mi>i</mi><mi>n</mi></mrow><mi>c</mi></munder><mfrac><mrow><msup><mi>I</mi><mi>c</mi></msup><mrow><mo>(</mo><mi>y</mi><mo>)</mo></mrow></mrow><msup><mi>A</mi><mi>c</mi></msup></mfrac><mo>)</mo></mrow></mrow> Where ω=0.95, x, y represent the coordinates of pixel points on the image, I ^c (y) represents each channel of the color image, Ω(x) represents the image block centered on pixel x, and c represents the r, g of the image ,b The three color channels A ^c and A both represent the intensity of atmospheric light; (2) According to the obtained transmittance t(x), calculate the defogging gear selection parameter α, <mrow><mi>&amp;alpha;</mi><mo>=</mo><mfenced open = "{" close = ""><mtable><mtr><mtd><msub><mi>&amp;alpha;</mi><mn>1</mn></msub></mtd><mtd><mrow><mo>|</mo><mo>|</mo><mi>t</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>|</mo><mo>|</mo><mo>&amp;le;</mo><msub><mi>&amp;beta;</mi><mn>1</mn></msub></mrow></mtd></mtr><mtr><mtd><msub><mi>&amp;alpha;</mi><mn>2</mn></msub></mtd><mtd><mrow><msub><mi>&amp;beta;</mi><mn>1</mn></msub><mo>&amp;le;</mo><mo>|</mo><mo>|</mo><mi>t</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>|</mo><mo>|</mo><mo>&amp;le;</mo><msub><mi>&amp;beta;</mi><mn>2</mn></msub></mrow></mtd></mtr><mtr><mtd><msub><mi>&amp;alpha;</mi><mn>3</mn></msub></mtd><mtd><mrow><msub><mi>&amp;beta;</mi><mn>2</mn></msub><mo>&amp;le;</mo><mo>|</mo><mo>|</mo><mi>t</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>|</mo><mo>|</mo></mrow></mtd></mtr></mtable></mfenced></mrow> Among them, β ₁ and β ₂ represent the transmittance threshold, α ₁ , α ₂ , α ₃ are gear control parameters, and ||t(x)|| represents the transmittance 1 norm; (3) Through t(x), A and α, the algorithm of over-parameter suppression is used in the dehazing process to obtain the fog-free image J(x), the formula is <mrow><mi>J</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>=</mo><mfrac><mrow><mi>I</mi><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow><mo>-</mo><mi>&amp;alpha;</mi><mi>min</mi><mrow><mo>(</mo><mi>A</mi><mo>,</mo><msub><mi>A</mi><mn>0</mn></msub><mo>)</mo></mrow></mrow><mrow><mi>max</mi><mrow><mo>(</mo><mi>t</mi><mo>(</mo><mi>x</mi><mo>)</mo><mo>,</mo><msub><mi>t</mi><mn>0</mn></msub><mo>)</mo></mrow></mrow></mfrac><mo>+</mo><mi>m</mi><mi>i</mi><mi>n</mi><mrow><mo>(</mo><mi>A</mi><mo>,</mo><msub><mi>A</mi><mn>0</mn></msub><mo>)</mo></mrow></mrow> Among them, I(x) represents the foggy image, t ₀ represents the lower limit of the transmittance, J(x) represents the haze-free image to be restored, A ₀ represents the upper limit of the atmospheric light intensity, and the parameter α is the selection parameter of the defogging gear. 2. the real-time high-definition video adaptive defogging method according to claim 1, is characterized in that: when the value of described t(x) is very small, when t(x)<0.1, can cause the value of J(x) is too large, so that the overall image is white, in order to avoid this problem, set a threshold t ₀ =0.1, when the value of t is less than t ₀ , use t ₀ for calculation; when the value of the atmospheric light intensity A is large, that is When A>220, it will cause color cast and a lot of color spots in the processed image, set a threshold A ₀ , when the A value is greater than A ₀ , use A ₀ for calculation. 3. A real-time high-definition video adaptive defogging device, characterized in that: it includes a video image acquisition module, a video image defogging processing module, a communication interface module, a video image encoding and display module; the video image acquisition module includes a video signal A receiving device and a decoding chip, the video image processing module includes a signal processor and an external expansion connected thereto, the communication interface module includes a main control chip and an external expansion connected thereto, and the video image encoding and display module includes Encoding chip, video image display. 4. A real-time high-definition video adaptive defogging device according to claim 3, characterized in that: the video signal receiving device is a CCD camera to realize real-time video information collection, and the decoding chip is GS2971A to realize real-time High-definition video signal decoding function, the decoded image signal is transmitted to the video processing module. 5. A kind of real-time high-definition video adaptive defogging device according to claim 3, is characterized in that: described video image processing module comprises processor and the external expansion that is connected with it, and described processor selects XC7K325T FPGA chip for use, Realize high-definition video adaptive defogging algorithm and calculation of fog penetration gear, the external expansion is DDR3, SRAM, FLASH, communication chip connected to it, wherein DDR3 and SRAM are used for image data cache, FLASH is used for storing program data , the communication chip is used for the main program to interact with the outside through the serial port. 6. A real-time high-definition video adaptive defogging device according to claim 3, characterized in that: the video image coding and display module includes a coding chip and a high-definition video image display, and the coding chip is GS2972 for use in The parallel image data is converted into serial data for easy transmission, and the high-definition video image display can be a general-purpose SDI high-definition video display for real-time display of the processed video. 7. A real-time high-definition video adaptive defogging device according to claim 3, characterized in that: the communication interface module includes a communication chip and a communication protocol, the communication chip is MAX3077E, and the communication protocol is very mature in industry The RS422 bus realizes the communication function between the main program and the external computer.