KR102261532B1 - Method and system for image dehazing using single scale image fusion - Google Patents

Abstract

The present invention relates to a system for removing fog based on mono-scale image fusion and a method thereof. According to the present invention, the method includes: a preprocessing step of performing preprocessing with a detailed emphasis algorithm to restore edge information for gamma correction; an under-exposure image acquisition gamma correction step of acquiring an under-exposure image by performing gamma correction with the image emphasized in detail; a normalization step of calculating and normalizing a weighted value to prevent an out-of-range issue which can occur during image fusion; a mono-scale image fusion step of performing a weighted sum of a virtual under-exposure image and a corresponding weighted value; and a post-processing step of improving a brightness channel in an adaptive tone remapping manner with respect to the fused image and emphasizing a color difference channel in accordance with the degree of the improvement. Accordingly, the present invention can achieve better performance with a considerably small amount of operations than a comparison algorithm but also embody small and fast hardware for real-time processing, thereby providing an effect of enabling a graft with an autonomous driving vehicle or a vision system such as an intelligent monitoring camera.

Description

{Method and system for image dehazing using single scale image fusion}

The present invention relates to a fog removal system and method based on single-scale image fusion, and more specifically, detail enhancement, gamma correction, single-scale image fusion, adaptive tone remapping, and a simple image quality improvement method A fog removal system and method based on single-scale image fusion for maintaining an effective amount of computation to achieve fog removal performance by using .

Conventional image processing algorithms are designed on the assumption of pleasant weather conditions, but in reality, when taking pictures or moving pictures outdoors, weather conditions such as fog may be adverse. Fog removal algorithms have been developed for clear image quality, and can be largely divided into single image and multiple images. Although the latter has excellent performance, it is not widely used due to very complex input image acquisition. The single input image method has received a lot of scientific attention, and it includes the image restoration viewpoint and the image quality improvement viewpoint.

First, considering the image restoration perspective, based on atmospheric scattering, the generation of fog images is represented by a physical model, and a clear image can be obtained by inverting this model. Equation 1 below represents a physical model called Koschmieder.

where x is the pixel position and I, J, A, and t are the input fog image, the resulting image (fog removed), atmospheric light, and the transmission map, in that order. Since the only known variable is I, we need estimates of A and t to restore J. For this reason, there is a problem with the conditional restrictions on fog removal. In the US patented method, He et al. proposed the famous Dark Channel Prior (DCP) for estimating the fog transfer map. Although DCP shows good performance for indoor fog removal, as mentioned earlier, the sky area is darkened during fog removal.

In the reference literature, Zhu et al. estimated the depth of an image by applying a machine learning technique called supervised learning. Since the transfer map is exponentially proportional to the depth, the transfer map can be estimated given the depth. For atmospheric intensity estimation, the method with a large computational amount of the two algorithms is used. Assuming that there are atmospheric intensity pixels in the transfer map or a predetermined ratio of depth, the pixel with the highest intensity among those pixels is set as the atmospheric intensity. This causes the hassle of having to align all the pixels and rearranging them even at a predetermined ratio.

Second, in terms of image quality improvement, a simple image quality improvement technique is used so that the image to be restored (input image) is similar to the original clear image without considering the physical cause of the distorted portion of the input image. Examples are low-light image quality improvement, detail emphasis, homomorphic filtering, and histogram equalization. There is a limitation in that these methods are effective only when the fog is light. So the fog removal method based on image fusion begins to be used. In the reference, Galdran proposed a method according to multi-scale fusion, and the weights consisted of contrast and saturation. Good performance was obtained because the image characteristics related to fog are taken into account, but there is a disadvantage in that the amount of computation or memory demand is large. Because the image fusion process combines many images into one image according to weight, the amount of computation is quite large. In addition, if the multi-scale method is used, the number of images to be combined increases, which increases the amount of computation and the memory demand.

In conclusion, the fog removal method using a multi-input image shows excellent performance, but is not practically utilized due to very complex input acquisition. In addition, the method using a single input image has a disadvantage in that the amount of computation is large because the atmospheric intensity and the transfer map are estimated. In addition, the single image fog removal method using image fusion has a disadvantage in that the amount of computation and memory demand are large due to the multi-scale method such as the Laplacian pyramid.

To summarize again, FIG. 1 is a classification diagram that summarizes the fog removal methods in patents and papers to date. In the proposed method, fog removal is performed as an image quality improvement method using a single image. And since it does not use a physical formula in terms of image restoration, which is generally used frequently, fog removal is performed by generating an under-exposure image through gamma correction. Therefore, it belongs to the viewpoint of image quality improvement.

As the fog removal algorithm divides the input image according to single image and multiple images, and multiple images are very difficult to acquire, researchers are very interested in the former instead of the latter. Regarding single image fogging, there are image restoration and image quality improvement approaches. The first approach shows good performance because it is based on a physical model called Koschmieder, but has a high amount of computation due to the ill-posed problem of the model. In the case of the second image quality improvement approach, the calculation process is simple and the processing speed is fast. However, since fog-related distortion is not considered or some methods have disadvantages of using a high amount of computation or a large memory, the algorithm proposed in the present invention follows the image quality improvement approach and compensates for the disadvantages of using a high amount of computation and a large memory. This enables excellent fog removal performance and real-time processing.

US Patent Publication No. 8340461 (2012.12.25.) "Single Image Haze Removal Using Dark Channel Priors" Republic of Korea Patent Publication No. 10-1997866 (2019.10.08.) "Method of removing fog components in a single image"

Qingsong Zhu, Jiaming Mai, Ling Shao, “A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior”, IEEE Transactions on Image Processing (2015), Vol. 24, pp. 3522-3533 Adrian Galdran, “Image dehazing by artificial multiple-exposure image fusion”, Signal Processing (2018), Vol. 149, pp. 135-147

The present invention is to solve the above problems, and to achieve good performance and maintain a fairly small amount of computation in order to compensate for two disadvantages (image restoration point of view, image quality improvement point of view) of the conventional fog removal algorithm. An object of the present invention is to provide a fog removal system and method based on single-scale image fusion.

In addition, the present invention provides a fog removal system based on single-scale image fusion to use a method of merging virtual under-exposure images in order to prevent the problem of large computational amount due to atmospheric intensity and transfer map estimation. and methods.

In addition, the present invention provides a fog removal system based on single-scale image fusion to provide a real-time processing function as well as reducing the amount of computation through post-processing called adaptive tone remapping for an image darkened during fog removal. to provide a way.

However, the objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the fog removal method based on single-scale image fusion according to an embodiment of the present invention is a pre-processing step with a detailed enhancement algorithm in order to save edge information for gamma correction, and gamma correction using the detailed enhancement image Gamma correction step to obtain an under-exposure image by gamma correction, a normalization step to calculate and normalize weights to prevent out-of-range problems that may occur during image fusion, and virtual exposure A single-scale image fusion step that performs a weighted sum of the under-exposure image and the corresponding weight, and improves the brightness channel with the adaptive tone remapping method for the fused image. It may then be characterized by including a post-processing step of emphasizing the chrominance channel according to the degree of improvement.

In this case, the pre-processing step is a step of restoring the distorted edges due to fog, converting the RGB → YCbCr color space to emphasize the details only on the brightness channel, extracting details using the Laplacian operator, the image It may be characterized in that it comprises the steps of generating a weight according to the amount of regional variation of , emphasizing details according to the weight, and converting YCbCr → RGB color space.

In addition, in the gamma correction step, a virtual under-exposure image is generated, but it is necessary to determine how many virtual under-exposure images are to be generated, and gamma correction is performed for each of the images. can be characterized as

In addition, in the normalization step, a weight is generated using an index called DCP (Dark Channel Prior) indicating fog, and since the DCP (Dark Channel Prior) index is inversely proportional to the fog distribution, it must be inverted to generate weights. Instead, it may be characterized in that the weights are normalized to prevent an out of range problem when performing image fusion.

In addition, the single-scale image fusion step combines the clear regions of the virtual under-exposure images to create one of the de-fog images, and weights the virtual under-exposure images and corresponding weights. It may be characterized in that it is performed in a single scale method of performing the sum.

In addition, in the post-processing step, since the fused image becomes dark due to the use of virtual under-exposure images and the disadvantages of DCP (Dark Channel Prior), in order to improve the image quality of the de-fog image, RGB → convert YCbCr color space, obtain brightness channel and color difference channel, improve picture quality on brightness channel, emphasize color difference according to the improved ratio, and finally convert YCbCr → RGB color space for output to display device can be characterized as

In order to achieve the above object, a fog removal system based on single-scale image fusion according to an embodiment of the present invention includes a pre-processing module 110 for performing pre-processing with a detailed enhancement algorithm in order to save edge information for gamma correction; a gamma correction module 120 that performs gamma correction to obtain an under-exposure image by using the detailed-enhanced image; a normalization module 130 for calculating and normalizing weights in order to prevent an out of range problem that may occur during image fusion; a fusion module 140 that performs single-scale image fusion that performs a weighted sum of a virtual under-exposure image and a corresponding weight; and a post-processing module 150 for improving a brightness channel in an adaptive tone remapping method on the fused image, and then performing a post-processing process of emphasizing a color difference channel according to the degree of improvement; It may be characterized in that it comprises a.

The fog removal system and method based on single-scale image fusion according to an embodiment of the present invention not only achieves better performance than the comparison algorithm with a significantly smaller amount of computation, but also can be implemented with small and fast hardware for real-time processing, so autonomous driving It can provide an effect that can be applied to a vision system such as a car or an intelligent surveillance camera.

1 is a classification diagram that summarizes the fog removal methods in patents and papers to date.
2 is a diagram illustrating a fog removal system 100 based on single-scale image fusion according to an embodiment of the present invention.
3 is a flowchart illustrating a fog removal method based on single-scale image fusion according to the present invention.
4 is a diagram illustrating an overall algorithm of a fog removal method based on single-scale image fusion according to the present invention.
5 shows the light scattering phenomenon to explain the basis of the algorithm proposed in the present invention.
6 is a diagram specifically illustrating a detailed highlighting process among the fog removal methods based on single-scale image fusion according to the present invention.
7 is a graph illustrating weights according to regional variations during a detailed emphasis process by the preprocessing module 110 in the fog removal method based on single-scale image fusion according to the present invention.
8 is a diagram specifically illustrating a gamma correction process in the fog removal method based on single-scale image fusion according to the present invention.
FIG. 9A is a diagram illustrating a detailed highlighted image, a gamma correction graph, and an under-exposure image to which gamma correction is applied for explaining the effect of the gamma correction process of FIG. 8 .
9B is a diagram illustrating a result of the single-scale image fusion process of FIG. 11 .
10 is a diagram specifically illustrating a weight calculation and normalization process for image fusion among the fog removal methods based on single-scale image fusion according to the present invention.
11 is a diagram specifically illustrating a single-scale image fusion process among the fog removal methods based on single-scale image fusion according to the present invention.
12 is a diagram specifically illustrating a post-processing process called adaptive tone remapping among the fog removal methods based on single-scale image fusion according to the present invention.
13 is a view showing the results of visual evaluation by the fog removal method based on single-scale image fusion according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, detailed description of preferred embodiments of the present invention will be described with reference to the accompanying drawings. In the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

In the present specification, when one component 'transmits' data or signal to another component, the component may directly transmit the data or signal to another component, and through at least one other component This means that data or signals can be transmitted to other components.

2 is a diagram illustrating a fog removal system 100 based on single-scale image fusion according to an embodiment of the present invention. Referring to FIG. 2 , the fog removal system 100 based on single-scale image fusion includes an input device unit 100a, a control unit unit 100b, and a display unit unit 100c, and the control unit unit 100b includes: A preprocessing module 110 that pre-processes with a detailed emphasis method, a gamma correction module 120 that performs gamma correction to obtain an under-exposure image, and a method that calculates and normalizes weights based on DCP (Dark Channel Prior) Post-processing is performed with the normalization module 130, the fusion module 140 that merges the virtual under-exposure image and the corresponding weight according to a single scale fusion method, and adaptive tone remapping. It may include a post-processing module 150 to

With this configuration, the fog removal system 100 based on single-scale image fusion maintains a fairly small amount of computation by using simple image processing techniques (detail enhancement, gamma correction, single-scale fusion, Adaptive Tone Remapping, etc.) and efficient image fusion And by achieving good fog removal performance due to the novel pre- and post-processing steps, it is possible to implement real-time processing of data and a small hardware device.

Therefore, the algorithm and hardware device used in the fog removal system 100 based on single-scale image fusion according to the present invention can be utilized in systems such as autonomous vehicles and intelligent surveillance cameras.

Hereinafter, a detailed embodiment of the fog removal method based on the fog removal system 100 based on single-scale image fusion according to the present invention will be described. Meanwhile, FIG. 3 is a flowchart illustrating a fog removal method based on single-scale image fusion according to the present invention. 4 is a diagram illustrating an overall algorithm of a fog removal method based on single-scale image fusion according to the present invention. 5 shows the light scattering phenomenon to explain the basis of the algorithm proposed in the present invention.

First, referring to FIG. 5 , a phenomenon in which visibility of an image acquired by a camera is reduced in foggy weather will be described. The light wave reflected by the object scatters with atmospheric particles and enters the aperture of the camera's lens. Because this scattering is based on the wavelength of light, older physical de-fog techniques have controlled the lens aperture and shutter speed. Because the effect of fog increases the intensity of bright objects, the physical intervention of the camera limits the amount of light reaching the image sensor. Adjusting the lens aperture and shutter speed has the same effect on the overall image, and by limiting the light through the lens aperture, information in dark areas is lost and bright areas become sharper. Therefore, the fog removal effect can be achieved by fusion of the obtained images after performing this measure (aka, under-exposure) several times.

In the algorithm proposed in the present invention, no physical action should be taken, so an under-exposure method is used through detailed emphasis and gamma correction. This process is called virtual under-exposure.

When the RGB image obtained by the digital sensor corresponding to the input device 100a is given to the control device 100b, the proposed algorithm is as follows step by step.

First, the [detailed emphasis] process by the preprocessing module 110 is performed (S100), and the detailed emphasizing process consists of a plurality of processes as shown in FIG. 6 .

First, in order to remove the fog, the pre-processing module 110 performs a pre-processing with a detailed emphasis algorithm. The reason for performing the pre-processing with a detailed emphasis is to preserve more edge information than when performing gamma correction.

In general, taking physical measures, such as adjusting the lens aperture and shutter speed, can restore the bright edges that are lost in the fog. However, gamma correction is performed to indicate under-exposure, as in Galdran's method in the reference. If gamma correction is performed, it is possible to restore the part that is slightly distorted by the fog, but the edge does not disappear. Therefore, in the present invention, detailed emphasis is performed as a pre-processing step before applying gamma correction in order to restore an edge that does not disappear even if it is heavily distorted.

Because the human eye is more sensitive to changes in brightness instead of changes in chrominance, we apply detail highlighting only to the brightness channel. Therefore, color space conversion steps (RGB→YCbCr, YCbCr→RGB) are required at both ends. To extract edge information, Laplacian operator is used. Next, in order to emphasize the extracted details, weights are generated according to the local variance of image, the details are emphasized according to the generated weights, and a detailed example will be described below.

In the first step (S101) of detail enhancement, the preprocessing module 110 performs color space conversion (RGB → YCbCr) on the RGB image according to Equation 2 below.

Here, R, G, and B are Red, Green, and Blue, respectively, and Cb and Cr are color differences.

In the second step S102 of emphasizing details, the preprocessing module 110 may extract details from the result Y of the color space conversion using Equation 3 below and a Laplacian operator.

where E represents the detailed extraction value, * represents the convolution operation, and h _v =[-1, 2, -1] ^T , h _v =[-1, 2, -1] is the vertical axis and the This is the horizontal Laplacian operator. T stands for Transpose.

In the third step S103 of emphasizing details, the pre-processing module 110 may generate a weight according to the amount of regional variation of the image according to Equation 4 below.

는 지역의 평균 값을 구하는 연산이다. 그리고

는 x를 중심으로 하는 일정 범위 안의 pixel들의 집합(local pathc)을

는 밝기를 의미한다. 또한

는 영상의 지역 변동량에 따른 가중치이고,

는 사용자 정의 변수이다. 제안한 가중치는 piecewise linear 형태를 갖으며

범위에 선형적으로 변화하는데 그 범위 밖에는 상수

로 제한된다. where v is the local variation of the image and y is the pixel position in a rectangular window centered at the x position.

is an operation to find the average value of a region. And

is a set of pixels within a certain range centered on x (local pathc).

means brightness. Also

is the weight according to the local variation of the image,

is a user-defined variable. The proposed weights have a piecewise linear form.

It varies linearly with the range, but outside that range is a constant

is limited to

범위에 가중치를 선형적으로 감소를 시킨다.As shown in FIG. 7 , a large weight is assigned to a small amount of variation indicated for an area in which a lot of detailed information has disappeared, and a small weight is allocated to a large amount of variance indicated for an area in which detailed information has hardly disappeared. In addition, in order to prevent visual problems caused by rapid conversion,

Decrease the weights linearly over the range.

In the fourth step ( S104 ) of detail emphasis, the preprocessing module 110 performs detailed emphasis according to the weight according to Equation 5 below.

는 가중치에 따른 세부강조 값,

는 밝기,

는 영상의 지역 변동량에 따른 가중치,

는 세부추출 값을 의미한다.here,

is the detailed emphasis value according to the weight,

is the brightness,

is the weight according to the regional variation of the image,

is the detailed extraction value.

In the fifth step (S105) of detail enhancement, the preprocessing module 110 may perform color space conversion (YCbCr→RGB) according to Equation 6 below.

는 세부강조된 R값,

는 세부강조된 G값,

는 세부강조된 B값,

와

는 색차를 의미한다.here,

is the detailed highlighted R value,

is the G-value with detailed emphasis,

is the value of detailed emphasis B,

Wow

stands for color difference.

Next, the [gamma correction] process ( S200 ) by the gamma correction module 120 will be described.

8 is a diagram specifically illustrating a gamma correction process for virtual under-exposure. The gamma correction module 120 performs gamma correction using the image highlighted in detail in step S100. An under-exposure image is acquired, and FIG. 9A is a diagram illustrating a detailed highlighted image, a gamma correction graph, and an under-exposure image to which gamma correction is applied.

Here, the number of virtual under-exposure images is experimentally determined according to an algorithm result value, and then gamma correction expressed as a power function is performed on each image. In addition, for the efficient hardware device of the algorithm proposed in the present invention, the power function is implemented as a look-up table, and a detailed embodiment will be described below.

In the first step of gamma correction ( S201 ), the gamma correction module 120 determines the number of virtual under-exposure images.

을 받고 네 개의 영상

을 생성한다.In this step, the gamma correction module 120 must determine how many virtual under-exposure images to generate through gamma correction. In the case of the present invention, four were experimentally set through numerical analysis of the results. Therefore, the detailed highlighted image

getting four images

create

In the second step of gamma correction ( S202 ), the gamma correction module 120 performs gamma correction using a look-up table using Equation 7 below.

는 감마,

는 개수를 의미한다.From here

is gamma,

means the number.

를 주소로 사용하여 원하는 출력 데이터(

)를 얻는다.There are several methods for hardware implementation of exponents, such as the Taylor series or look-up table. In the former case, the latency is rather large, but the gamma variable can be adjusted. In the latter case, the latency takes only one clock cycle, but if it is implemented so that the gamma variable can be adjusted, the demand for hardware increases. In the present invention, a gamma variable is set in advance by conducting a lot of experiments with several image data sets and an implementation method is selected using a look-up table. Calculate the above gamma correction formula in advance and save it as a table. then input data

to the desired output data (

) to get

Next, [weight generation and normalization] by the normalization module 130 will be described. 10 is a diagram illustrating a weight calculation and normalization process for image fusion.

Referring to FIG. 10 , the fog removal method through image fusion creates one clear image by fusing clear areas (parts without fog) of virtual under-exposure images.

For this reason, an indicator called Dark Channel Prior (DCP) that accurately shows fog is used to indicate a clear area. The smaller the fog, the smaller the DCP value, and the calculated DCP is negatively transformed to be used as a weight for fusion. Therefore, the smaller the fog, the greater the weight. Finally, in order to prevent an out-of-range problem that may occur during image fusion, the weights should be normalized, and a detailed embodiment will be described below.

As a first step of weight generation and normalization, the normalization module 130 performs Dark Channel Prior (DCP) calculation and inversion by using Equation 8 below.

는 영상

의 dark channel이다. 중괄호 안의 min 연산은 색 채널(R,G,B)의 세 개의 최솟값을 구하는 연산이고 중괄호 밖의 min 연산은 필터링 윈도우의 최솟값을 구하는 연산이고,

는 x를 중심으로 하는 일정 범위 안의 pixel들의 집합(local pathc)를 의미한다. here

the video

is the dark channel of The min operation inside the braces is an operation to find the minimum of three color channels (R, G, B), and the min operation outside the braces is an operation to find the minimum value of the filtering window,

denotes a set of pixels within a certain range centered on x (local pathc).

를 반전시킨다. 예를 들어, 정규화된 데이터를 사용하는 경우는 1에서

를 빼면 되고 일반적으로 [0, 255] 8bit 데이터 범위를 사용하는 경우는 255에서

를 빼면 된다.

는 DCP에 기반을 둔 가중치를 의미한다.next

invert the For example, when using normalized data, from 1 to

In general, [0, 255] is 255 when using an 8-bit data range.

can be subtracted from

is a weight based on DCP.

As a second step of weight generation and normalization, the normalization module 130 performs weight normalization according to Equation 9 below.

는 가중치 정규화를 의미한다.here

stands for weight normalization.

An out of range can occur when merging four virtual under-exposure images. Therefore, such a problem can be prevented by normalizing the weights according to the above formula.

The fusion module 140 performs [single-scale image fusion]. 11 is a diagram illustrating a single-scale image fusion process for achieving a fog removal effect.

Referring to FIG. 11 , most of the fog removal algorithms based on other image fusion are multi-scale methods called Laplacian pyramids. This fusion method can have a good effect on the image quality, but the memory demand is high. This is because frame buffer memory is required whenever an image is down-scaled or up-scaled. Therefore, the algorithm proposed in the present invention uses only a single-scale fusion scheme and applies post-processing called adaptive tone remapping to improve image quality. A single scale image fusion is just a weighted sum of the virtual under-exposure image and its weights. If there are four virtual under-exposure images, the corresponding weights are also four, and a detailed embodiment will be described below.

The fusion module 140 performs [single-scale image fusion] according to Equation 10 below.

는 단일 스케일 영상 융합을 통한 안개 제거된 영상,

는 가중치 정규화,

는 세부 강조된 under-exposure 영상을 의미한다. From here

is the de-fog image through single-scale image fusion,

is weight normalized,

is a detailed under-exposure image.

Single-scale image fusion is a weighted sum of a virtual under-exposure image and its weights. Here, J is the de-fog image.

Meanwhile, FIG. 9B is an image that has undergone a single-scale image fusion process. The fog has been removed, but the overall darkness can be seen. Accordingly, finally, the visibility of the dark image is improved by using the adaptive tone remapping algorithm previously developed by the post-processing module 150 .

The post-processing module 150 performs [post-processing] by adaptive tone remapping (S500).

12 is a diagram illustrating a post-processing process called adaptive tone remapping. Referring to FIG. 12 , the output image is darkened because fog is removed by fusing virtual under-exposure images. In addition, since the weights for image fusion are based on the DCP method, there is a disadvantage in that the sky area becomes dark when fog is removed, which is a disadvantage of DCP. To compensate for these problems, post-processing is applied. After the adaptive tone remapping method improves the brightness channel, the chrominance channel is emphasized according to the degree of improvement, and a specific embodiment will be described.

As a first step (S501) of the post-processing, the post-processing module 150 performs color space conversion (RGB → YCbCr).

More specifically, the post-processing module 150 may use the formula introduced in the detailed emphasis part to convert the color space. However, here the post-processing module 150 uses YCbCr 4:2:2 instead of the standard YCbCr 4:4:4. Therefore, the post-processing module 150 obtains a brightness channel (L) and a chrominance channel (C) after converting the color space.

In the second step (S502) of the post-processing, the post-processing module 150 performs image quality improvement on the brightness channel according to Equation 11 below.

은 밝기 채널이고,

는 밝기 계수이고

는 적응적인 밝기 가중치이다. 입력 밝기의 히스크램을 사용하여 CDF(cumulative distribution function)를 계산한 후 ALP(adaptive limit point)를 구한다.

는 ALP에 따른 비선형 거듭제곱 함수이고

는 간단한 선형 함수이다.where EL is the improved brightness,

is the brightness channel,

is the brightness coefficient

is the adaptive brightness weight. After calculating the cumulative distribution function (CDF) using the histogram of the input brightness, the adaptive limit point (ALP) is obtained.

is a nonlinear power function according to ALP and

is a simple linear function.

In the third step ( S503 ) of the post-processing, the post-processing module 150 performs color difference enhancement according to the improvement of the brightness channel according to Equation 12 below.

는 색차 채널이고,

는 색차 계수이고

는 적응적인 색차 가중치이다.

를 입력 색차(C)와 밝기 향상 정도

의 곱으로 정의하였다.

는 세부 강조 부분에서 소개한 가중치와 비슷하게 piecewise linear 함수로 설계하였다.where EC is the accentuated color difference,

is the color difference channel,

is the chrominance coefficient

is the adaptive color difference weight.

Enter the color difference (C) and the degree of brightness enhancement

It is defined as the product of

is designed as a piecewise linear function similar to the weights introduced in the detailed emphasis section.

As a fourth step (S504) of the post-processing, the post-processing module 150 performs color space conversion (YCbCr → RGB). That is, the post-processing module 150 converts the RGB color space to output to the display device.

In order to verify the superior performance of the proposed algorithm, it is compared with other algorithms. The fog removal method used for comparative purposes is the method of He et al. registered in the US patent, and the method proposed by Zhu et al. and Galdran mentioned in References. Here, the result of the visual evaluation is shown in FIG. 13 .

In order to evaluate the proposed algorithm, representative papers on fog removal were compared with He's DCP method and Zhu's CAP method. In the DCP result, it can be confirmed that the color of the sky part is distorted or the image becomes dark when the fog is removed because the atmospheric intensity is incorrectly estimated (red square box). In addition, in the Zhu method, it can be confirmed that color distortion occurs in the rocky part of the mountain when the fog is removed by overestimating the amount of transmission in the physical fog removal formula as the fog removal from the viewpoint of image restoration (blue square box). On the other hand, in the proposed method, it is possible to check the image in which the fog is effectively removed without color distortion in the sky part and the rock part part.

Meanwhile, Verilog language (IEEE Standard 1364-2005) is used to implement the hardware device of the proposed algorithm. The hardware synthesis results are shown in Table 1 below.

Xilinx Design Analyzer Device xc7z045 - 2ffg900 Design - Proposed Algorithm Slice Logic Utilization Available Used Utilization Slice Registers (#) 437,200 30,705 7% Slice LUTs (#) 218,600 35,950 16% RAM36E1s / FIFO36E1s 545 48 8% Minimum Period (ns) - 3,992 Maximum Frequency (MHz) - 250,501

For hardware design verification, Xilinx's xc7z045 - 2ffg900 board was used, and 7% and 16% of slice resistor and sline LUT were used, respectively. And the maximum operating frequency achieved 250.501MHz.

Compare with other fog removal hardware to show the advantages of the designed hardware. Park et al. implemented the fog removal method of He et al. and improved this process to reduce the computational amount of atmospheric intensity estimation. In addition, the hardware implementation of the domestic patented fog removal algorithm is also considered. Referring to the table below, the hardware device introduced in the present invention may exhibit a smaller hardware resource demand as well as a faster processing speed.

The present invention can also be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored.

Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and is also implemented in the form of a carrier wave (eg, transmission through the Internet). also includes

In addition, the computer-readable recording medium is distributed in a computer system connected through a network, so that the computer-readable code can be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the technical field to which the present invention pertains.

As described above, preferred embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms are used, these are only used in a general sense to easily explain the technical content of the present invention and help the understanding of the present invention. , it is not intended to limit the scope of the present invention. It is apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

100: Fog removal system based on single-scale image fusion
100a: input device unit
100b: control unit
100c: display device unit
110: preprocessing module
120: gamma correction module
130: regularization module
140: fusion module
150: post-processing module

Claims (7)

When performing gamma correction, the image is pre-processed with a detailed enhancement algorithm to preserve edge information, but converting RGB → YCbCr color space to perform detailed enhancement only on the brightness channel to restore the distorted edges due to fog, Laplacian ( Laplacian) operator, generating a weight according to the local variation of the image, emphasizing the detail according to the weight, and a preprocessing step including converting YCbCr → RGB color space;
Create a virtual under-exposure image, determine how many virtual under-exposure images to generate, and, for each of the created images, use a detailed-emphasized image a gamma correction step for obtaining an under-exposure image by performing gamma correction;
Since the DCP (Dark Channel Prior) indicator representing fog is inversely proportional to the fog distribution, the calculated DCP is negatively transformed to generate weights, and out-of-range problems that may occur during image fusion are prevented. a normalization step of performing normalization by calculating the weights;
A weighted sum of the virtual under-exposure image and the corresponding weight is performed, but the clear area of the virtual under-exposure image is combined to generate one of the de-fog images, a single-scale image fusion step of performing a weighted sum of a virtual under-exposure image and a corresponding weight; and;
For the fused image, the brightness channel is improved by the adaptive tone remapping method, and the color difference channel is emphasized according to the degree of improvement, but the virtual under-exposure image and the DCP (Dark Channel Priority) ) the so the image is dark, the fusion based, an improved rate to improve the picture quality of the removed fog image, converts the RGB → YCbCr color space to obtain the brightness of a channel and a chrominance channel, and then subjected to image quality improvement in the brightness of a channel a post-processing step of emphasizing the color difference according to YCbCr → RGB color space for output to a display device;
A method of removing fog based on single-scale image fusion, comprising:
delete delete delete delete delete Preprocess the image with a detailed enhancement algorithm to save edge information for gamma correction, convert the RGB → YCbCr color space, extract details with the Laplacian operator, and create a weight according to the local variation of the image according to the weight a pre-processing module 110 that emphasizes details and performs pre-processing by converting YCbCr → RGB color space;
Create a virtual under-exposure image, determine how many virtual under-exposure images to generate, and expose each of the created images using an image with detailed emphasis a gamma correction module 120 that performs gamma correction to obtain an under-exposure image;
Since the DCP (Dark Channel Prior) indicator representing fog is inversely proportional to the fog distribution, the calculated DCP is negatively transformed to generate weights, and out-of-range problems that may occur during image fusion are prevented. a normalization module 130 for performing normalization by calculating the weights;
A weighted sum of the virtual under-exposure image and the corresponding weight is performed, but the clear area of the virtual under-exposure image is combined to generate one of the de-fog images, a single-scale image fusion module 140 for performing a weighted sum of a virtual under-exposure image and a corresponding weight; and
For the fused image, the brightness channel is improved by the adaptive tone remapping method, and the color difference channel is emphasized according to the degree of improvement, but in order to improve the image quality of the de-fogged image, RGB → YCbCr color space , obtain a brightness channel and a chrominance channel, improve the image quality on the brightness channel, emphasize the color difference according to the improved ratio, and finally perform a post-processing process of converting YCbCr → RGB color space for output to a display device. Post-processing module 150 to perform;
A fog removal system based on single-scale image fusion, comprising a.

Images