KR102187634B1 - Apparatus for removing noise for infrared image - Google Patents

Abstract

The present invention relates to a noise removal apparatus for an infrared image, and more particularly, to remove noise from an infrared image based on a local filter and global optimization. According to the present invention, the accuracy of image recognition, region segmentation, object detection, etc. is improved by removing noise included in the background while preserving contour or structure information including important information of an infrared image using a contour preserving flattening filter. To be able to.

Description

Noise reduction device for infrared image {APPARATUS FOR REMOVING NOISE FOR INFRARED IMAGE}

The present invention relates to a noise removal device for an infrared image, and more particularly, to a noise removal device for an infrared image capable of removing noise while preserving contour or structure information of an infrared image using a contour preserving flattening filter. About.

Infrared imaging systems are recently being used in various fields. In general visible (visible) imaging systems, infrared imaging systems are used for various purposes in environments where proper image information cannot be obtained. Fields of application include military surveillance and reconnaissance systems, non-destructive inspection equipment for industrial facilities, medical inspection and diagnostic equipment, and fire department equipment for lifesaving, and are recently used in the field of night object recognition in the automotive field. As the application fields of infrared imaging systems are expanded, interest in preserving or improving image quality is increasing.

In general, infrared images have problems such as low signal-to-noise ratio and contrast ratio, blurred outlines, loss of detailed information of images, and background noise. In order to use it, it is necessary to improve image clarity and remove noise.

Among image processing technologies, image smoothing removes insignificant details or textures while preserving important edge information and structure information from an image.This image flattening technology In the image processing field, it has been used in various fields such as detail improvement, noise removal, image extraction, and image recognition.

However, the conventional image flattening technique has a problem in that the contour information of the image is also removed while noise is removed.

Accordingly, there is a need to develop a noise reduction technique that can remove noise included in a background, while preserving contour or structure information including important information of an infrared image.

Accordingly, the present invention has been proposed in order to solve the above-described problems, by removing noise included in the background, while preserving the outline or structure information including important information of the infrared image by using an outline preserving flattening filter, An object of the present invention is to provide a method and apparatus for removing noise for an infrared image that can improve the accuracy of image recognition, region segmentation, and object detection.

In addition, the present invention is to provide a noise removal method and apparatus for an infrared image capable of clearly emphasizing the detail of an infrared image without a halo phenomenon and a noise amplification phenomenon occurring when the detail component of an infrared image is emphasized.

Objects of the present invention are not limited to those mentioned above, and other objects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

A noise removal apparatus for an infrared image according to the present invention for achieving the above object removes noise from an infrared image based on a local filter and global optimization.

In addition, the apparatus for removing noise for an infrared image according to the present invention includes: an input unit receiving a collected infrared image; A preprocessor for preprocessing the input infrared image; A filter unit for flattening the preprocessed infrared image based on the local filter; A global optimization unit for performing global optimization on the infrared image flattened by the filter unit to perform flattening; And an output unit for outputting a final globally optimized infrared image by the global optimization unit.

In addition, the noise removal apparatus for an infrared image according to the present invention removes noise from an infrared image based on a contour preserving flattening algorithm in which a local filter and a global optimization are fused.

According to the present invention, the accuracy of image recognition, region segmentation, object detection, etc. is improved by removing noise included in the background while preserving contour or structure information including important information of an infrared image using a contour preserving flattening filter. To be able to.

In addition, according to the present invention, it is possible to clearly emphasize details of an infrared image without a halo phenomenon and noise amplification that occur when the detail component of an infrared image is emphasized.

The effects of the present invention are not limited to those mentioned above, and other effects that are not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

1 is a block diagram showing the configuration of a noise removal apparatus according to an embodiment of the present invention.
2 is a flow chart illustrating a noise removal method according to an embodiment of the present invention.
3 and 4 are diagrams showing experimental results for comparing planarization performance according to an embodiment of the present invention.
5 is a view showing a result of emphasizing image detail based on RoGIF according to an embodiment of the present invention.
6 is a diagram showing experimental results for comparing detail enhancement performance according to an embodiment of the present invention.

Objects and effects of the present invention, and technical configurations for achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. In describing the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of donation in the present invention, which may vary according to the intention or custom of users or operators.

However, the present invention is not limited to the embodiments disclosed below, and may be implemented in various different forms. These embodiments are provided only to make the disclosure of the present invention complete, and to fully inform the scope of the invention to those skilled in the art to which the present invention pertains, and the present invention is defined by the scope of the claims. It just becomes. Therefore, the definition should be made based on the contents throughout this specification.

On the other hand, in the entire specification, when a part is said to be "connected" with another part, it is not only "directly connected", but also "indirectly connected" with another member interposed therebetween. Include. In addition, when a part "includes" a certain component, it means that other components may be further provided, rather than excluding other components unless specifically stated to the contrary.

Edge Preserving Smoothing Filter is the most effective technology that can remove noise while preserving contour information. It is possible to improve the accuracy of image recognition, region segmentation, object detection, etc. by preserving contours or structure information including important information of an image and removing noise included in the background.

Contour-preserving flattening filters can be roughly divided into two ways.

First, there is a kernel-based local smoothing filter technology. Representative examples include a bilateral filter and a guided image filter. Local flattening is a method of flattening while preserving the outline of an image within a window.

The second is Global Optimization, and representative examples include Weighted Least Squares (WLS), Total Variation (TV), and Relative Total Variation (RTV). Global optimization is a method of flattening while preserving the outline by finding the optimal value by reflecting the correlation between the pixel value of the ROI and the entire image.

In other words, conventionally, in order to preserve the contour of an infrared image and remove noise, an algorithm for flattening using a local filter and an algorithm for flattening through global optimization have been used as a contour preserving flattening algorithm.

In the present invention, noise of an infrared image is removed based on an algorithm having a method similar to RoG (Relativity of Gaussian) as a method of fusing a local filter and a global optimization. Specifically, the image details are further preserved by using Guided Image Filtering (GIF) instead of the Gaussian kernel, and noise of the image is removed by performing global optimization.

Hereinafter, a method and apparatus for removing noise from an infrared image according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a noise canceling apparatus according to an embodiment of the present invention.

Referring to FIG. 1, a noise removal apparatus 10 according to an exemplary embodiment of the present invention includes an input unit 11, a preprocessor 13, a noise removal unit 15, and an output unit 17.

The input unit 11 inputs an infrared image collected based on an infrared camera or an infrared sensor to the preprocessor 13.

The preprocessor 13 performs preprocessing using the infrared image input from the input unit 11.

The noise removal unit 15 removes noise by performing flattening on the infrared image preprocessed by the preprocessing unit 13 based on a contour preserving flattening algorithm.

At this time, as a contour preserving flattening algorithm, RoGIF (Relativity of Guided Image Filter), a method similar to RoG, is used as a method of fusing local filter and global optimization. Preservation and global optimization to remove image noise.

To this end, the noise removal unit 15 includes a filter unit 151 and a global optimization unit 153.

를 이용하여 하기와 같은 <수학식 1>로 정의한다.As described above, RoGIF uses different normalization parameters in a similar way to RoG.

It is defined as <Equation 1> as follows by using.

<Equation 1>

₁,

₂는 정규화 파라미터 및 윈도우 크기를 RoGIF 파라미터로 표기한 것이고, GIF는 윈도우

_k의 크기와 정규화 파라미터

를 통해 평탄화 효과가 달라진다. 이 중 윈도우 크기는 정규화 파라미터

에 비해 평탄화 효과에 큰 영향을 주지 않기 때문에 윈도우 반지름 r을 2로 고정하고 정규화 파라미터만 변경하였다. GIF는 정규화 파라미터

이 커질수록 평탄화 효과가 더 크게 발생한다. here,

₁ ,

₂ is the normalization parameter and window size expressed as RoGIF parameters, and GIF is the window size.

size and normalization parameters of _k

Through this, the flattening effect is different. Of these, the window size is a normalization parameter

Compared to, the window radius r was fixed at 2 and only the normalization parameter was changed because it did not significantly affect the smoothing effect. GIF is the normalization parameter

The larger is, the greater the flattening effect occurs.

는 평탄화 효과가 작아 미세한 윤곽선을 보존하면서 평탄화한 결과이고,

는 평탄화 효과가 커 강한 윤곽선을 보존하면서 평탄화한 것이다.Meanwhile,

Is the result of flattening while preserving fine outlines due to the small flattening effect,

Is flattened while preserving strong outlines due to its great flattening effect.

That is, the filter unit 151 has a local filter applied and performs local normalization using GIF to obtain contour information at different scales through Equation 1 above.

Meanwhile, the global optimization unit 153 finds an optimum value by reflecting the correlation with the entire image of the RoGIF defined by the filter unit 151 and removes noise while preserving the outline.

_x,y는 하기 <수학식 2>로 정의할 수 있고, 글로벌 최적화 결과는 하기 <수학식 3>을 반복적 재 가중치화 된 최소 자승법으로 해를 구한다.In this case, the nonlinear weight

_x,y can be defined by the following <Equation 2>, and the global optimization result is obtained by using the following <Equation 3> repeatedly reweighted least squares method.

<Equation 2>

<Equation 3>

은 양의 값을 갖는 파라미터이다. 한편,

의 L2 데이터 신뢰도(data fidelity)는 평탄화 결과 S와 입력 영상 I의 거리를 최소화 하는 것이다.Here, I is the input image, S is the flattened result image,

Is a positive parameter. Meanwhile,

The L2 data fidelity of is to minimize the distance between the flattened result S and the input image I.

That is, the global optimizer 153 assumes an initial value S ⁰ of the flattening result image as I, obtains a GIF for the gradient value of the input image using Equation 2, and calculates a weight value, and then After repeating K times using Equation 3>, a flattening result image is obtained.

The output unit 17 outputs an infrared image from which noise has been removed according to the result of the final global optimization by the global optimization unit 153.

2 is a flow chart showing a noise removal method according to an embodiment of the present invention.

First, the input unit 11 inputs an infrared image collected based on an infrared camera or an infrared sensor to the preprocessor 13 (S201), and the preprocessor 13 uses the infrared image input from the input unit 11 Then, pre-processing is performed (S203).

Thereafter, the filter unit 151 first inputs the infrared image collected based on an infrared camera or an infrared sensor to the preprocessor 13 (S201), and the preprocessor ( 13) performs pre-processing using the infrared image input from the input unit 11 (S203).

Thereafter, the filter unit 151 performs local normalization on the preprocessed infrared image using GIF, and performs flattening by extracting contour information at different scales (S205).

The global optimizer 153 obtains a weight value by obtaining a GIF for the gradient value of the infrared image input in step S201, and performs flattening by repeating the global optimization K times (S207).

The infrared image from which noise has been removed is finally output through steps S205 and S207 (S209).

3 and 4 are diagrams showing experimental results for comparing planarization performance according to an embodiment of the present invention.

In order to compare the planarization performance according to an embodiment of the present invention, each image result was derived by adding Gaussian noise to an infrared camera image of 1280x1024 size and applying Bitonic, GIF, RTV, WLS, RoG, and RoGIF.

In order to quantitatively analyze the contour-preserving flattening results, the peak signal-to-noise ratio (PSNR) and the Structural Similarity Index (SSIM) values were compared with the original image and each result image of contour-preserving flattening. Here, PSNR and SSIM are the same as the following <Equation 4> and <Equation 5>, respectively.

<Equation 4>

<Equation 5>

It can be determined that the higher the PSNR and SSIM indices, the more effective the noise removal was while the contour was preserved.

Table 1 below shows the PSNR and SSIM values calculated for FIGS. 3 and 4.

<Table 1>

3 and 4, (a) is an original image, (b) is an image to which Gaussian noise is added, (b) of the existing contour preserving flattening method using an image, (c) is Bitonic, and (d) is GIF, (e) represents RTV, (f) represents WLS, (g) represents RoG, and (h) represents the result of RoGIF.

In addition to the quantitative values shown in <Table 1>, when viewing through the flattening result image, it can be seen that the contour-preserving flattening technique according to the present invention in Figs. 3 and 4 has better preserved the contour information and removed noise compared to the existing techniques. have.

5 is a diagram illustrating a result of emphasizing image detail based on RoGIF according to an embodiment of the present invention.

One of the examples of using contour preservation flattening is to emphasize the detail component of an image. In general, a method of saving the detail component of an image is a method of clearly emphasizing the outline of the image. When the detail component is emphasized by using the outline, there is a problem that a halo artifact and a noise component are amplified. In order to solve such a problem, a technique that emphasizes only the contour while reducing the noise component is required.

를 사용하여 (a), (b)와 같이 각각 콜스 디테일(coarse detail) 및 파인 디테일(fine detail)을 강조한 결과를 이용하여 최종 디테일 개선된 결과를 (c)와 같이 얻을 수 있다.Referring to FIG. 5, different parameters for emphasizing image detail based on RoGIF according to an embodiment of the present invention

Using the result of emphasizing coarse detail and fine detail, respectively, as in (a) and (b), a final detail-improved result can be obtained as shown in (c).

In FIG. 5, it can be seen that (a) emphasizes a thick outline in the image, and (b) emphasizes a detailed outline in the image.

6 is a diagram showing experimental results for comparing detail enhancement performance according to an embodiment of the present invention.

Referring to FIG. 6, (a), (e), and (i) are results of the Bitonic filter, and (a) are enlarged results of the resulting images. It can be seen that blurring of the image occurs due to the noise component in the rice field and the halo phenomenon from the branches under the road. In addition, (b), (f), and (j) are the result images of RTV. It can be seen that the halo phenomenon is more clearly visible than the Bitonic filter. On the other hand, (c), (g), and (k) are RoG results, which show relatively less halo and noise amplification, but compared to the results of RoGIF, it is confirmed that detail components are unclear in green houses and branches I can.

Finally, (d), (h), and (l) clearly emphasize the detail component of the image as the result of RoGIF, and it can be confirmed that the halo phenomenon and noise amplification phenomenon are less than that of the existing methods. have.

As described above, according to the present invention, it can be seen that noise is effectively removed while preserving the outline, and it can be considered to be superior in quantitative evaluation. In addition, the effect of clearly emphasizing the detail of the image was confirmed without the halo phenomenon and noise amplification that occur when emphasizing the detail component of the image. Compared with existing research methods, it can be confirmed that the quantitative evaluation and image improvement are excellent.

In the present specification and drawings, a preferred embodiment of the present invention has been disclosed, and although specific terms are used, these are merely used in a general meaning to easily explain the technical content of the present invention and to aid understanding of the present invention. It is not intended to limit the scope. It is obvious to those of ordinary skill in the art that other modifications based on the technical idea of the present invention may be implemented in addition to the embodiments disclosed herein.

Claims (3)

In the noise reduction device for infrared image,
An input unit receiving the collected infrared image;
A preprocessor for preprocessing the input infrared image;
A filter unit for flattening the preprocessed infrared image based on the local filter;
A global optimization unit that removes noise from the infrared image by performing global optimization on the infrared image flattened by the filter unit; And
And an output unit for outputting a final globally optimized infrared image by the global optimization unit,
The filter unit,
For the infrared image, local normalization is performed using a Relativity of Guided Image Filter (RoGIF) without using a Gaussian filter, and contour information at different scales is extracted through Equation 6 below,
The global optimization unit,
Flattening is performed by repeating the global optimization K times,
The RoGIF is defined by <Equation 6>,
<Equation 6>

The ε ₁ and ε ₂ represent the normalization parameter and the window size as a parameter of the RoGIF,
The GIF in Equation 6 is a window

The flattening effect varies depending on the size of _k and the normalization parameter ε,
Reminder window

The size of _k is fixed to 2, and only the normalization parameter ε is changed,
remind

Represents the result of flattening while preserving fine outlines due to the small flattening effect,

Represents the result of flattening while preserving strong outlines due to a large flattening effect,
The global optimization unit removes noise while preserving an outline by finding an optimal value by reflecting the correlation with the entire image of the RoGIF defined by the filter unit,
Nonlinear weight

_x,y are defined by the following <Equation 7>, and the result of the global optimization is calculated using the iteratively reweighted least squares method as shown in the following <Equation 8>,
<Equation 7>

<Equation 8>

I is an input image, S is a flattened result image, and

Is a positive parameter, wherein

L2 data fidelity of minimizes the distance between the flattening result S and the input image I,
Calculate a Peak Signal-to-Noise Ratio (PSNR) between the input infrared image and the infrared image to which the RoGIF is applied,
The noise removal apparatus for an infrared image, characterized in that the PSNR of the infrared image to which the RoGIF is applied is higher than a reference value. delete delete

Images