KR20020078610A - Segmentation method for interesting object in natural infrared images - Google Patents

Abstract

PURPOSE: A method of separating an object of concern in an infrared image is provided to separate the object of concern from the infrared image rapidly and effectively by using both of fuzzy image separation method and edge detection method. CONSTITUTION: An area of concern including an object of concern is selected from an infrared image, a reference brightness and a reference position of the area of concern are obtained. Fuzzy thresholding is performed for the selected area of concern to separate the background and the object of concern from each other(240). For the selected area of concern, an expanded image and an eroded image are generated. The two images are compared with pixel values of the original image to be replaced with pixel values similar to the pixel values of the original image, to create a contrast expanded image(300). The edge is detected from the contrast expanded image(400). The fuzzy-thresholded image and the image from which the edge was detected are ORed(500).

Description

Segmentation of interest in infrared images {SEGMENTATION METHOD FOR INTERESTING OBJECT IN NATURAL INFRARED IMAGES}

The present invention relates to a method for processing an infrared image, and more particularly, by combining a new fuzzy image segmentation method based on fuzzy theory and a conventional canny edge detection method when dividing a background and an object in an infrared natural image. The present invention relates to a method of segmenting an object of interest in an infrared image that can detect a boundary of an object of interest.

In general, in image processing technology, image segmentation is a technology for dividing and segmenting an object region of interest and a background region in an image, which is the most basic and important process in many image processing applications.

In particular, the infrared image is obtained by detecting a microwave proportional to heat radiated by a distant object through a sensor and converting the microwave into an image. FIG. 1 shows an example of an infrared image.

As shown in FIG. 1, since the target object is directly exposed to the climate and environment in the infrared image, there is a heat exchange between the object and the surroundings, and the contrast between the object and the background is due to scattering and absorption of light in the atmosphere. Lowers.

Also, there are many unwanted clutters in the background.

For this reason, the infrared image blurs the boundary of the object, and various brightness values coexist in the object area. Therefore, it is difficult to separate only the object region of interest from the actual natural image.

However, in the case of the Automatic Target Recognition (ATR) system where many infrared images are actually applied, the accurate separation of targets (targets) is the key to directly determining the performance of the system.

Therefore, many researches have been conducted on image processing in an infrared image, in particular, a method of accurately dividing an object of interest from a background. Among the most recently used methods, an object using only brightness value information in an infrared image is used. There is a threading technique of partitioning.

However, as mentioned above, since several brightness values exist simultaneously within one object, when using a thread-solving method using only brightness value information, one object is divided into several objects or an object part. There is a problem that the background part is often separated.

Therefore, it is not easy to divide the background and the object in the infrared image only by the conventional threading method.

The present invention has been proposed to solve the above-described problems, and its object is to apply a fuzzy image segmentation method and a conventional edge detection method together in an infrared image to quickly and effectively separate an object of interest from an infrared image. It is to provide a segmentation method of interest.

1 is a photograph showing a representative example of an infrared natural image.

2 is a process diagram sequentially illustrating a method of segmenting an object of interest in the infrared region in accordance with the present invention.

3 is a schematic view for explaining a method of selecting a region of interest in the division method according to the present invention.

4A is a photograph showing a result of the ROI selection process in the division method according to the present invention.

Figure 4b is a photograph showing the result of the fuzzy threading processing in the partitioning method according to the present invention.

FIG. 5A is a photograph showing a general infrared image, and FIG. 5B is a histogram of the infrared image of FIG. 5A.

6A is a photograph of a contrast-enhanced image during the segmentation process according to the present invention, and FIG. 6B is a histogram for the image of FIG. 6A.

7 is a photograph showing a result of the division process according to the present invention.

The present invention as a means for achieving the above object, the method of dividing the object of interest in the infrared image

(A) selecting a region of interest including an object of interest in the infrared image and obtaining reference brightness and a reference position of the region of interest;

(B) separating the background and the object of interest by fuzzy threading the selected region of interest;

(C) creating an expanded image and an eroded image with respect to the selected region of interest, and then comparing the two images with pixel values of the original image and replacing the two images with a near pixel value to create a contrast-extended image;

(D) detecting an edge in the contrast-enhanced image; And

(E) binary post-combining the fuzzy threaded image and the edge-detected image and performing post-processing.

Hereinafter, a method of dividing an object of interest in an infrared image according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a flowchart sequentially illustrating a segmentation method in an infrared image according to the present invention.

Referring to the flowchart of FIG. 2, the process of segmentation of interest in the infrared image according to the present invention will be described sequentially.

(A) Defining a region of interest (100)

First, a region of interest is selected from the input infrared natural image.

As shown in FIG. 1, since the background of the infrared natural image occupies more than half of the entire image, it is essential to remove the complex background portion and separate only the region of interest in which the object exists.

The present invention satisfies the following Equation 1 when h ( g ) is the frequency at the brightness value g in the histogram when the information on the minimum and maximum object sizes is given prior knowledge to define the region of interest. Get the threshold that is the maximum t value.

In Equation 1, M is the width of the image, N is the height of the image, Xmin and Ymin is the width and height of the minimum object, respectively. After such thresholding, only eight areas are selected by applying eight connectivity component labeling. This region is the reference region Rr on which the region of interest is selected.

The ROI is defined as shown in FIG. 3 by using the reference region obtained as described above and the maximum object size ( Xmax , Ymax ).

That is, the reference brightness value gr and the reference position (x ', y') are defined in the reference region Rr having Nr pixels. In this case, the reference brightness value g _r is defined as in Equation 2 below and represents an average brightness value in the reference area.

The position (x ', y') of the reference pixel represents the center of gravity of the reference region Rr and is defined as in Equation 3 below.

4A is a photograph showing a result of selecting a region of interest for an object of interest in the infrared natural image shown in FIG.

(B) Fuzzy Thresholding Step (200)

The fuzzy thrashing step 200 is a pixel clustering process, in which the most suitable threshold value is determined in the membership plane so that the pixel of the object and the pixel of the background are well separated.

The present invention proposes a new threshold selection method using both brightness information and spatial information of an image. The fuzzy thresholding process is performed as follows.

(a) First, similarity and proximity are calculated (step 210).

In the above description, the similarity (similarity) indicates how similar the pixel g _xy is to the reference brightness value, and the similarity S is intuitively dependent on the difference in the brightness value, so that the similarity is large when the difference in the brightness value is small, on the contrary, the brightness value. If the difference is large, it is defined as in Equation 4 to reduce the similarity.

Where gr is a reference brightness value and C is a constant representing the variation of the brightness value difference. (Wherein g _min is the minimum brightness value) and 0 ≦ S ( g _xy , g _r ) ≦ 1.

In Equation 4, the reference brightness value g _r is defined as in Equation 2 described above.

It can be said that the similarity with respect to any pixel thus obtained is likely to be classified as the object of interest. However, if only the similarity is used, clutters that are similar to the object's brightness value and near the object's boundary may be classified as objects, and pixels with low brightness values within the object may not be selected as the object to separate the object. Can also be.

Therefore, in the present invention, the adjacency is calculated to use the position information of the pixel together with the similarity. The adjacency A is a reference position of the pixel P at the position (x, y) It is defined as shown in Equation 5 below.

Where 0 ≦ A (P, P ′) ≦ 1. And, when the position P of one pixel is (x, y) and the reference position P 'is (x', y '), the distance d (P, P') is Is defined as

(b) When the similarity and the proximity are defined as described above, a membership value is next calculated (220). It first uses the following equation (6) to find the membership plane.

Here, 0 ≦ μ ( g _xy ) ≦ 1, and β is a constant for differently assigning the weights of the similarity S and the proximity A, and as an example, β = 0.4.

Next, in order to obtain a histogram of membership values in the ROI, the membership plane is converted into a membership plane having an integer value using Equation 7 below.

here, Is a stop value with an integer value, and int [ x ] takes the integer value closest to the real value x. C ₁ represents the variation width of the integer membership value. In this embodiment, C ₁ = 100, which has little effect on the image segmentation result.

(c) Then determine a final threshold for the membership plane (step 230).

this is When the frequency of integer membership values in the reference region is expressed, the maximum frequency value Mr is obtained by the following equation (8).

As shown in FIG. 3, the region excluding the reference region Rr from the region of interest is referred to as Rb. When denotes the frequency of the integer membership value, the maximum frequency value Mb is calculated as shown in Equation 9 below.

As above, Is a frequency of integer membership values for the region of interest, the final threshold is given by Equation 10 below.

That is, the final threshold value T _f is obtained by the above equation (10).

(d) then the object pixelq _O Background pixel withq _b To classify the threshold value T for the ROI, as shown in Equation 11 below._fPerform thread soldering using.

Where q represents a pixel of the ROI.

The photograph of FIG. 4B is a result of the fuzzy threshold processing described above for the infrared natural image shown in FIG. 1, and it can be seen that the object of interest and the background are correctly classified.

(C) Expansion of contrast (step 300)

As described above, fuzzy threading is a powerful means but not sufficient for object segmentation in infrared images. It is often difficult to include a cloth as part of an object in an object such as a tank shown in FIG. This is because the brightness value of the gun is often lower than the brightness value of the object body, and the location of the gun is generally far from the reference position. Therefore, in the present invention, at the same time as the fuzzy thresholding process, using the Canny edge detection method to compensate for the shortage of the thresholding. The reason why this method is specifically selected among the various edge detection methods is that the non-maximal suppression function of the Canny edge detection method is effective in eliminating the background noise stuck to the object boundary.

The canny edge detection method is a conventional method. However, in the present invention, as a preprocessing of edge detection, a new contrast extension process is added to make the edge stronger in the selected region of interest.

A histogram of a general infrared natural image as shown in FIG. 5A is shown in FIG. 5B. As can be seen from FIG. 5A, when the boundary region of the object is seen in a general natural image, blurring is severe, and in particular, the difference in brightness value from the background is small in the area of the cloth. In addition, the histogram has a constant shape except for a dark background region as shown in FIG. 5B.

Therefore, contrast enhancement processing is added for the purpose of hardening the edges and flattening the histogram. This contrast enhancement process is based on the gray morphology using a 9 x 9 window. That is, after the image created by the expansion and the image produced by the erosion is made, the pixel value of the original image is compared with which of the pixel values of the two images is close and replaced with the nearest pixel value. This is expressed by the following equation (12).

Here, fd (x, y) represents an expanded image, f e (x, y) represents an eroded image, and fnew (x, y) represents a newly generated contrast-extended image.

FIG. 6A is an image after contrast-expanded general infrared natural image as shown in FIG. 5A. As you can see in the picture, the brighter parts of the original image are made brighter and the darker parts are made darker, making the edges stronger. In addition, FIG. 6B is a histogram of the contrast-expanded infrared image shown in FIG. 6A, and as shown, it can be seen that the histogram becomes flat as a result of histogram smoothing.

(D) Edge Detection (Step 400)

Then, the edge is detected by the Canny edge method in the contrast-enhanced image as described above.

In general, the canny edge method is realized in three stages: Derivative Of Gaussian (DOG), non-maximal suppression, and hysteresis thresholding. When the Canny edge method is applied to the contrast-extended image, the Gaussian filter has a standard deviation of 1.0, a high threshold of 0.75, and a low threshold of 1.0. This allows all pixels with a threshold lower than the high threshold to be considered not all edges. These parameters do not have a significant impact on the results of the segmentation, but for infrared imaging of military equipment that requires accurate analysis of the target, the values given were optimal.

(E) Combination and post-processing of the two methods (step 500 to step 700)

As above, the binary fuzzy threading result and the edge detection result are combined for the region of interest by performing a binary OR (step 500). Then, the combined object and the surrounding noises are present, in which component labeling is performed to select only the object region (step 600). Then, the final image segmentation result is obtained by performing binary closing as a post-process. The structure element in the closing process is a 3 x 3 window. 7 shows the final image segmentation result processed as described above. As shown, it can be seen that the overall contour of the tank, including the artillery far from the reference position and having a lower brightness value than the reference, is clearly distinguished.

As described above, the present invention has an excellent effect of accurately and quickly separating the object area and the background area from the infrared natural image input from the infrared sensor or the infrared camera by combining fuzzy thrusting and edge detection. It can be used for object tracking or object recognition technology using infrared rays, which has an excellent effect to further improve the function. In particular, it can be widely used in the field of automatic tracking and automatic recognition of military equipment which is equipped with infrared sensor for night observation, and can be widely used in infrared surveillance system in civil field to improve performance.

Claims (8)

In the segmentation method of interest in the infrared image, (A) selecting a region of interest including an object of interest in the infrared image and obtaining reference brightness and a reference position of the region of interest; (B) separating the background and the object of interest by fuzzy threading the selected region of interest; (C) creating an expanded image and an eroded image with respect to the selected region of interest, and then comparing the two images with pixel values of the original image and replacing the two images with a near pixel value to create a contrast-extended image; (D) detecting an edge in the contrast-enhanced image; And And (e) binary post-combining the fuzzy thresholded image and the edge-detected image to perform post-processing. The method of claim 1, wherein the selecting of the region of interest comprises: When the information on the minimum and maximum object sizes is given prior knowledge, when h ( g ) is the frequency at the brightness value g in the histogram, obtaining the maximum threshold value by the following equation, and threading; Where M is the image width, N is the image height, and Xmin and Ymin are the minimum object width and height, respectively. Applying maximum connectivity component labeling to select a maximum area and setting the reference area as Rr; And And a region of interest defined in the infrared image by expanding in four directions from the reference position of the reference region Rr by the size of the maximum object ( Xmax , Ymax ). The method of claim 1, wherein the step (b) of purging (a) Similarity S with reference brightness for each pixel in the selected region of interest Where gr is the reference brightness value and C is the constant representing the variation (Wherein g _min is the minimum brightness value), 0 ≦ S ( g _xy , g _r ) ≦ 1, and g _r is the reference brightness value; (b) the proximity A to the reference position P 'for each pixel position P in the selected region of interest. (here, D (P, P ') is the distance between two positions); (c) Membership Plan using the similarity and adjacency (Where 0 ≦ μ ( g _xy ) ≦ 1 and β is a constant for differently assigning weights of similarity S and proximity A); (d) the membership plane obtained in step (c) (here, Is a stop value with an integer value, int [ x ] is an integer value closest to the real value x, and C ₁ is the variation width of the integer membership value). Converting to; (e) obtaining a frequency of integer membership values in the region of interest to obtain a final threshold value T _f ; And and dividing the object pixel and the background pixel into the membership plane using the final threshold value T _f . 4. The distance d (P, P ') according to claim 3, wherein the pixel position P is (x, y) and the reference position P' is (x ', y') in the step (b) of obtaining the adjacency. ) Method of segmentation of interest in the infrared image, characterized in that defined as. 4. The method of claim 3, wherein β = 0.4 in the step (c). 4. The method of claim 3, wherein in step (d), C ₁ = 100. 4. The method of claim 3, wherein the step (e) of obtaining the final threshold value T _f is Is the frequency of integer membership values in the reference region, Obtaining by; The region of interest except for the reference region is called Rb, Is the integer membership value frequency in the region Rb, the maximum frequency value Mb Obtaining by; And The maximum frequency values Mr and Mb of the reference region Rr and the region Rb excluding the reference region in the ROI are expressed as And obtaining a final threshold value T _f by substituting for. The method of claim 1, wherein the generating of the contrast-expanded image (c) When the image of the expanded region of interest is called f _d (x, y) and the image of the eroded region of interest is called f _e (x, y), And converting the original shape f (x, y) into a new image f _new (x, y).