KR20120000732A - An automatic segmentation method for object-based analysis using high resolution satellite imagery - Google Patents

Abstract

PURPOSE: An automatic video allocation method is provided to create an effective allocation image by executing analysis based on the object of a high resolution satellite image. CONSTITUTION: A multi-spectral image(10) and a high resolution black and white image(20) are processed through an image convergence process in order to improve the spatial resolution of the multi-spectral image and the high resolution black and white image(100). An initial seed point is created(200). An initial image is allocated using the extracted seed point(300). Area mergence is executed in order to improve an initial image allocation result(400).

Description

Automatic segmentation method for object-based analysis using high resolution satellite imagery

The present invention relates to an automatic image segmentation method for object-based analysis using high resolution satellite images, and more particularly, to an object using high resolution satellite images, which can generate an effective segmented image for object-based analysis of various high resolution satellite images. An automatic image segmentation method for based analysis.

In general, as the spatial resolution of an image increases, even if the same object exhibits various spectral characteristics and spatially different shapes, existing medium and low resolution image processing techniques are required to extract meaningful information from high resolution satellite images. It is true that the direct application of the pixel-based technique, which is widely used as a technique, is unreasonable. To compensate for this, prior to classification, the pixels having the same characteristics are divided into object or segment units through the image segmentation process, and object-based classification is performed in consideration of the characteristic information of these objects. based) is emerging as a classification method suitable for high resolution images. In the object-based classification method, the image segmentation process is a very important preprocessing process in the object-based classification because the accuracy of the object information generated through the image segmentation process directly affects the classification result.

Image segmentation refers to the task of separating a given image into a set of components or objects in the image. Each region must satisfy homogeneity and connectivity, and the sum of all divided regions constitutes the entire image. Image segmentation can be divided into histogram-based, edge-based, and region-based approaches. Histogram-based techniques are not suitable for application to satellite images with high image complexity. In addition, edge-based methods basically use only the edge information of the image, which requires complicated post-processing to maintain the connectivity of the region. The image segmentation algorithm applied to the satellite image is mainly based on the area-based method. Among them, the watershed segmentation algorithm is actively applied to the satellite image. However, because the method using the watershed segmentation algorithm basically uses only the edge information of the image, the accuracy of segmentation is not only sensitive to the edge information used but also does not consider the multispectral information of the satellite image due to the characteristics of the algorithm.

Referring to the satellite image segmentation method provided by most applications in the field of remote sensing in the related art, the image is displayed on the screen, and the seed is given to the object of interest depending on the visual judgment of the analyst to expand the area. It provides an outgoing semi-automatic technique and a simple method of creating a histogram on a given image and the user segmenting the image by determining thresholding.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a satellite image segmentation methodology using various characteristic information of the image for the effective use of high resolution satellite image.

In particular, the present invention integrates various characteristics information (spectral, edge, texture) of the input image in the image segmentation step to minimize the influence on the segmentation result according to the selection and method of the initial seed point in the region-based image segmentation method. To solve this problem, over-segmentation of initial image segmentation is solved.

To achieve these and other advantages and in accordance with the purpose of the present invention,

(a) The multispectral image and the black and white image provided from the satellite are input to the main memory, and the production of the high resolution multispectral image, which is a fusion image using the two images, is performed using the Gram-Schmidt technique. Performing using a program in memory;

(b) The improvement of the fusion image of step (a) is performed by using a program in main memory including a modified maximum homogeneity neighbor technique to consider multidimensional data, and extracting entropy-based multispectral edges. Edge information combining all band information of multispectral image is extracted by using operator, and texture information is obtained by using program in main memory including Block Difference of Inverse Probabilities algorithm. Extracting from;

(c) The homogeneity index generated by combining the improved fusion image and the edge image which passed through step (b) into blocks of a user-specified size and integrating the spectral information and the edge information in the block for each block. A center position of a block whose homogeneity index value is greater than or equal to a specific value is selected as an initial seed point, which is performed by using a program in main memory;

(d) performing initial image segmentation using a program in the main memory including the initial seed point and the MSRG technique in step (c), and

(e) In order to improve the initial image segmentation result of step (d), the final image segmentation result image is generated using a program in the main memory including an iterative region merging process based on a region adjacency graph (RAG) data structure. The basic feature consists of the steps that are generated.

As described above, the automatic image segmentation method for object-based analysis using the high resolution satellite image of the present invention has an effect of generating an effective segmented image for object-based analysis of various high resolution satellite images.

In particular, the present invention minimizes the influence on the segmentation result according to the selection and method of the initial seed point in the region-based image segmentation method, and integrates various characteristic information (spectral, edge, texture) of the input image in the image segmentation step. The over-segmentation problem of the initial image segmentation can be solved.

1 shows a schematic block diagram of a process according to the invention.
2 is a diagram illustrating a data input and preprocessing step according to the present invention.
3 is a view showing an automatic initial seed point generation step according to the present invention.
4 is a diagram illustrating a step of performing initial image segmentation according to the present invention.
5 illustrates a region merging step according to the present invention.
FIG. 6 illustrates an embodiment of initial image segmentation and region merging using FIGS. 4 and 5;

When described in detail with reference to the accompanying drawings a preferred embodiment of the present invention configured as described above are as follows.

The image segmentation method of the present invention is a preprocessing step of extracting feature information necessary for image quality improvement and segmentation prior to image segmentation, and a meaningful initial seed point extraction step in the image, based on a general approach of the region-based segmentation method. From the extracted seed point, we decompose it into an initial image segmentation step based on spatial / spectral characteristics and a region merging step to improve initial image segmentation results, and extract and apply characteristic information in the image through a methodology corresponding to each stage.

1 is a schematic block diagram of a proposed process based on the configuration of the present invention. As shown in FIG. 1, the present invention provides an image fusion process and preprocessing step 100 for inputting two image data of a multispectral image 10 and a high resolution black and white image 20, and for improving spatial resolution of the multispectral image. Initial seed point generation step 200 used as an input value of the partitioning step, step of performing initial image segmentation using the extracted seed point 300, and performing region merging to improve initial image segmentation result 400. And the methodology used in each process. The purpose of these steps is as follows.

(100) Data input and preprocessing steps: Effective use of satellite images such as Geoeye-1, IKONOS-2, and Quickbird-2, which simultaneously provide high resolution black and white and multispectral images. For this purpose, the image fusion process forcibly increasing the spatial resolution of the multispectral data and the preprocessing process for improving the contrast of the image are performed. In addition, edge information and texture information used as characteristic information are generated in the image segmentation step.

(200) Automatic initial seed point generation step: Automatically extract the initial seed point from the entire image based on the homogeneity index (Homogeneity index) integrating the spectral information and edge information of the high-resolution multispectral image processed through the preprocessing Perform the process.

(300) Performing initial image segmentation: Initial image using SRG (Seeded Region Growing (MSRG)) technique, which is modified to allow simultaneous consideration of spectroscopic and edge information of the image using the initial seed information extracted earlier. Perform the split.

(400) Region Merging Step for Improving Initial Image Segmentation: The initial segmentation result has an over-segmentation problem in which too many divisions occur. In order to improve this initial segmentation result, the region merging process based on the Region Adjacency Graph (RAG) data structure is performed.

Hereinafter, the image segmentation procedure will be described in detail.

1. Data entry and preprocessing steps

Although the two images 10 and 20 provided by the high resolution satellite have the same coordinates, the size of the multispectral image 10 is small as the spatial resolution difference between the two images with respect to the black and white image 20 due to the spatial resolution difference. Exists as. Most image segmentation techniques using high resolution satellite images have been performed using only black and white images with high resolution. However, for more accurate image segmentation, it is preferable to use spectral information of multispectral data.

For the production of high resolution multispectral images (spectral information generation), the fusion images are generated using the Gram-Schmidt technique, which is known as an accurate fusion technique, using multispectral and black and white images provided by high resolution satellites as input data. Produce (101). Here, the multispectral image 10 and the black and white image 20 are input to the main memory as input data for a preprocessing process, and the high resolution multispectral image, which is a fusion image, is also stored in the main memory.

Meanwhile, the Gram-Schmidt technique is performed by a program stored in the main memory by coding an algorithm directly through a programming language in order to perform it by a computer.

 1) Edge preservation image improvement

In general, in the case of a fused image, the spectral information is distorted, and thus the color of the fused image is changed. This problem causes a lot of problems in the future image segmentation step, it is necessary to improve the image. In the case of the general image smoothing technique, severe blurring occurs in the edge region, which affects the edge extraction process later.

In order to solve this problem, image enhancement is performed by modifying the existing maximum homogeneity neighbor technique to consider multidimensional data so as to improve the multispectral fusion image while preserving edge information of the image (102). . This technique uses nine window masks consisting of four pentagons, four hexagons, and one rectangle, and image enhancement is the mean spectral vector of the mask having the maximum homogeneity among the nine window masks. To replace the value of. The specific method of implementation is as follows.

)를 다음의 수학식 1로 계산한다.First, the mean spectral vector (

) Is calculated by the following equation.

Where N denotes the number of window masks used, S _k is a set of pixels included in the window region, f (i, j) is a spectral vector of the pixel at point i, j, and m is a pixel included in the mask region. Indicates the number of these.

② The homogeneity of each window mask area is calculated using the variance vector (V (k)) of Equation 2 below and the original spectral vector is replaced with the average spectral vector of the window mask having the largest dispersion vector. .

Here, f (x, y) is a spectral vector of the window mask center position.

③ Repeat steps ① and ② for all the pixels in the image.

In this case, the image improvement process is performed by a program stored in the main memory by coding an algorithm directly through a programming language in order to perform this operation by a computer.

 2) Multispectral Edge Extraction and Texture Information Generation

Since the edge and texture information of the image includes the geometric structure and spatial information of the image, it can be used as useful information for the initial seed point extraction and image segmentation.

Edge information is generally extracted by using the high resolution black and white image 20. However, the edge detection error and a part of the continuous boundary are frequently broken. In order to solve this problem, edge information integrating all band information of a multispectral image is extracted using an entropy-based multispectral edge extraction operator (103). The entropy measurement E in a specific window mask region of a single band is defined as in Equation 3 below.

Here, a _i represents a pixel value of n neighboring pixels existing in the window area, and p _i represents a probability value thereof. The result of extending the above entropy measurement to the multidimensional spectral feature space is expressed as a linear combination of entropy measurements of individual dimensions as shown in Equation 4 below.

Here, b _k represents the pixel value of the window center pixel, q _k represents its probability value, and N represents the number of bands of the image. The edge strength H has a value between 0 and 1, and a larger value represents stronger edge information and is assigned to each pixel position.

Texture information is extracted from a high resolution black and white image using a Block Difference of Inverse Probabilities (5) algorithm that well reflects the edge and local characteristics of the image (104) and is assigned to each pixel position. (FIG. 2).

Here, B denotes a set of pixels in a fixed block area designated by a user, f (i, j) denotes an intensity at (i, j), and M denotes a block size.

In addition, the multispectral edge extraction and the texture information generation are also performed by a program stored in the main memory by coding an algorithm directly through a programming language in order to perform this by a computer.

2. Automatic Initial seed  Point creation step

As with most region-based image segmentation methods, the MSRG scheme proposed in the present invention may produce different segmentation results depending on the selection of initial seed points. In particular, when the initial seed is placed at the edge where the color change is severe, erroneous splitting may occur because it may be merged with different objects. Therefore, in order to perform successful image segmentation using MSRG, it is most important to extract meaningful initial seed points from the image.

For automatic initial seed point extraction, the initial input images 119 and 129 are divided into blocks of a user-specified size (201), and the homogeneity index generated by integrating the spectral information and the edge information in the block for each block is obtained. In operation 202, a center position of a block having a value equal to or greater than a specific value is selected as an initial seed point (219). Here, the threshold value, which is a specific value, may be set based on the expert's experience and subjective judgment, but a reliable result may be obtained when a value between 0.8 and 1 is set for various experimental results. 3 shows a detailed process thereof.

When the initial M × N size input image is divided into p × q size blocks, the original image is divided into m × n (m = [M / p], n = [N / q]) blocks, and the homogeneity index of each block is Is derived from normalized total standard deviation information of feature information in a block. First, the normalized total standard deviation σ _NTS for multispectral image information in each block is defined as in Equation 6 below.

Where σ _total represents the sum of the standard deviations for each band in the block,

σ _max = max (σ _i ) represents the largest value among the band-specific standard deviations of the multispectral image. The normalized total standard deviation σ _ETS for edge information in each block is defined as in Equation 7 below.

Here, sigma _block represents the standard deviation in the block, and sigma _edgemap represents the total standard deviation of the edge image, respectively. Σ _ETS and σ _NTS for each block have a value between 0 and 1, and the larger the value, the higher the degree of heterogeneity of the corresponding region.

The homogeneity index HI of each block integrating the spectral information and the edge information of the multispectral image is defined as in Equation 8 below.

The homogeneity index HI also has a value between 0 and 1, with a larger value indicating that the region is homogeneous.

Here, the initial seed point generation process is performed by a program stored in the main memory by coding an algorithm directly through a programming language in order to perform this process by a computer.

3. Performing initial image segmentation

SRG image segmentation is a method of gradually merging regions by grouping neighboring pixels having similar properties from an initial seed point. Overall region expansion is repeatedly performed until all pixels are included in regions according to merging criteria. Adams (1994) used Sequentially Sorted List (SSL), a simple linked list structure, to sort the elements of set T in order to implement the SRG algorithm. As a reference function, the difference of the pixel value in gray level was used. However, SSL, which is a list data structure, has the disadvantages of complicated memory management and slow processing as the amount of data increases when inserting and sorting data generated at every stage of SRG domain growth.

로 설정하고, 단순 밝기값의 차이를 이용하는 기존기법과는 달리 분광정보 뿐 아니라 에지정보를 고려할 수 있도록 다음의 수학식 9와 같이 정의한다.To solve this problem, we use priority queue (PQ), a data structure that stores and manages the position of data in the queue according to the priority of data for efficient memory management and processing speed improvement. The implementation uses a heap, a special binary tree that automatically sorts all data when inserting and deleting data. In addition, the cost function to determine the priority of the data

Unlike the existing technique using the difference of the simple brightness value, it is defined as in Equation 9 so that edge information as well as spectral information can be considered.

와

는 각각 영역의 평균벡터와 인접한 이웃화소의 벡터를 나타내고, G_c와 G_p는 해당영역의 평균 에지강도와 이웃화소의 에지강도를 나타낸다. 구체적인 실행과정은 아래의 과정, 즉, 도 4와 같고, 도 6의 (b)는 상기 방법론이 실제 자료에 적용되었을 때의 예시를 보여준다.here,

Wow

Denote the vectors of neighboring pixels adjacent to the mean vector of the regions, respectively, and G _c and G _p denote the edge strengths of the neighboring pixels and the mean edge strength of the corresponding regions. The specific execution process is the following process, that is, as shown in Figure 4, Figure 6 (b) shows an example when the methodology is applied to the actual data.

① Insert all initial seed points into the priority queue

가 가장 낮은 초기시드포인트 데이터를 추출② Cost function in the priority queue

The lowest initial seedpoint data

③ Check the initial seed point data area allocation extracted in step ② and update the area information (spectral, edge).

를 계산하여 다시 우선순위 대기열에 삽입④ Cost function for pixels in 8 directions adjacent to initial seed point data extracted in step ②

Is calculated and put back into the priority queue

⑤ Repeat steps ②-④ until the priority queue becomes empty.

On the other hand, the initial image segmentation result of performing the MSRG process has a problem of over-division, as shown in (b) of FIG. 6, since the entire image is divided into the same number of regions as the initial seed point number. The execution is done by a program stored in main memory by coding the algorithm directly via a programming language to do this with a computer.

4. Region Merging Stage to Improve Initial Image Segmentation

The MSRG-based initial segmentation result still has an over-segmentation problem in which too many partitions are generated. In order to improve the initial image segmentation result, the present invention generates a final segmentation result image using an iterative region merging process based on a region adjacency graph (RAG) data structure. An adjacency graph data structure is a weighted graph that allocates a segmented region to a graph node and assigns a cost that indicates the difference between two partitions adjacent to a graph edge. The cost (Equation 10) representing the difference between two adjacent regions is calculated by integrating the spectral information and the texture information, and the value is assigned to the edge.

Where CD (R _i , R _j ) is the region R _i Is a function that reflects the difference in the mean spectral vectors between and R _j , where TD (R _i , R _j ) is the region R _i And R _j The function reflects the difference in the texture intensity between and N represents the number of bands in the image. The range of Hf (R _i , R _j ) values is between 0 and 1 because data normalization is performed to adjust the variation of CD (R _i , R _j ) and TD (R _i , R _j ) values. Has the value of.

By setting the threshold of the merge condition of the adjacent region, the merge is performed to the adjacent region where Hf (R _i , R _j ) becomes the minimum for the regions smaller than the threshold. Threshold settings yield reliable results at values below 0.3 for various experiments, but can be set by expert experience. After each region merging, the cost and RAG information of all edges connected to the combined region are updated, and the final segmented image is produced by repeating until no further merge occurs. The specific execution process is the following process, that is, as shown in Figure 5, Figure 6 (c) shows an example when the methodology is applied to the actual data.

① Preparation of RAG using initial image segmentation result

② Calculation of average spectroscopic vector and texture intensity

③ Select all adjacent regions of region R _i through RAG

④ Determination of merging between region R _i and adjacent region by measurement index Hf (R _i , R _j ) integrating spectral vector and texture information

⑤ if present, the region R _i and R _j are merged area updates the following information:

_-Add neighboring region information of region R _j to neighboring region information of region R _i

-Update the mean spectral vector and texture intensity of the region R _i by averaging the mean spectral vectors of the regions R _i and R _j

⑥ Delete the adjacent area of area R _j

⑦ Repeat steps ③-⑥ for all areas.

⑧ Repeat process ③-⑦ until no more area merge occurs.

In addition, the process is performed by a program stored in the main memory by coding an algorithm directly through a programming language to perform this with a computer.

While specific preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the above-described embodiments, and a person skilled in the art to which the present invention pertains has the technical gist of the present invention. Various changes can be made without departing.

Claims (6)

(a) The multispectral image and the black and white image provided from the satellite are input to the main memory, and the production of the high resolution multispectral image, which is a fusion image using the two images, is performed using the Gram-Schmidt technique. Performing using a program in memory;
(b) The improvement of the fusion image of step (a) is performed by using a program in main memory including a modified maximum homogeneity neighbor technique to consider multidimensional data, and extracting entropy-based multispectral edges. Edge information combining all band information of multispectral image is extracted by using operator, and texture information is obtained by using program in main memory including Block Difference of Inverse Probabilities algorithm. Extracting from;
(c) The homogeneity index generated by combining the improved fusion image and the edge image which passed through step (b) into blocks of a user-specified size and integrating the spectral information and the edge information in the block for each block. A center position of a block whose homogeneity index value is greater than or equal to a specific value is selected as an initial seed point, which is performed by using a program in main memory;
(d) performing initial image segmentation using a program in the main memory including the initial seed point and the MSRG technique in step (c), and
(e) In order to improve the initial image segmentation result of step (d), the final image segmentation result image is generated using a program in the main memory including an iterative region merging process based on a region adjacency graph (RAG) data structure. Automatic image segmentation method for object-based analysis using high resolution satellite image, characterized in that the step consisting of generating.
The method of claim 1,
The modified maximum homogeneous neighborhood technique of step (b)
(f) For each of the nine window masks consisting of four pentagons, four hexagons, and one rectangle, use the program in main memory containing the equation

) Is calculated;

Where N is the number of window masks used, S _k is the set of pixels contained in the window area, f (i, j) is the spectral vector of the pixel at point i, j, and m is the number of pixels contained in the mask area. Count)
(g) replacing the original spectral vector of the fused image with the average spectral vector of the window mask having the largest value of the variance vector V (k) using a program in the main memory including the following equation, and

(Where f (x, y) is the spectral vector at the center of the window mask)
(h) the automatic image segmentation method for object-based analysis using high resolution satellite images, wherein the steps (f) and (g) are repeated for all the pixels of the image.
The method of claim 1,
Edge information extraction of step (b) is the following equation, that is,

Where b _k is the pixel value of the window center pixel, q _k is its probability value, and N is the number of bands in the image,

, a _i is a pixel value of n neighboring pixels existing in the window region, and p _i is a program in main memory including the probability value thereof.
Extraction of texture information is given by the following equation,

Where B is a set of pixels within a fixed block area specified by the user, f (i, j) is the intensity at (i, j), and M is the block size. Automatic image segmentation method for object-based analysis using a high resolution satellite image, characterized in that performed by using.
The method of claim 1,
The calculation of the homogeneity index (HI) of the step (c) is the following equation, that is,

(here,

, σ _block is the standard deviation within the _block , σ _Edgemap is the overall standard deviation of the edge image,

σ _total is the sum of the standard deviations per band in the block, and σ _max = max (σ _i ) is the largest value of the standard deviations per band of the multispectral image). The threshold value is set to a value between 0.8 ~ 1 Automatic image segmentation method for object-based analysis using a high-resolution satellite image.
The method of claim 1,
The MSRG technique of step (d)
(i) inserting all initial seedpoints into a priority queue;
(j) Cost function in priority queue

(here,

Wow

Extracting initial seed point data having the lowest average vector strength of the region and neighboring pixels, G _c and G _p are the average edge strength of the corresponding region and the edge strength of the neighboring pixel, respectively;
(k) checking the initial seed point data area allocation extracted in the step (j) and updating area information (spectral, edge);
(l) Cost function for pixels in eight directions adjacent to the initial seed point data extracted in step (j)

Is calculated and inserted back into the priority queue, and
(m) The automatic image segmentation method for object-based analysis using high resolution satellite images, characterized in that the steps (j) to (l) are repeated until the priority queue becomes empty.
The method of claim 1,
The area merging process of step (e)
(n) generating a region adjacent graph (RAG) using the initial image segmentation result;
(o) calculating average spectral vectors and texture intensities for each region;
(p) selecting all adjacent regions of the region R _i through the region adjoining graph RAG in step (n);
(q) Measurement index integrating spectral vector and texture information of step (o)

Where CD (R _i , R _j ) is the region R _i Is a function that reflects the difference in the mean spectral vectors between and R _j , where TD (R _i , R _j ) is the region R _i And R _j A function that reflects the difference in texture intensity between N, where N is the merged area between R _i and the adjacent area based on the number of bands of the image, and is determined. Merging is performed to an adjacent region where Hf (R _i , R _j ) is minimum and setting a threshold of Hf (R _i , R _j ) to a value between 0 and 0.3;
(r) region R _i and being merged area if there is R _j, region R _j adding the adjacent region information to the adjacent region information of the region R _i and region R _i and R _j average spectral vector average to region R _i of the Updating the mean spectral vector and the texture intensity of;
(s) deleting the adjacent region of region R _j ;
(t) repeating steps (p) to (s) for all regions, and
(u) The automatic image segmentation method for object-based analysis using high resolution satellite images, characterized in that the steps (p) to (t) are repeated until the region merging no longer occurs.

Images