KR101150901B1 - An automatic segmentation method for object-based analysis using high resolution satellite imagery - Google Patents

Abstract

The present invention relates to an automatic image segmentation method for object-based analysis using satellite images, the method comprising: (a) inputting a multispectral image and a monochrome image provided by a satellite into a main memory as input data; In which a program in a main memory including a Gram-Schmidt technique is used; (b) preservation of the edge information of the fusion image is performed using a program in the main memory including the modified maximum homology proximity technique, edge information combining all the band information of the multispectral image is extracted, Extracting a program using a program in a main memory including a difference algorithm; (c) the fusion image and the edge image are divided into a predetermined block, a homogeneity index is calculated for each block, a center position of a block having the homogeneity index value of a specific value or more is selected as an initial seed point, A step of performing a selection process using a program in the main memory; (d) performing an initial image segmentation using a program in a main memory including an initial seed point and an MSRG technique; and (e) performing an iterative region merging process based on an area adjacency graph data structure And generating an area-merged image from the initially divided image by using the program in the main memory, which is included in the main memory. Thus, an effective divided image for object-based analysis of various high resolution satellite images can be generated.

Description

[0001] The present invention relates to an automatic segmentation method for object-based analysis using satellite images,

The present invention relates to an automatic image segmentation method for object-based analysis using satellite images, and more particularly, to an object-based analysis method for generating effective segment images for object-based analysis of various high resolution satellite images The present invention relates to an automatic image segmentation method.

Generally, as the spatial resolution of an image increases, it exhibits various spectral characteristics even in the same object. In addition, since it has a spatially different shape, in order to extract meaningful information from a high resolution satellite image, It is true that the direct application of the pixel-based technique, which has been widely used, is unreasonable. In order to compensate for this problem, it is necessary to classify the pixels having the same characteristics in the image segmentation process prior to classification into an object or a segment, and to classify the objects based on the characteristic information of the objects, based method has emerged as a suitable classification method for high resolution images. In the case of the object-based classification method, since the accuracy of the object information generated through the image segmentation process directly affects the classification result, the image segmentation process is a very important preprocessing process in object-based classification.

Image segmentation refers to the task of separating a given image into its constituent elements or a set of objects. Each region must satisfy the homogeneity and connectivity, and the sum of all the divided areas constitutes the whole image. The image segmentation method can be divided into histogram-based approach, edge-based approach and region-based approach. Histogram-based techniques are suitable for application to satellite images with high image complexity. Since the edge-based methods use only the edge information of the image basically, a complicated post-processing process is required to maintain the connectivity of the area. Image segmentation algorithm applied to satellite image is mainly based on area based method, and watershed segmentation algorithm is actively applied to satellite image. However, since the method using the watershed segmentation algorithm basically uses only the edge information of the image, the accuracy of the segmentation is not only sensitive to the edge information used but also does not consider the multispectral information of the satellite image due to the characteristics of the algorithm.

Conventionally, in a satellite image segmentation method provided in most application programs in the remote sensing field, an image is output on a screen to give a seed point to an object of interest depending on a visual judgment of an analyst, In this paper, we propose a simple method of segmenting images by creating a histogram on a given image and determining the thresholding by the user.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a satellite image segmentation method using various characteristic information of an image for effective use of a high resolution satellite image.

Particularly, the present invention minimizes the influence on the selection result of the initial seed point and the division result according to the method appearing in the region-based image segmentation method, and integrates various characteristic information (spectroscopic, edge, texture) And over-segmentation of the initial image segmentation.

To achieve these and other advantages and in accordance with the purpose of the present invention,

(a) Multispectral images and monochrome images provided by satellites are input to the main memory as input data, and a fusion image using the two images is generated using a program in the main memory including a Gram-Schmidt technique ;

(b) preserving edge information of the fused image of step (a) is performed using a program in a main memory including a modified maximum homogeneity neighbors technique capable of considering multidimensional data, and entropy-based multi-spectroscopy Edge information combining all the band information of the multispectral image is extracted using the edge extraction operator and the texture information is extracted using a program in the main memory including the Block Difference of Inverse Probabilities algorithm Extracting from a monochrome image;

(c) dividing the fusion image and the edge image through the step (b) into blocks of a size designated by the user, and calculating a homogeneity index generated by integrating the spectral information and the edge information in each block for each block A center position of a block having the homogeneity index value of a specific value or more is selected as an initial seed point, the selection of the initial seed point is performed using a program in the main memory;

(d) performing an initial image segmentation using a program in a main memory including an initial seed point and an MSRG technique in step (c); and

(e) analyzing an initial segmented image using a program in a main memory including an iterative region merging process based on a Region Adjacency Graph (RAG) data structure for region merging of the initially segmented image of the step (d) And a step of generating an area-merged image.

The automatic image segmentation method for object-based analysis using the satellite image according to the present invention has the effect of generating an effective segment image for object-based analysis of various high resolution satellite images.

Particularly, the present invention minimizes the influence of the selection of the initial seed point and the division result according to the method in the region-based image segmentation method, and integrates various characteristic information (spectroscopic, edge, texture) of the input image in the image segmentation step , And over-segmentation problem of the initial image segmentation can be solved.

1 shows a schematic block diagram of a process according to the invention;
FIG. 2 illustrates data input and preprocessing steps in accordance with the present invention. FIG.
3 is a diagram illustrating an automatic initial seed point generation step according to the present invention.
Figure 4 illustrates performing initial image segmentation in accordance with the present invention.
5 illustrates the merging of regions according to the present invention.
FIG. 6 illustrates an embodiment of an initial image segmentation and region merging using FIGS. 4 and 5. FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The image segmentation method of the present invention includes a preprocessing step of extracting feature information necessary for image quality improvement and segmentation prior to image segmentation based on a general approach of a region-based segmentation method, a step of extracting a meaningful initial seeded point An initial image segmentation step based on the spatial / spectral characteristics from the extracted seed point, and an area merging step for improving the initial image segmentation result, and extracts and applies characteristic information in the image through the corresponding methodology for each step.

1 is a schematic block diagram of a proposed process based on the configuration of the present invention. As shown in FIG. 1, the present invention includes an image fusion process and a pre-processing step (100) for inputting two image data of a multispectral image (10) and a high resolution monochrome image (20) An initial seed point generation step 200 used as an input value of the segmentation step, an initial image segmentation using an extracted seed point 300, a region merging 400 for improving an initial image segmentation result, And the methodology used in each process. These step-by-step objectives are as follows.

(100) Data input and preprocessing stage: Effective use of satellite images such as Geoeye-1, IKONOS-2 and Quickbird-2 which provide high-resolution monochrome and multispectral images at the same time The image fusion process for forcibly increasing the spatial resolution of the multispectral data and the preprocessing process for improving the contrast of the image are performed. Also, edge information and texture information used as characteristic information are generated in the image segmentation step.

(200) Automatic initial seed point generation step: Automatically extract initial seed points from a whole image based on a homogeneity index that integrates spectral information and edge information of a high-resolution multispectral image processed through the preprocessing process .

(300) Performing the initial image segmentation: Modifying the initial seed image information so that the spectral information and the edge information of the image can be simultaneously considered Using the supplemented SRG (Seeded Region Growing) Perform segmentation.

(400) Area merging step to improve initial image segmentation result: The initial segmentation result has an over-segmentation problem in which a divided region occurs too much. In order to improve the initial segmentation result, a region merging process based on a Region Adjacency Graph (RAG) data structure is performed.

Hereinafter, the image segmentation procedure will be described in detail.

1. Data entry and preprocessing steps

Although the two images 10 and 20 provided by the high resolution satellite have the same coordinates, the size of the multispectral image 10 due to the spatial resolution difference is smaller than the spatial resolution difference between the two images with respect to the monochrome image 20 Lt; / RTI > Most of image segmentation methods using existing high resolution satellite images have been performed using only high resolution black and white images. However, it is preferable to use spectroscopic information of multispectral data for more accurate image segmentation.

In order to produce a high-resolution multispectral image (spectral information generation), the Gram-Schmidt technique, which is known as a highly accurate fusion technique using multispectral images and monochrome images provided by high-resolution satellites, (101). Here, the multispectral image 10 and the monochrome image 20 are input to the main memory as input data for the preprocessing process, and the high resolution multispectral image, which is a fusion image, is also stored in the main memory.

The Gram-Schmidt technique is performed by a program stored in the main memory by directly coding an algorithm through a programming language to perform the Gram-Schmidt technique.

 1) Edge preservation image improvement

Generally, in the case of a fusion image, the spectral information is distorted and the color of the fusion image is deteriorated. Such a problem causes many problems in the image segmentation step in the future, so it is necessary to improve the image. In the case of general image smoothing, severe blurring occurs in the edge region, which affects the edge extraction process.

In order to solve this problem, the existing maximum homogeneity neighbors technique is modified to consider multidimensional data in order to improve the multispectral fusion image while preserving the edge information of the fusion image (102 ). This technique uses nine window masks composed of four pentagons, four hexagons, and one rectangle, and the image enhancement is the average spectral vector of the mask having the greatest homogeneity among the nine window masks, Is replaced with a value of. The specific implementation method is as follows.

)를 다음의 수학식 1로 계산한다.First, the average spectral vector (

) Is calculated by the following equation (1).

Here, N denotes the number of window masks used, S _k denotes a set of pixels included in the window region, f (i, j) denotes a spectral vector of a pixel at i, j, m denotes a pixel Lt; / RTI >

(2) The homogeneity for each window mask region is calculated using the dispersion vector V (k) shown in Equation 2 and the original spectral vector is substituted for the average spectral vector of the window mask having the largest variance vector .

Here, f (x, y) is a spectral vector at the center position of the window mask.

(3) Repeat steps (1) and (2) for all pixels of the converged image.

Here, the image enhancement process is performed by a program stored in the main memory by directly coding an algorithm through a program language to perform the image enhancement process with a computer.

 2) Multi-spectral edge extraction and texture information generation

Since the edge information and texture information of the image include the geometric structure and spatial information of the image, it can be used as information useful for the extraction of the initial seed point and the image segmentation step.

Generally, edge information is extracted using a high resolution black and white image (20), and edge detection error and a part of a continuous boundary line are frequently disconnected. To solve this problem, we extract edge information that integrates all band information of multispectral image using entropy-based multi-spectral edge extraction operator (103). The entropy measurement (E) in a particular window mask area of a single band is defined as: < EMI ID = 3.0 >

Here, a _i denotes a pixel value of n neighboring pixels existing in the window region, and p _i denotes a probability value of n neighboring pixels existing in the window region. The result of extending the above entropy measure to the multidimensional spectral feature space is represented by a linear combination of the entropy measurements of the individual dimensions as:

Here, b _k represents the pixel value of the center pixel of the window, q _k represents the probability value of the center pixel of the window, and N represents the number of bands of the image. The edge intensity H has a value between 0 and 1. The larger the value, the stronger the edge information is to be represented and assigned to each pixel position.

In the case of the texture information, the texture information is extracted from the high-resolution black-and-white image using the block difference inverse probabilities (Equation 5) algorithm that well reflects the edges of the image and the local characteristics (104) (Fig. 2).

Where B is the set of pixels in the fixed block area specified by the user, f (i, j) is the intensity at (i, j) and M is the block size.

Also, the multi-spectral edge extraction and texture information generation are performed by a program stored in the main memory by directly coding an algorithm through a program language to perform the multi-spectral edge extraction and the texture information generation.

2. Automatic Initial seed  Point creation step

Similar to the majority-area-based image segmentation method, the MSRG technique proposed by the present invention can also result in different segmentation results depending on the selection of the initial seed point. Particularly, if the initial seed is placed at the edge portion where the color change is severe, an incorrect division result may occur because it can be merged with different objects. Therefore, in order to perform successful image segmentation through the MSRG technique, it is most important to extract meaningful initial seed points from an image.

In order to extract the automatic initial seed point, the initial input images 119 and 129 are divided into blocks of a size designated by the user (201), and a homogeneity index generated by integrating the spectral information and the edge information in each block is calculated (202), and the center position of the block whose value is greater than or equal to a specific value is selected as the initial seed point (219). Here, the threshold value can be set based on expert experience and subjective judgment. However, when various values are set between 0.8 and 1, a reliable result can be obtained. FIG. 3 shows a detailed process for this.

The original image is divided into m × n (m = [M / p], n = [N / q]) blocks and the homogeneity index Derives from the normalized total standard deviation information of the feature information in the block. First, the normalized total standard deviation? _NTS for the multispectral image information in each block is defined as the following Equation (6).

Here, σ denotes the sum _total of each band the standard deviation in the block,

σ _max = max (σ _i ) represents the largest value among the standard deviations of all bands in the multispectral image. The normalized total standard deviation [sigma] _ETS for the edge information in each block is defined as Equation (7).

Here, σ _block represents a standard deviation within a _block , and σ _Edgemap represents an overall standard deviation of an edge image, respectively. For each block, σ _ETS , σ _NTS has a value between 0 and 1, and the larger the value, the higher the degree of heterogeneity of the region.

The homogeneity index HI of each block combining the spectroscopic information and the edge information of the multispectral image is defined as the following Equation (8).

The homogeneity index HI also has a value between 0 and 1. The larger the value, the more homogeneous the region is.

Also, the execution of the initial seed point generation process is performed by a program stored in the main memory by directly coding an algorithm through a programming language to perform the initial seed point generation process.

3. Performing initial image segmentation

The SRG image segmentation is a method of grouping neighboring pixels having similar properties from the initial seed point and gradually merging the regions. The overall region expansion is repeatedly performed until all the pixels are included in the regions according to the merging criterion. Adams (1994) used a sequential sorted list (SSL) as a data structure for sorting the elements of the set T in the implementation of the SRG algorithm, and judged similarity with neighboring pixels As a reference function, the difference of pixel values at gray level was used. However, SSL, a list data structure, has a disadvantage in that memory management becomes complicated when inserting and sorting data occurring at every stage of SRG region growth, and the processing speed is slower as the amount of data increases.

로 설정하고, 단순 밝기값의 차이를 이용하는 기존기법과는 달리 분광정보 뿐 아니라 에지정보를 고려할 수 있도록 다음의 수학식 9와 같이 정의한다.In order to solve this problem, the priority queue (PQ), which is a data structure for storing and managing the location of data in the queue according to the priority of the data, is used for efficient memory management and improvement of the processing speed, The implementation uses a heap, which is a special binary tree type in which all data is automatically sorted when inserting and deleting data. The cost function to determine the priority of the data

And is defined as Equation (9) so as to consider not only spectral information but also edge information, unlike the conventional technique using difference of simple brightness values.

와

는 각각 영역의 평균벡터와 인접한 이웃화소의 벡터를 나타내고, G_c와 G_p는 해당영역의 평균 에지강도와 이웃화소의 에지강도를 나타낸다. 구체적인 실행과정은 아래의 과정, 즉, 도 4와 같고, 도 6의 (b)는 상기 방법론이 실제 자료에 적용되었을 때의 예시를 보여준다.here,

Wow

Where G _c and G _p represent the average edge intensity of the corresponding region and the edge strength of neighboring pixels, respectively. The concrete procedure is shown in the following process, that is, as shown in FIG. 4, and FIG. 6 (b) shows an example when the methodology is applied to actual data.

Insert all initial seed points into the priority queue

가 가장 낮은 초기시드포인트 데이터를 추출② Cost function in priority queue

Extracts the lowest initial seed point data

(3) Check whether or not the initial seed point data area extracted in (2) is allocated and update the area information (spectroscopic edge)

를 계산하여 다시 우선순위 대기열에 삽입(4) For the 8-directional pixels adjacent to the initial seed point data extracted in (2) above, the cost function

Into the priority queue again

⑤ Repeat steps ②-④ until the priority queue becomes empty.

Meanwhile, since the initial image segmentation result obtained by performing the MSRG process is divided into the same number of regions as the initial seed points, the MSRG process has a problem of overdrawing as shown in FIG. 6 (b) The execution is performed by a program stored in the main memory by coding the algorithm directly through the program language to perform it with the computer.

4. Region merging step to improve initial image segmentation result

The initial image segmentation result based on MSRG still has an over-segmentation problem in which an excessive number of partitions occur. In order to improve the initial image segmentation result, the present invention generates a final segmentation result image (region-merged image from the initially segmented image) using the iterative region merging process based on the Region Adjacency Graph (RAG) data structure do. The area adjacency graph data structure is a weight graph that allocates a divided area to a graph node and assigns a cost indicating the difference between two adjacent areas on the graph edge. The cost (Equation 10) representing the difference between two adjacent areas is calculated by integrating the spectral information and the texture information, and the value is assigned to the edge.

Where CD (R _i , R _j ) is the area R _i And R average spectral vector function, TD, which reflects the difference between _j (R _i, R _j) is a region R _i And R _j And N represents the number of bands in the image. Since the data normalization process is performed to adjust the variation of the values of CD (R _i , R _j ) and TD (R _i , R _j ), the range of Hf (R _i , R _j ) Lt; / RTI >

The threshold value of the merge condition of the adjacent region is set and the merge is performed to the adjacent region where Hf (R _i , R _j ) is minimum for the regions smaller than the threshold value. Threshold settings can yield reliable results at values below 0.3 in a variety of experiments, but can be set by expert experience. After merging each region, update the cost and RAG information of all edges connected to the merged region, and repeat the process until the merging no longer occurs. The concrete procedure of execution is as follows, that is, as shown in FIG. 5, and FIG. 6 (c) shows an example when the methodology is applied to actual data.

① Creation of RAG using initial image segmentation result

② Average spectral vector and texture intensity calculation by region

③ Select all adjacent areas of area R _i through RAG

④ Determine whether the region R _i and the neighboring region are merged by the measurement index Hf (R _i , R _j ) that integrates the spectral vector and the texture information

⑤ if present, the region R _i and R _j are merged area updates the following information:

- Adds the neighboring region information of the region R _{j to} the neighboring region information of the region R _i

- averaging the average spectral vectors of the regions R _i and R _j to update the average spectral vector and the texture intensity of the region R _i

⑥ Delete the adjacent region of the region R _j

⑦ Repeat steps ③-⑥ for each region of the initially segmented image

⑧ Repeat steps ③-⑦ until area merging no longer occurs

In addition, the execution of the above process is performed by a program stored in the main memory by directly coding the algorithm through the program language to perform the process with a computer.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. You will be able to make various changes without departing.

Claims (6)

(a) Multispectral images and monochrome images provided by satellites are input to the main memory as input data, and a fusion image using the two images is generated using a program in the main memory including a Gram-Schmidt technique ;
(b) preserving edge information of the fused image of step (a) is performed using a program in a main memory including a modified maximum homogeneity neighbors technique capable of considering multidimensional data, and entropy-based multi-spectroscopy Edge information combining all the band information of the multispectral image is extracted by using the edge extraction operator. The texture information is extracted by using a program in the main memory including the Block Difference of Inverse Probabilities algorithm Extracting from a monochrome image;
(c) dividing the fusion image and the edge image through the step (b) into blocks of a size designated by the user, and calculating a homogeneity index generated by integrating the spectral information and the edge information in each block for each block A center position of a block having the homogeneity index value of a specific value or more is selected as an initial seed point, the selection of the initial seed point is performed using a program in the main memory;
(d) performing an initial image segmentation using a program in a main memory including an initial seed point and an MSRG technique in step (c); and
(e) analyzing an initial segmented image using a program in a main memory including an iterative region merging process based on a Region Adjacency Graph (RAG) data structure for region merging of the initially segmented image of the step (d) And generating an area-merged image. The method of claim 1,
The method according to claim 1,
The modified maximum homogeneous proximity scheme of step (b)
(f) For each of the nine window masks consisting of four pentagons, four hexagons, and one rectangle, the average spectral vector (

≪ / RTI >

(Where, N is the number of window masks used, S _k is the set of pixels contained in the window region, f (i, j) is the spectral vector of the pixel at i, j, Count)
(g) replacing the original spectral vector of the fusion image with an average spectral vector of the window mask having the largest value of the variance vector (V (k)) using a program in main memory,

(Where f (x, y) is the spectral vector at the window mask center position)
(h) repeating steps (f) and (g) for all pixels of the fusion image.
The method according to claim 1,
The edge information extraction of step (b) may be performed using the following equation:

(Where H is the edge intensity, b _k is the pixel value of the window center pixel, q _k is the probability value of the window center pixel, and N is the number of bands of the image,

, a _i is a pixel value of n neighboring pixels existing in a window region, and p _i is a probability value of n neighboring pixels existing in a window region)
The extraction of the texture information is performed by the following equation:

(I, j) is a brightness value at a point (i, j), and M is a block size), where BDIP is a conduction probability block difference, B is a set of pixels in a fixed block area designated by the user, f Wherein the program is executed using a program in the main memory.
The method according to claim 1,
The calculation of the homogeneity index (HI) in the step (c) can be performed by the following equation:

(here,

, σ _block is the standard deviation in the block, σ _Edgemap is the total standard deviation of the edge image,

, σ _total is the sum of the standard deviations of the bands in the block, and σ _max = max (σ _i ) is the largest value among the standard deviations of all the bands of the multispectral image) Wherein the threshold value is set to a value between 0.8 and 1. The automatic image segmentation method for object-based analysis using a satellite image.
The method according to claim 1,
The MSRG scheme of step (d)
(i) all initial seed points are inserted into a priority queue;
(j) cost function in priority queue

(here,

Wow

Is a step in which the lowest initial seed point data is an edge strength of the vector, G _c and G _p is the average edge strength of the neighboring pixels of the corresponding area of neighboring pixels close to the mean vector of each zone) extraction;
(k) checking whether or not the initial seed point data area allocated in step (j) is allocated and updating the area information (spectroscopic edge);
(i) calculating a cost function for the pixels in eight directions adjacent to the initial seed point data extracted in the step (j)

And then inserted into the priority queue again, and
(m) repeating the steps (j) to (l) until the priority queue becomes an empty set. The automatic image segmentation method for object-based analysis using satellite images.
The method according to claim 1,
The area merging process of the step (e)
(n) generating an area neighborhood graph (RAG) using an initially divided image;
(o) calculating a mean spectral vector and a texture intensity for each region;
(p) selecting all contiguous regions of region R _i through the region proximity graph (RAG) of step (n);
(q) integrating the spectral vector of step (o) with texture information,

(Where CD (R _i, R _j) is a function reflecting the difference of the average spectral vector between the regions R _i and R _j, TD (R _i, R _j) is reflecting the texture intensity difference between the regions R _i and R _j function, N is the number of bands in the image) above for the according region R _i and jidoe merge is whether the decision between the adjacent areas, area smaller than the threshold value by setting the threshold value of the merge condition of a region adjacent to the Hf (R _i, R _j) is the merged adjacent region is minimum is performed, the threshold value of the Hf (R _i, R _j) step is set to a value between 0 and 0.3;
(r) region R _i and being merged area if there is R _j, region R _j adding the adjacent region information to the adjacent region information of the region R _i and region R _i and R _j average spectral vector average to region R _i of the The average spectral vector and the texture intensity of the texture are updated;
(s) the adjacent region of the region R _j is deleted;
(t) repeating the steps (p) to (s) for each region of the initially segmented image, and
wherein the steps (p) to (t) are repeated until the merging of the (u) regions is no longer generated.

Images