KR101507732B1 - Method for segmenting aerial images based region and Computer readable storage medium for storing program code executing the same - Google Patents

Abstract

The present invention relates to an image segmentation technique for an unmanned aerial vehicle image, and more particularly, to an image segmentation technique optimized for aerial image by combining a technique commonly used in image processing and a newly designed technique, This paper is concerned with intelligent image processing such as surveillance and reconnaissance of simulation images, and area - based aerial image segmentation method that classifies terrain into classes to which the terrain belongs by using context information in the field of terrain analysis.

Description

Field of the Invention [0001] The present invention relates to an area-based aerial image segmentation method and a computer readable storage medium storing the program code for executing the method.

The present invention relates to an image segmentation technique for an unmanned aerial vehicle image, and more particularly, to an image segmentation technique optimized for aerial image by combining a technique commonly used in image processing and a newly designed technique, This paper is concerned with intelligent image processing such as surveillance and reconnaissance of simulation images, and area - based aerial image segmentation method that classifies terrain into classes to which the terrain belongs by using context information in the field of terrain analysis.

Scene segmentation is a technique for separating different semantic objects and spaces appearing in the input image and has recently been applied to a variety of intelligent image processing fields such as surveillance, reconnaissance, data collection, and guidance. Geographical analysis is the final goal in GIS. Given the input image of the terrain, based on the geographical characteristics of each area, it is desirable to classify it as a class to which the terrain belongs. System (GIS) is the goal.

The technique of segmenting images is an important process in many digital image processing fields.

Image segmentation technology is a technique for dividing an image into regions having similar characteristics. For example, when a desired specific region is to be obtained, it is used to obtain an object by separating a desired object and a background portion from the boundary of an object in the region do.

However, if this image segmentation technique is very small in color change between neighboring objects or the algorithm will not make an optimal clear separation often between different segments. As a representative image segmentation algorithm that has been developed so far to solve these problems little by little, there are a lightness value processing method, an area expansion method, a division integration method, and a division method using an edge. Each method has its own characteristics.

First of all, the Threshold method is a method of dividing a value larger than the warning value and a part smaller than the warning value by using the warning value. This method is effective for binary images, but it is difficult to apply to practical images in which various values exist.

Next, the region growing method is a method of finding a region having similar characteristics such as brightness value, edge, and color with the seed as a seed at an arbitrary position in the image. That is, when a small region being processed and a small region adjacent to the small region have the same characteristics, each region is integrated into one region, and the region having the same characteristic is gradually grown to finally divide the region .

Split-and-merge is a method of dividing a region into parts having similar characteristics and then combining the two regions if they have similar characteristics compared to the surrounding region. However, the region fixation method and the segmentation integration method have a lot of difficulty in dividing the entire image even though it requires a lot of calculation amount and repetition in order to find the region having the same characteristics.

1. Korean Patent Publication No. 10-2013-023752 2. Korean Patent No. 10-0980282 3. Korean Patent No. 10-0988872

1. Kwon Hyun et al., "A Study on the Analysis of Numerical Aerial Image Using Image Segmentation Technique", Journal of Geographical Spatial Information, Vol. 2, No. 2, pp.131-142, 1994.

The present invention has been proposed in order to solve the problems according to the above background art, and recently, it has been proposed to optimize a segmentation technique applied in various intelligent image processing fields such as surveillance, reconnaissance, data collection, In this paper, we propose an area - based aerial image segmentation method which solves the problems that the aerial image has low learning data, low resolution and insufficient feature points.

It is another object of the present invention to provide an area-based aerial image segmentation method that improves accuracy in dividing and recognizing aerial images by category by developing a technology specialized in such problems of aerial image.

It is another object of the present invention to provide a computer-readable storage medium storing program codes for executing the area-based aerial image segmentation method.

In order to achieve the above-mentioned object, the present invention provides a method for optimizing a segmentation technique applied in various intelligent image processing fields such as surveillance, reconnaissance, data collection, and guidance to a unmanned aerial vehicle image or an aerial image, Based aerial image segmentation method for solving the problems that the resolution is low and the feature points are insufficient.

The method of claim 1,

A method for segmenting an aerial image captured by an unmanned aerial vehicle into regions based on regions,

An initialization step of applying a filter bank, which is a set of filters, to pixels of the aerial image to extract filter responses;

A reaction obtaining step of dividing the filter reactions into a narrow region and a wide region and applying a division technique to obtain a reaction;

A texton map is generated by textonizing each of the pixels with respect to the aerial image, generating a cluster, extracting texton, mapping the center points of the respective filter reactions and the cluster, and using the texton map as the texture map A texton dictionary generation step of generating a degree of reaction;

A boosting step of extracting final responses for each of the pixels of the aerial image by minimizing an error and assigning a weight to the response degree; And

And a category determination step of classifying the category of the pixel among the pixels of the aerial image according to the highest ratio among the extracted final responses.

Here, the texton dictionary creation step may include generating the filter reactions as clusters using a clustering technique and a distance measurement technique; Generating a texture map by mapping each of the filter reactions and a closest cluster center point using a KD tree technique; And obtaining a degree of reaction by comparing the responses with a specific position of the texture map generated by the predetermined number of rectangular regions r _i and the texture t _i . .

(여기서, area(r)은 직사각형 영역(r_i)의 면적을 나타내고, T_j는 상기 직사각형 영역 내의 특성을 표현하는 텍스쳐를 나타낸다)로 표현되는 것을 특징으로 할 수 있다.Also, the degree of reaction is represented by a probability that the texture t _i occupies a plurality of rectangular areas r _i in the texture map between 0 and 1,

(Where area (r) represents an area of the rectangular area (r _i ), and T _j represents a texture representing characteristics in the rectangular area).

In addition, the filter bank includes at least 17 filters, and the constituent filter of the filter bank is a combination of a Gaussian filter, a Laplacian filter, and a Gaussian filter. can do.

Also, in the case of the Gaussian differential filter, the parameter sigma value may be one of 2 and 4 for the x-axis or the y-axis.

Also, in the case of the Laplacian filter, the parameter sigma value may be one of 1, 2, 4, and 8 for the L channel.

In the case of the Gaussian filter, the parameter sigma values may be one of 1, 2, and 4 for all three channels of the wrap (LAB).

In addition, the components of the aerial image may include at least one of an RGBs color configuration and a CIELab color configuration.

In addition, an average moving method is applied to the wide area using three different factor values, and a K-means partitioning technique is applied using three different factor values for the narrow area have.

Also, the boosting step may be characterized by using a joint boosting technique.

The feature may be one of a texture feature, a SIFT feature, and a feature including position information.

Also, the final reactions may be characterized as a result of six reactions, three of which are reaction results in a large zone and three of which are reaction results in a narrow zone.

On the other hand, another embodiment of the present invention provides a computer-readable storage medium storing program code for executing the above-described region-based aerial image segmentation method.

According to the present invention, the improvement of the accuracy of the aerial image segmentation not only contributes much to the development of technology related to image processing related to surveillance and reconnaissance in the military, but also to the problem of low learning data, low resolution and insufficient feature information The image can be classified with high accuracy.

Another advantage of the present invention is that the efficiency and the cost can be improved by accurately and quickly classifying the aerial image into categories such as roads and forests even in the areas such as digital map development and terrain analysis.

1 is a flowchart illustrating an area-based aerial image segmentation process according to an exemplary embodiment of the present invention.
FIG. 2 is a diagram illustrating an average moving division method applied to a wide area (group) according to an embodiment of the present invention.
3 is a diagram illustrating a K-means partitioning scheme for a narrow region (group) according to another embodiment of the present invention.
FIG. 4 is a diagram illustrating a concept of changing cluster values for a divided region according to another embodiment of the present invention.
5 is a view showing a result of aerial image segmentation when only one parameter and region according to an embodiment of the present invention are considered.
FIG. 6 is a view showing a result of aerial image segmentation according to another embodiment of the present invention, in which six parameters are taken into consideration.
FIG. 7 is a graph illustrating the accuracy when the segmentation technique according to an embodiment of the present invention is applied.
FIG. 8 is a flow chart showing in detail the dividing step (S102) shown in FIG.
FIG. 9 is a flowchart showing details of the texton dictionary creation step (S103) shown in FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Thus, in some embodiments, well known process steps, well-known structures, and well-known techniques are not specifically described to avoid an undue interpretation of the present invention.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. It is to be understood that the terms comprise and / or comprise are used in a generic sense to refer to the presence or addition of one or more other elements, steps, operations and / or elements other than the stated elements, steps, operations and / It is used not to exclude. And "and / or" include each and any combination of one or more of the mentioned items.

Further, the embodiments described herein will be described with reference to cross-sectional views and / or schematic drawings that are ideal illustrations of the present invention. Thus, the shape of the illustrations may be modified by manufacturing techniques and / or tolerances. Accordingly, the embodiments of the present invention are not limited to the specific forms shown, but also include changes in the shapes that are generated according to the manufacturing process. In addition, in the drawings of the present invention, each component may be somewhat enlarged or reduced in view of convenience of explanation. Like reference numerals refer to like elements throughout the specification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a method for dividing an area-based aerial image according to an embodiment of the present invention and a storage medium for storing program codes for executing the method will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating an area-based aerial image segmentation process according to an exemplary embodiment of the present invention. Referring to FIG. 1, an area-based aerial image segmentation method according to the present invention includes an initialization step S101, a segmentation step S102, a texton dictionary creation step S103, a boosting step S104, (S105), and the like. Here, texton represents a symbol string existing in the text.

First, in the initialization step (S101), a filter bank (a set of filters) composed of the following filters is created. The elements used when creating the filter were the CIELab type color configuration, not the RGBs value.

1. Four Derivative Of Gaussian Filters:

Consider only L channel

=2,4x-axis directional filter:

= 2,4

=2,4
y-axis directional filter:

= 2,4

=1,2,4,82. Four Laplacian filters: Consider only L channel,

= 1,2,4,8

=1,2,43. Gaussian filter (9): Consider all three channels of Lab,

= 1,2,4

We apply the filter bank with a total of 17 filters to all the pixels in the data set to extract the filter response.

Next, in the dividing step (S102), different partitioning techniques are applied according to a wide area and a narrow area as shown in FIG. Considering three cases in a wide area and three cases in a narrow area, a total of six cases and six responses are obtained. Three cases in a large area are illustrated in FIG. 2, and three cases in a narrow area are shown in FIG.

The reason why we divide the area into a large area and the narrow area and then calculate the reaction is summarized in Figure 4. The size of the calculated area for only one parameter is prone to false value changes and errors for the pixel and neighboring pixels. Therefore, the partitioning technique should be applied considering bundles of various sizes (super pixels).

Further, in the case of the wide area 400, the pixel value in the sub-area 400-2 on the basis of the pixel 400-1 is "3" because there are many pixels having "3" in the peripheral area , "2" is changed to "3 ". That is, the sub area 400-2 is all changed to "3 ".

On the other hand, in the case of the narrow area 410, only the value of the pixel 410-1 in the sub area 410-2 is changed from "2" to "1" based on the pixel 410-1. That is, the sub region 410-2 is all changed to "1 ". Thus, there are a total of six reactions that change in one pixel.

As shown in FIG. 2, when the division technique is applied to a wide area of three cases, it selects an appropriate reference parameter value and two other parameter values (R _spatial , R _color ) within an allowable range based on the reference parameter value. do. The average moving method is applied to the selected parameter values to extract the reaction in three cases in a wide area. That is, according to the two parameter values (R _spatial = 5,7,9, R _color = 7) Three images 202, 203, and 204 are extracted from the original image 201. [

Then, applying the K-means division technique within a specific radius based on the color value, an effect like an abstract super-pixel can be obtained from the original image 301 of FIG. 3 as in 302. Of course, the original image 301 shown in Fig. 2 and the original image 301 shown in Fig. 3 are the same.

Here, superpixel means that pixels with high similarity near the pixel are grouped into small regions and act as one pixel. K-means partitioning is applied to all three cases within the allowable range based on the appropriate distance (Distance). By doing so, it is possible to extract responses for three cases (302, 303, 304) according to the area size according to the distance (Distance = 10, 15, 20).

Generally, given a data set D with n objects and k as the number of clusters, the partitioning algorithm divides the objects into k clusters. As a result, objects in the cluster are similar, and objects in different clusters do not. K-means partitioning is one such segmentation algorithm.

The reaction of the three cases through the division step (S810) using the K-means division method of FIG. 8 and the division step (S820) using the average movement division method are shown in the division step of FIG. 1 S102).

Referring to FIG. 1, a texton dictionary creation step (S103) is composed of three steps (S910, S920, S930) in detail as illustrated in FIG. The first is the step of learning the texture dictionary (S910), the second is the step of textureizing all the images by applying the KD tree method (S920), and finally the texture-layout filter And filtering (S930). Here, a k-dimensional tree is a space-division data structure for constructing points in a k-dimensional space, which is a special case of a binary tree.

In the Texon advance learning step (S910), the filter responses for each pixel of the image are clustered. The clustering technique used was K-means partitioning and the distance was measured using the Euclidian technique.

In the textonization step (step S920), which is based on the center point of the cluster derived from the texture learning step (S910), the KD tree algorithm maps the filter responses and the closest cluster center point through the filter bank To create a texton map.

과 이 직사각형 영역(ri) 내의 특성을 보여주는 텍스쳐(Texture)

에 대해 이전 단계(S920)에서 만들어진 텍스톤 맵(Textons map)의 임의의 위치에 해당하는 반응들과 비교하며 이때 텍스톤 맵(Textons map)에서

내의

가 차지하는 비중을 0과1 사이 확률로서 나타내어 반응정도를 얻는다. 예를 들면, r₁과 t₁을 갖는 피쳐(feature1)를 입력 영상에 위치한 경우 영상의 컬러가 t1의 컬러와 완전히 일치하면 반응 정도의 값은 "1"이고, 반정도만 일치하면 반응 정도의 값은 "0.5"이고, 완전히 다른 컬러이면 반응 정도의 값은"0"이 된다. 여기서, 직사각형 영역은 가로 직사각형, 또는 세로 직사각형 등으로 표현될 수 있다. In the texture-layout filtering step (S930), which is the last step, a rectangular region having a size of 100,

And a texture showing characteristics in the rectangular area ri.

Is compared with responses corresponding to arbitrary positions of the Textons map created in the previous step (S920) with respect to the Textons map,

undergarment

And the degree of reaction is obtained by expressing the specific gravity occupied by the probability as 0 and 1. For example, r ₁ and t if in a feature (feature1) having a _first input image when the color of the image completely matches the t1 color value of the reaction degree is "1" and, if they match only about half the value of the reaction degree Is "0.5 ", and if the color is completely different, the value of the reaction degree becomes" 0 ". Here, the rectangular area may be represented by a horizontal rectangle, a vertical rectangle, or the like.

Random-sized rectangular region features have {x, y, width, height} values. Where x and y represent position coordinates, and width and height represent the height and width of a random-sized square.

The probability that the specific gravity (v) occupied by t in the rectangular region r is expressed by a probability between 0 and 1 is expressed as the following equation.

Where _Tj is a texture.

Therefore, if all of the textures (T _j ) coincide with the entire region r, the value of v becomes 1.

Continuing referring to FIG. 1, in the boosting step (S104), a joint boosting technique is applied. The joint boosting technique finds weak learners and then calculates the error by applying the equation (2) to each learning image. This can be expressed by the following equation.

Here, w represents a weight, z represents a learner who has already been raised (answer), and h is a current weak learner calculated by the above reactions.

We choose a set of weak learners that minimize errors and update weak learners with strong learners by adding them to strong learners. And updates the weight based on the calculated weak learner value.

Therefore, since the joint boosting method is widely known, a further explanation will be omitted.

This process is repeated to extract six reactions for each pixel and the obtained responses are used in the final step of category determination step (S105).

In the category determination step (S105), the category value that occupies the highest ratio is finally selected to classify the category of the pixel.

At this time, besides using the texton feature obtained by boosting, it is also possible to use a feature derived from a commonly used SIFT (Scale Invariant Feature Transform) algorithm and a feature for a position commonly appearing in the image.

5 is a view showing a result of aerial image segmentation when only one parameter and region according to an embodiment of the present invention are considered.

FIG. 6 is a view showing a result of aerial image segmentation according to another embodiment of the present invention, in which six parameters are taken into consideration.

FIG. 7 is a graph illustrating the accuracy when the segmentation technique according to an embodiment of the present invention is applied. 7, an accuracy curve 710 of only Texton, an accuracy curve 720 of Texton + Loc (Locator), an accuracy curve 730 of Texton + Loc + LBP (Local Binary Pattern) + Loc + Local Binary Pattern (LBP) + multiple division accuracy curve 740 is shown.

In particular, the area-based aerial image segmentation method according to an embodiment of the present invention may be implemented in the form of program command codes that can be executed through various computer means and recorded in a computer-readable storage medium.

The computer-readable storage medium may include program instructions, data files, data structures, etc., alone or in combination.

The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software.

Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, floppy disks, Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

The medium may be a transmission medium such as an optical or metal line, a wave guide, or the like, including a carrier wave for transmitting a signal designating a program command, a data structure, or the like.

Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, one embodiment of the present invention may be implemented in hardware, software, or a combination thereof. (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a controller, a microprocessor, and the like, which are designed to perform the above- , Other electronic units, or a combination thereof.

In a software implementation, it may be implemented as a module that performs the functions described above. The software may be stored in a memory unit and executed by a processor. The memory unit or processor may employ various means well known to those skilled in the art.

201,301: Original video

Claims (13)

A method for segmenting an aerial image captured by an unmanned aerial vehicle into regions based on regions,
An initialization step of applying a filter bank, which is a set of filters, to pixels of the aerial image to extract filter responses;
A reaction obtaining step of dividing the filter reactions into a narrow region and a wide region and applying a division technique to obtain a reaction;
A texton map is generated by extracting a texton from each of the pixels of the aerial image, mapping the center points of each of the filter reactions and the cluster, generating a texture map by using the texture map, A texton dictionary generation step of generating a degree of text;
A boosting step of minimizing an error with respect to the degree of reaction and assigning a weight to extract final responses for each of the pixels of the aerial image; And
A category determination step of classifying the category of the pixel among the pixels of the aerial image according to the highest ratio among the extracted final responses;
Based aerial image segmentation method.
The method according to claim 1,
Wherein the texton dictionary creation step comprises:
Generating clusters of filter responses using a clustering technique and a distance measurement technique;
Generating a texture map by mapping each of the filter responses and a closest cluster center point using a KD tree technique; And
And obtaining a degree of reaction by comparing the responses with a specific position of the texture map generated by the specific number of the plurality of rectangular regions r _i and the texture t _i , doing Region - based aerial image segmentation method.
3. The method of claim 2,
The degree of the reaction is represented by the probability that the texture t _i occupies a plurality of rectangular areas r _i in the texture map between 0 and 1,

Wherein the area (r) represents an area of the rectangular area (r _i ), and Tj represents a texture representing a characteristic in the rectangular area. The method according to claim 1,
Characterized in that the filter bank comprises at least 17 filters, the constituent filter of the filter bank being a combination of a Derivative Of Gaussian Filter, a Laplacian Filter and a Gaussian Filter. A method of aerial image segmentation.
delete delete delete The method according to claim 1,
Wherein the components of the aerial image include at least one of an RGBs color scheme and a CIELab color scheme.
The method according to claim 1,
And a K-means segmentation scheme is applied using three different factor values for a narrow region, using an average moving method using three different factor values for a wide region, Split method.
The method according to claim 1,
Wherein the boosting step uses a joint boosting technique.
delete The method according to claim 1,
Wherein the final responses are the results of the six reactions, three are the results of the reactions in the broad zone and three are the results of the reactions in the narrow zone.
A computer-readable storage medium storing program code for executing an area-based aerial image segmentation method according to any one of claims 1 to 4, 8 to 10, and 12.

Images