KR20140085964A - Apparatus and method for topographical change detection using aerial images photographed in aircraft - Google Patents

Abstract

Provided is an apparatus to detect a topographic change using an aircraft. The apparatus includes: a first calculation unit which generates a difference image between an image of the terrain by the first time the aircraft captures and an image of the terrain by the second time the aircraft captures; and a second calculation unit which generates a change mask by binarizing the difference image.

Description

[0001] APPARATUS AND METHOD FOR TOPOGRAPHICAL CHANGE DETECTION USING AERIAL IMAGES [0002] PHOTOGRAPHED IN AIRCRAFT [0003]

The present invention relates to an apparatus and method for detecting terrain change using an image, and more particularly to an apparatus and method for providing terrain change detection using an aerial image taken on an aircraft.

In Korea, due to rapid industrialization such as economic development plan, the development of the national industry and the rapid development of the land have been achieved as the era of high growth began. As a result, the standard of living has also been improved to achieve sustainable economic growth.

However, due to the development-oriented policies for simple economic development, overcrowding of the metropolitan area, imbalance between regions, indiscriminate development of the land, and destruction of the environment are becoming problems.

Although the government has solved the above problems and established a comprehensive national land plan for the balanced development of the land, and puts enormous budget and manpower into it, it is necessary to establish measures for systematic and comprehensive management of the land.

In order to establish the above measures, it is necessary to establish an accurate understanding of the land, an efficient data collection for the development plan, a status analysis and management system. Therefore, continuous observation of topographic changes and efficient information collection method are needed to enable detection of topographic changes in the land and quantitative analysis of natural phenomena.

In addition, as well as the geographical survey related to the land as mentioned above, it is necessary to identify the damaged areas due to natural disasters such as typhoons, to crack down on illegal buildings in compensation for redevelopment, and to exploit the terrain change do.

According to one aspect, there is provided an apparatus for detecting a terrain change using an aerial image taken on an aircraft. The terrain change detection apparatus includes a first calculation unit for generating a difference image by subtracting a first image obtained by aerial photographing at a first point of time and a second image obtained by aerial photographing the terrain at a second point of time, And a second calculation unit for generating a change mask.

According to an embodiment, the terrain change detection apparatus may further include a processing unit for estimating a global homography of at least one of the first image and the second image.

According to one embodiment, the processing unit of the terrain change detection apparatus may reduce the size of at least one of the first image and the second image using a Gaussian kernel, and apply the RANSAC algorithm to the global It is possible to estimate the red homography.

According to an embodiment, the first calculation unit of the terrain change detection apparatus may generate the difference image on a block-by-block basis by comparing each of the first image and the second image on a block-by-block basis.

According to an embodiment of the present invention, the terrain change detection apparatus may further include a noise removing unit for removing a region having a size smaller than a threshold value among the detection regions included in the change mask to perform morphological noise removal.

According to one embodiment, the terrain change detection apparatus stitches a plurality of first sub images aerial photographed at the first viewpoint for each of a plurality of spatially divided regions of the terrain to generate the first image And a stitching unit for stitching a plurality of second sub images photographed at the second time point for each of the plurality of areas to generate the second image.

According to an embodiment, the stitching unit may perform brightness remapping of each of the plurality of first sub images or each of the plurality of second sub images before the stitching.

According to one embodiment, the stitching unit may perform the stitching by extracting and comparing Scintillation Invariant Feature Transformation (SIFT) feature points in the stitching process.

According to one embodiment, the stitching unit performs seamless processing at a boundary between the plurality of first sub images or at a boundary between the plurality of second sub images using at least one technique of interpolation and blending Can be performed.

According to one embodiment, the stitching unit may perform multiband blending in which the low frequency component is relatively broadly blended and the high frequency component is relatively narrowly blended in the blending.

According to an embodiment, the stitching unit may search for a marker area that is not related to the terrain during the stitching process, and may remove the marker area by in-painting the marker area.

According to another aspect, there is provided a method of detecting a change in terrain using an aerial image taken on an aircraft. The terrain change detection method includes generating a difference image by subtracting a first image obtained by aerial photographing a terrain at a first time point and a second image obtained by aerial photographing the terrain at a second time point by binarizing the difference image, For example.

According to an embodiment of the present invention, the topographic change detection method may further include performing morphological noise removal by removing a region having a size smaller than a threshold value among the detection regions included in the change mask.

According to an embodiment of the present invention, the terrain change detection method stitches a plurality of first sub images aerial photographed at the first viewpoint for each of a plurality of regions spatially dividing the terrain to generate the first image And generating the second image by stitching a plurality of second sub images aerial photographed at the second time point for each of the plurality of areas.

1 is a block diagram of a topographic change detection device, according to one embodiment.
FIG. 2 is a diagram illustrating a plurality of sub-images shot on an aircraft, according to an embodiment.
3 (a) and 3 (b) illustrate an embodiment for performing brightness remapping for each of the plurality of images, according to an embodiment.
FIG. 4 illustrates an embodiment for extracting and comparing SIFT feature points for each of the plurality of images, according to an exemplary embodiment. Referring to FIG.
FIG. 5 is a diagram illustrating a first image obtained by aerial photographing a terrain at a first viewpoint and a second image obtained by aerial photographing the terrain at a second viewpoint, according to an embodiment.
6 is a diagram illustrating an example of generating a difference image by performing a difference between the first image and the second image according to an embodiment.
7 is a diagram illustrating an embodiment for generating a change mask by binarizing the generated difference image, according to an embodiment.

In the following, some embodiments will be described in detail with reference to the accompanying drawings. However, it is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

Although the terms used in the following description have selected the general terms that are widely used in the present invention while considering the functions of the present invention, they may vary depending on the intention or custom of the artisan, the emergence of new technology, and the like.

Also, in certain cases, there may be terms chosen arbitrarily by the applicant for the sake of understanding and / or convenience of explanation, and in this case the meaning of the detailed description in the corresponding description section. Therefore, the term used in the following description should be understood based on the meaning of the term, not the name of a simple term, and the contents throughout the specification.

Since the change of the topography shows a wide variety of shapes in time and space, it is required to properly monitor and acquire the topographic change information. The present invention can utilize an aerial image photographed on an aircraft to solve the above-mentioned problems.

In addition, terrain change detection technology can be used in various fields such as calculation of damage area in case of disasters, calculation of damage area in the case of forest fire, calculation of development area according to land development, and monitoring of movement of enemy units.

According to an embodiment of the present invention, it is possible to detect the change of the terrain by using two images photographed with time difference with respect to the terrain, buildings, and the like photographed by the camera. The two images may be a still image or a moving image. In order to compare the two images, there may exist an image (for example, a first image) photographed first in time, and an image (such as a second image) photographed at a later time with a certain time difference .

The type of the image according to an exemplary embodiment may be one image or an image or moving image in which an image stitching operation is performed to compare a large area.

According to one embodiment, the stitching time of two 640 x 480 UAV images is within about 2 to 3 seconds, and can be doubled as the image is added. The time required for image change detection (image change detection) by matching three or four images can be maintained at about 15 seconds or less.

1 is a block diagram of a topographical change detection device 100, in accordance with one embodiment. The terrain change detecting apparatus may include a stitching unit 110, a first calculating unit 120, a processing unit 130, a second calculating unit 140, and a noise removing unit 150.

According to an exemplary embodiment, a first image obtained by aerial photographing a terrain at a first viewpoint and a second image aerial photographing the terrain at a second viewpoint may include a plurality of image regions spatially divided. For example, the first image may include a plurality of first sub images aerial photographed at a first viewpoint, and the second image may include a plurality of second subviews aerographically photographed at a second viewpoint, Images.

The stitching unit may stitch a plurality of first sub images aerial photographed at the first viewpoint to generate the first image. In addition, the second image may be generated by stitching a plurality of second sub images aerial captured at the second time point.

The stitching unit according to an embodiment may perform a preprocessing process on each of the plurality of first sub images and second sub images before stitching, which will be described in more detail below.

The first calculation unit according to an embodiment may generate a difference image by subtracting the first image and the second image generated by the stitching unit. The processing unit may estimate a global homography of at least one of the first image and the second image.

The second calculation unit according to an embodiment may generate a change mask by binarizing the difference image generated by the first calculation unit. The noise removing unit may remove morphological noise by removing a region having a size smaller than a threshold value among the detection regions included in the change mask.

FIG. 2 is a diagram illustrating a plurality of sub-images shot on an aircraft, according to an embodiment. As described above, the terrain change detection apparatus according to an embodiment uses two images (for example, a still image or a moving image) photographed with a time difference with respect to a terrain (or a building or the like) And a function of extracting a changed portion between the two. The two images may be images captured in time, such as the first image and images taken later in time, such as the second image.

According to an embodiment, the first image may be generated by stitching a plurality of first sub images aerial photographed at a first viewpoint on each of a plurality of regions spatially dividing the terrain.

In contrast, the second image may be generated by stitching a plurality of second sub images aerial photographed at a second point in time for each of a plurality of regions spatially dividing the terrain.

2 (a), 2 (b), 2 (c), and 2 (d) illustrate an example in which a plurality of second sub-images included in the second sub- Respectively. Each of the second subs shown in the figure is stitched together by the stitching unit to generate the second image. The method for such stitching is described in more detail below.

3 (a) and 3 (b) illustrate an embodiment for performing brightness remapping for each of the plurality of images, according to an embodiment.

According to an embodiment, the stitching unit may perform brightness remapping for each of the plurality of first sub images or each of the plurality of second sub images before stitching.

The brightness remapping is a technique of similarly matching brightness distributions to two input images. FIG. 3 shows an example of performing brightness remapping as an example for FIGS. 2 (a) and 2 (b).

As mentioned above, (a) and (b) of FIG. 3 may be the second sub images constituting the second image. However, the input sub-images may have different brightness distributions depending on the viewpoint. For example, it can be assumed that FIG. 3 (a) is darker than FIG. 3 (b) and FIG. 3 (b) has an appropriate brightness.

Therefore, in the case of FIG. 3 (a), the sub-image whose brightness is adjusted as shown in FIG. 3 (b) through the brightness remapping process can be adjusted brightly. As such, by performing brightness remapping on the sub images before stitching, the brightness distribution can be similarly matched, and the sub images can be stitched to produce a single image that is clearer.

FIG. 4 illustrates an embodiment for extracting and comparing SIFT feature points for each of the plurality of images, according to an exemplary embodiment. Referring to FIG. The stitching unit may perform stitching by extracting and comparing Scale Invariant Feature Transformation (SIFT) feature points in the stitching process.

According to one embodiment, the transformation relation between the images is referred to as homography, and a transformation relation can be obtained through a corresponding point between the images. In calculating the corresponding points, it is most important to extract the feature points that are most stable and can display the characteristics of the images. Therefore, when extracting the feature points, SIFT (Scale Invariant Feature Transformation) feature points can be used. The SIFT suffers from rotation and size conversion, and is relatively stable to brightness change, and is the most widely used feature point extraction algorithm at present.

In addition, the conversion relation between the images can be largely divided into Euclidean, similarity, affine and projective conversion. Of these, the projective transformation has the highest degree of freedom and can express various forms. In the present invention, the project conversion may be used to generate a natural mosaic image.

The key to the conversion relationship between images, i.e., the problem of obtaining the homography, is to reduce the influence on outliers. In addition, outliers of about 10% of the corresponding points may be significantly degraded. Therefore, by using RANSAC (Random Absolute Consensus Set) technique, outliers can be removed by discriminating the most reliable feature points among the feature points.

If homography estimation is a technique for increasing the geometrical similarity between images, interpolation and blending techniques may be a kind of rendering technique that makes the matched image results more natural. The interpolation technique mainly refers to an interpolation method that fills a hole in a panoramic image generated because there is no data information in the project conversion process. In the present invention, bilinear interpolation, which is fast and effective in processing time, can be used.

According to one embodiment, another technique of the rendering technique in addition to the interpolation technique is a technique used to remove a seam occurring at a boundary when different images are combined. The blending technique is a technique of naturally mixing a seam of a boundary portion. In the present invention, by using the multi-band blending technique in the blending technique, it is possible to minimize the error of detailed components around the line by widening the low frequency component and narrowly mixing the high frequency component in the boundary line attention (or performing seamless processing can do).

As shown in FIG. 4, a plurality of the second sub images may be stitched to generate the second image. And a brightness remapping process so that the brightness of the images become similar before the second sub images are stitched.

As shown in FIG. 4, when the sub images are stitched, a complex part 410 between the image and the image can be seamlessly processed by narrowly mixing high-frequency components by the above-described multi-band blending technique. On the contrary, the portion 420 in which the boundary between the image and the image is relatively simple can be seamlessly processed by mixing the low frequency components with the multiband blending technique described above.

According to one embodiment, the stitching unit can search for a marker area that is not related to the terrain during the stitching process, and can remove the marker area by inpainting the marker area.

According to one embodiment, when images including a plurality of markers are connected to each other, particularly, when providing flight information to an operator in an aerial image, most of the information may be mixed with the original image.

However, such information can be detrimental to visual processing as well as processing steps in the image analysis process. In addition, the marking can be removed using the imprinting technique because it can lead to poor results in detection of future topographic changes.

FIG. 5 is a diagram illustrating a first image obtained by aerial photographing a terrain at a first viewpoint and a second image obtained by aerial photographing the terrain at a second viewpoint, according to an embodiment. As described above, the terrain change detection apparatus and method of the present invention can use two images that are photographed with an appropriate time difference for an aerial photographing terrain (or a building, etc.). The two images may be still images or moving images.

In order to compare the two images, the terrain change detection apparatus according to an exemplary embodiment of the present invention includes a first image captured in time in the first time and a second image captured in time in FIG. 5 (a) For example, the second image photographed at the second point in time, as shown in (b) of FIG.

5A and 5B, the two vinyl houses 501 and 502, which are not present in the first image photographed at the first time point, and the container greenhouse 503 ) Is present in the second image photographed at the second point in time.

There is basically a time difference between two images (the first image and the second image) provided to obtain a difference image between the first image and the second image according to an embodiment, The point of time of exposure and the state of exposure may be very different.

Therefore, it is possible to use a pyramid that reduces the image size and obtains the relationship using the Gaussian kernel. By using the pyramid, the global homography between heterogeneous images can be estimated using RANSAC (Random Access Consensus) by reducing each image to only one level.

 According to one embodiment, finding the difference value for each overlapping area obtained through geometric matching and brightness correction as described above may be referred to as differencing. Generally, pixel-based difference values are obtained for low-difference images, but the geometric difference is large because the images in the case of the detection of the topographic change include a considerably large area.

Therefore, the terrain change detection apparatus and method can use a block-based difference image method that is not pixel based. The generation of the difference image and the generation of a change mask by binarizing the difference image will be described in detail below with reference to the drawings.

6A is a diagram illustrating an example of generating a difference image by subtracting the first image and the second image from each other according to an embodiment.

As mentioned earlier, it can be said that finding the difference value for each overlapping sub-region obtained through the above-mentioned geometric matching and brightness remapping is referred to as differencing.

For example, the first image aerial photographed at the first point of time and the second image aerial photographed at the second point of view shown in FIG. 5 (b), shown in FIG. 5 (a) If it is calibrated, the difference image as shown in Fig. 6 (a) can be obtained.

6 (b) is a diagram illustrating an embodiment for generating a change mask by binarizing the generated difference image, according to an embodiment.

The second calculation unit according to the embodiment may generate the change mask as shown in FIG. 6B by binarizing the difference image as shown in FIG. 6A generated by the above process . The region indicated by the change mask may be a region of interest, which detects the difference between the heterogeneous images.

As described above, even if the binary image is generated, noises may remain in the sikik image. In general, since an object of a certain size or less can be assumed to be noise, a morphology method can be used to remove noise. That is, since the operator does not take into consideration an object of a certain size or less by adjusting the threshold value, the terrain change detection donation can be enhanced.

As described above, the noise removing unit according to the embodiment may remove morphological noise by removing a region having a size smaller than the threshold value among detection regions included in the change mask. In addition, the threshold value may be adjustable by the operator.

7 is a flow diagram of the method for detecting topographic change according to one embodiment.

According to an embodiment, the first image and the second image may be aerial photographed at a predetermined time difference. First, a plurality of sub images related to the first image and the second image may be acquired by aerial photographing (710).

For example, the first image may be generated by processing a plurality of first sub images aerial photographed at a first viewpoint, and the second image may be generated by processing a plurality of aerial photographed aerial photographs at a second viewpoint, 2 < / RTI > sub-images. This process is described in detail in the following steps.

The stitching unit may stitch a plurality of first sub-images aerial photographed at the first point in time to generate the first image. Before the stitching process, Remapping may be performed 720. The brightness remapping according to an exemplary embodiment may be a process of similarly matching the brightness distribution through preprocessing when the input images have different brightness distributions according to the viewpoints.

After the brightness distribution between the sub-images is appropriately adjusted through the brightness remapping process, feature points may be extracted 730 so that the SIFT feature points can be stabilized between the images and can be well characterized. Here, the conversion relation between the images is called homography, and the conversion relation can be obtained through the correspondence point between the images.

The SIFT according to an exemplary embodiment suffers from rotation and size conversion, and is relatively stable to brightness variations.

In step 740, the first image or the second image may be generated by stitching a plurality of sub regions through the stitching process.

The first calculation unit of the terrain change detection apparatus may subtract the first image and the second image, which are two images having a time difference, with each other (750). As described above, the first image and the second image are separated from each other to generate a difference image of the difference between the two images.

The second calculator according to an exemplary embodiment may generate the change mask by binarizing the difference image (760). Although not shown in the flowchart of the topographical change detection method of FIG. 7, after generating the change mask, a region having a size smaller than the threshold value among the detection regions included in the change mask by the noise removing unit is removed, Noise removal can be performed. According to one embodiment, the threshold may be preset by the operator.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (15)

A first calculation unit for generating a difference image by subtracting a first image obtained by aerial photographing a terrain at a first point of time and a second image obtained by aerial photographing the terrain at a second point of time; And
A second calculation unit for binarizing the difference image to generate a change mask,
And a terrain change detection device. The method according to claim 1,
A processor for estimating a global homography of at least one of the first image and the second image,
Further comprising: a terrain change detection device for detecting the terrain change. 3. The method of claim 2,
Wherein the processing unit reduces the size of at least one of the first image and the second image using a Gaussian kernel and estimates the global homography by applying a RANSAC algorithm. The method according to claim 1,
Wherein the first calculation unit compares each of the first image and the second image on a block basis to generate the difference image on a block basis. The method according to claim 1,
A noise removing unit for removing a region having a size smaller than a threshold value among the detection regions included in the change mask to perform morphological noise removal;
Further comprising: a terrain change detection device for detecting the terrain change. The method according to claim 1,
Stitching a plurality of first sub images aerial photographed at the first viewpoint for each of a plurality of spatially divided regions of the terrain to generate the first image, and for each of the plurality of regions, And a stitching unit for stitching a plurality of second sub images photographed at a time point of the second image to generate the second image,
Further comprising: a terrain change detection device for detecting the terrain change. The method according to claim 6,
Wherein the stitching unit performs brightness remapping of each of the plurality of first sub images or each of the plurality of second sub images before stitching. The method according to claim 6,
Wherein the stitching unit performs the stitching by extracting and comparing Scale Invariant Feature Transformation (SIFT) feature points in the stitching process. The method according to claim 6,
The stitching unit may include at least one of a terrain change detection unit that performs seamless processing at a boundary between the plurality of first sub images or at a boundary between the plurality of second sub images using at least one of interpolation and blending, Device. 10. The method of claim 9,
Wherein the stitching unit performs multiband blending in which the low frequency component is relatively broadly blended and the high frequency component is relatively narrowly blended in the blending. The method according to claim 6,
Wherein the stitching unit searches for a marker area that is not related to the terrain in the stitching process and removes the marker area by in-painting the marker area. Generating a difference image by subtracting a first image aerial photographed at a first point of time and a second image aerial photographed at a second point of time; And
A step of binarizing the difference image to generate a change mask
And detecting the change of the terrain. 13. The method of claim 12,
Performing morphological noise removal by removing a region having a size smaller than a threshold value among detection regions included in the change mask
And detecting the change of the terrain. 13. The method of claim 12,
Stitching a plurality of first sub images aerial photographed at the first viewpoint for each of a plurality of spatially divided regions of the terrain to generate the first image, and for each of the plurality of regions, Stitching a plurality of second sub images photographed at a time point to generate the second image
And detecting the change of the terrain. A computer-readable recording medium containing a program for performing the terrain change detection method of any one of claims 12 to 14.

Images