JP7102106B2 - Machine learning method and surface change judgment method - Google Patents

Description

The present invention generates a model used in a system for determining ground surface fluctuations based on an interfering SAR image generated by interfering two-period SAR complex images obtained by a Synthetic Aperture Radar (SAR). The present invention relates to a machine learning method and a surface change determination method using the model.

Interferometric SAR analysis can measure the displacement of the ground surface smaller than the wavelength of the radar used for it, and is a useful analysis method for monitoring the ground surface movement in millimeters, for example. The analysis result by the differential interference SAR analysis is generally represented by an interference image (interferogram) in which the phase difference is shown by rainbow-colored stripes. Interpret the fluctuations.

Japanese Unexamined Patent Publication No. 2004-309178 Special Table 2003-500358 Gazette

It takes the skill of a reader to extract the fluctuation fringes due to the target surface movement from the interference image. Therefore, it is not easy to secure human resources for reading, and the cost and processing time required for reading increase. In addition, since the extraction criteria must be qualitative in visual interpretation, the interpretation results tend to vary, and it is not easy to ensure quality. And the above issues make it difficult to monitor surface changes over a wide area.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a machine learning method capable of automating the determination of the presence or absence of surface change based on an interference SAR image, and a method for determining surface change. do.

(1) In the machine learning method according to the present invention, the phase difference θ of each pixel of the interference SAR image obtained by interfering with the SAR images of the ground surface at two periods obtained by the synthetic aperture radar mounted on the flying object. A component image generation step for generating n types of component images (n is a predetermined number of 2 or more) in which pixel values are defined based on _S , and a user's interpretation result for a predetermined surface change are acquired. , A learning data generation step for generating learning data by combining the n types of component images and the interpretation result for a plurality of sample regions, and a variation determination model are generated by machine learning using the learning data. The component image having the model generation step and k (k is an arbitrary natural number of n or less) has the value of the function _pk (θ) at θ = θ _S as the pixel value, and each of the above. The function _pk (θ) is continuous with D, with the interval D of the width 2π radian corresponding to the range of the phase difference as [α-π, α ₊ π], and pk (α-π) = pk (α ₊ π). If the coordinates defined by the set of pk (θ) for all _k are expressed as P (θ), then θ ₁ ≠ θ ₂ other than both ends of D, ∀θ ₁ , θ ₂ ∈ D. On the other hand, P (θ ₁ ) ≠ P (θ ₂ ).

(2) In the machine learning method described in (1) above, the n is 2, and the functions are sinθ and cosθ.

(3) In the machine learning method according to (1) or (2) above, the fluctuation determination model can be configured by a convolutional neural network.

( 4 ) The surface change determination method according to the present invention determines the surface change in the target region using the change determination model generated by the machine learning method according to any one of (1) to ( 3 ) above. In the method, the phase difference θ _T in each pixel is extracted from the interference SAR image generated for the target region, and the value pk (θ _T ) ( _k ) of the function is an arbitrary natural number equal to or less than n. ) Is used as a pixel value, and the step of generating the n types of component images and the input data including the n types of component images for the target region are input to the fluctuation determination model, and the fluctuation is described for the target region. It has a step of obtaining a determination result of the presence or absence of.

According to the present invention, it is possible to automate the determination of the presence or absence of surface movement based on the interference SAR image.

It is a block diagram which shows the schematic structure of the learning system which concerns on embodiment of this invention. It is a schematic diagram which shows the schematic flow of the machine learning of the change determination model of the earth surface in embodiment of this invention. It is a graph which shows another example of the transformation function which generates the component image from the interference SAR image. It is a block diagram which shows the schematic structure of the surface change determination system which concerns on embodiment of this invention. It is a schematic diagram which shows the schematic flow of the change determination process of the earth surface in embodiment of this invention.

Hereinafter, the learning system 2 and the ground surface change determination system 4, which are embodiments of the present invention (hereinafter referred to as embodiments), will be described with reference to the drawings. The learning system 2 is a system that generates a variation determination model used in the ground surface variation determination system 4 by the machine learning method according to the present invention, based on the phase difference information obtained from the interference SAR image. On the other hand, the surface change determination system 4 is a system that determines the surface change in the target area by using the change determination model generated by the learning system 2 by the surface change determination method according to the present invention. With this system, for example, surface movements due to landslides and land subsidence can be determined and detected.

[Learning system]
FIG. 1 is a block diagram showing a schematic configuration of the learning system 2 according to the embodiment. The learning system 2 includes a learning operation unit 20, a learning storage unit 21, a learning control unit 22, and a learning output unit 23. The learning operation unit 20, the learning storage unit 21, and the learning output unit 23 are connected to the learning control unit 22.

The learning operation unit 20 is a user interface device for inputting to the learning control unit 22, and includes a keyboard, a mouse, and the like. The learning operation unit 20 is operated by the user, and in addition to inputting the reading result of the presence or absence of fluctuation in the sample area to the learning control unit 22, a learning start instruction and an output instruction of the generated fluctuation determination model are given to the learning control unit 22.

The learning storage unit 21 is a storage device such as a ROM, RAM, or hard disk, and stores programs and data used by the learning control unit 22. The learning storage unit 21 inputs and outputs these programs and data to and from the learning control unit 22. In the present embodiment, the data stored in the learning storage unit 21 includes interference SAR image data 211, terrain image data 212, interpretation result (GT data) 213, and variation determination model 214.

The interference SAR image data 211 is data of an interference SAR image generated by interfering two-period SAR complex images (SAR images) obtained by a synthetic aperture radar mounted on a flying object.

The topographic image data 212 is two-dimensional information representing topography such as a digital elevation model (DEM) and a topographic map.

The interpretation result 213 is correct answer data (Ground Truth: GT), and is generated by, for example, a user interpreting the interference SAR image data 211.

The interference SAR image data 211 and the terrain image data 212 are stored in the learning storage unit 21 in advance during learning by the learning system 2. In the present embodiment, the interpretation result 213 is configured to be acquired by the learning data generation process in the learning system 2, but it may be acquired prior to the learning process by the learning system 2 and stored in the learning storage unit 21. good.

The fluctuation determination model 214 is a learning model generated by the learning system 2, and is updated as the learning control unit 22 sequentially processes learning data of a plurality of sample areas by machine learning.

The learning control unit 22 is configured by using, for example, an arithmetic unit such as a CPU (Central Processing Unit). Further, as the arithmetic unit constituting the learning control unit 22, instead of the CPU, an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit) that executes image processing at high speed, or the like may be used. For example, each function according to the present embodiment may be realized by using GPGPU (General-Purpose computing on Graphics Processing Units), which is a technique for diverting the function of the GPU to an application other than image processing. Specifically, the learning control unit 22 is a computer, which reads a program from the learning storage unit 21 and executes it, and functions as a component image generation means 221, a learning data generation means 222, and a model generation means 223, which will be described later. .. Each means of the learning control unit 22 will be described later along with the processing of the learning control unit 22.

The learning output unit 23 includes a USB terminal for outputting the fluctuation determination model generated by learning to the outside of the learning system 2, an interface circuit such as a CD drive and a network adapter, and each driver program. In the present embodiment, the variation determination model is passed to the ground surface variation determination system 4 via the learning output unit 23. Further, the learning output unit 23 may include a user interface device such as a display and a printer that enables the user to grasp the operation of the learning control unit 22 and the result thereof.

FIG. 2 is a schematic diagram showing a schematic flow of machine learning of a surface change determination model. An interference SAR image is generated from the SAR image 300 of the second period by the interference SAR analysis process 304, and is stored in the learning storage unit 21 as the interference SAR image data 211.

Here, in the interference SAR analysis process 304, it is possible to perform a process of removing the influence other than the phase change (fluctuation fringe) due to the fluctuation of the ground surface from the interference SAR image. Specifically, in addition to the fluctuating fringes, the interferometric SAR image is superposed with phase changes (orbit fringes and topographic fringes) that occur because the orbits of the satellites at two periods do not match. Therefore, when the ground surface movement is obtained by the interference SAR, a process of removing the track fringes and the terrain fringes and extracting only the fluctuation fringes is performed. For example, terrain fringes can be simulated and removed from the DEM, for which the DEM 302 is used in the interference SAR analysis process 304.

Further, the error and noise may be further removed from the interference SAR image, such as removing the global error by using a removal filter or the like.

The learning control unit 22 functions as the component image generation means 221 and has n types of pixel values defined based on the phase difference θ _S of each pixel of the interference SAR image data 211 (n is a predetermined number of 2 or more). The component image of is generated and passed to the learning data generation means 222. In the present embodiment, the component image generation means 221 performs the process 306 of converting the phase difference θ _S into sin θ _S and cos θ _S , and uses the converted interference image 308 as the component image. Specifically, two types of component images are generated, a sin image whose pixel value is defined by sinθ _S and a cos image whose pixel value is defined by cosθ _S.

The learning control unit 22 functions as a learning data generation means 222, performs a process 310 of cutting out a learning patch image for each of a plurality of small areas set in the area where the interference SAR image data 211 is obtained, and performs learning. Generate data 312.

Here, the small area from which the learning patch image is cut out is referred to as a sample area. The sample area is set by the user, for example. By the way, the number of sample regions is set to a relatively large number so that a fluctuation judgment model with sufficient accuracy can be obtained by learning. In addition, the size and shape of each sample area are basically the same.

The learning data 312 is generated for each sample region by combining the converted interference image 308, which is n types of component images, and the user's interpretation result regarding the presence or absence of surface fluctuations. Therefore, the learning data generation means 222 performs the process 314 for acquiring the interpretation result. Specifically, as the user interpretation process 314, for example, the entire area of the interference SAR image data 211 is displayed on the display of the learning output unit 23, the user is made to interpret the presence or absence of fluctuation, and the interpretation result 213 is displayed from the learning operation unit 20. To get. In the interpretation process 314, in addition to the interference SAR image data 211, the terrain image data 212 and other data obtained by a high-precision sensor such as LiDAR (Light Detection and Ranging) and the local GPS (Global Positioning System) are used. The data obtained by the measurement used can be presented to the user as the data 316 of the judgment material. These data can be used together for reading processing 314, for example, by combining the interference SAR image 211 and the terrain image data 212 or the like. Further, when measurement data higher than the interference SAR can be obtained by LiDAR, GPS, or the like, the data can be used alone for the interpretation process 314 to generate the interpretation result 213. For example, the user marks a fluctuating part and generates a reading result (GT data) while performing operations such as enlarging / reducing the display magnification of these data. The generated interpretation result 213 is stored in the learning storage unit 21.

When the interpretation result 213 is obtained, the learning data generation means 222 is obtained from the converted interference image 308 and the interpretation result 213, which are n kinds of component images, obtained in the region corresponding to the interference SAR image data 211. The data of the portion corresponding to each sample area is cut out as a patch image to generate the training data 312. The learning data 312 can also be combined with other information useful for analysis of fluctuations. For example, in the present embodiment, the patch image of the terrain image data 212 is included in the learning data 312.

The learning control unit 22 functions as a model generation means 223, and generates a variation determination model 214 by machine learning using the learning data 312 generated for a large number of sample regions. Specifically, the machine learning in the present embodiment is performed by deep learning 318 using a convolutional neural network (CNN), and the learned CNN becomes the fluctuation determination model 214. In CNN, the sample area is basically rectangular.

One of the features of the learning system 2 is that the interference SAR image data 211 is not used as it is for machine learning, but preprocessing for converting the interference SAR image data 211 into a component image is performed, and the learning process is performed using the component image. There is a point to do.

Here, the function for converting the phase difference θ at each pixel value of the interference SAR image data 211 into the pixel value of the component image of the kth (k is an arbitrary natural number of n or less) is expressed as _pk (θ). In the above-described embodiment, the sin image is used as the first component image and p ₁ (θ) = sin θ, and the cos image is used as the second component image and p ₂ (θ) = cos θ.

Where the two phase states η ₁ and η ₂ in which the difference between the phases θ ₁ and θ ₂ is 2π radians [rad] are physically the same, in the learning using the interference SAR image data 211, θ ₁ and θ Since ₂ is a different value, the phase states η ₁ and η ₂ are treated separately. On the other hand, in learning using the component image, the set of function values (sinθ, cosθ) associated with the phase difference θ is (sinθ ₁ , cosθ ₁ ) = (sinθ ₂ , cosθ ₂ ), and the phase state. It is preferable in that η ₁ and η ₂ are physically the same, and the determination accuracy is improved in the fluctuation determination model generated by the learning.

The function pk (θ) of _n kinds of component images (n ≧ 2 as described above) from which the effect of such pretreatment can be obtained basically has the following features (1) to (3). Here, the range [α−π, α + π] of the width 2πrad that defines the phase difference θ is defined as the interval D. For example, when θ is defined as 0 to 2π, α = π and D = [0,2π], and when θ is defined as −π to π, α = 0 and D = [ -Π, π].

(1) Each function _pk (θ) is continuous in the interval D (function continuity)
(2) Each function p _k (θ) has the same value at both ends of the interval D, that is, p _k (α − π) = p _k (α + π) (periodicity of the function).
(3) When the coordinates defined by the set of pk (θ) for all _k are expressed as P (θ), P (θ) is given to different values of θ in the section D except for both ends of the section D. ) Does not match, that is, P (θ ₁ ) ≠ P (θ ₂ ) for ∀θ ₁ , θ ₂ ∈ D such that θ ₁ <θ ₂ unless θ ₁ = α − π and θ ₂ = α + π. ) (Single-shotness of the function).

The feature (2) is described as "periodicity" in the sense that the value of _pk (θ) returns to the original value when θ changes from one end to the other end of the section D. Therefore, if the function p _k (θ) is a periodic function with a period of 2π, it has the characteristic, whereas the function p _k (θ) need only be defined in the interval D and does not need to be defined outside the interval D. , The function _pk (θ) may not be a periodic function with a period of 2π.

FIG. 3 is a graph showing an example of the function pk (θ) having the above-mentioned features (1) to (3), and _n kinds of component images are displayed using the set of the functions _pk (θ) shown in the example. It can also be defined. The example shown in FIG. 3 (a) is the case of n = 2, and p ₁ (θ) and p ₂ (θ) are expressed by the following equations. Note that 0 <β <π.

FIG. 3B shows an example in the case of n = 3, and p ₁ (θ) to p ₃ (θ) are represented by the following equations.
p ₁ (θ) = sin 2θ
p ₂ (θ) = sin 3θ
p ₃ (θ) = cos θ

In the above-described embodiment, an example is shown in which the learning data includes terrain image data 212 in addition to n types of component images and interpretation results. This configuration can be expected to improve the accuracy of the generated variation determination model. On the other hand, the learning data may be configured not to include the terrain image data 212.

Further, in the above-described embodiment, the example in which machine learning uses CNN has been described, but other algorithms may be used.

[Ground surface change judgment system]
FIG. 4 is a block diagram showing a schematic configuration of the surface change determination system 4 according to the embodiment. The ground surface change determination system 4 includes an input unit 40, a storage unit 41, a control unit 42, and an output unit 43. The input unit 40, the storage unit 41, and the output unit 43 are connected to the control unit 42.

The input unit 40 is a user interface device for inputting to the control unit 42, and includes a keyboard, a mouse, and the like. The user operates the input unit 40, specifies, for example, data to be subjected to the fluctuation determination and a target area, and instructs the control unit 42. Further, the input unit 40 includes a USB terminal for inputting the fluctuation determination model 214 from the learning system 2, an interface circuit such as a CD drive and a network adapter, and each driver program.

The storage unit 41 is a storage device for a ROM, RAM, hard disk, etc., and stores programs and data used by the control unit 42. The storage unit 41 inputs / outputs these programs and data to / from the control unit 42. In the present embodiment, the data stored in the storage unit 41 includes interference SAR image data 411, terrain image data 412, and variation determination model 413.

The interference SAR image data 411 is the data of the interference SAR image like the interference SAR image data 211 used in the learning system 2, and is the data of the ground surface range including the target area for determining the fluctuation.

The terrain image data 412 is two-dimensional information representing terrain such as a DEM and a topographic map, like the terrain image data 212 of the learning system 2, and is data of a ground surface range including a target area for determining fluctuation.

The interference SAR image data 411 and the terrain image data 412 are stored in the storage unit 41 in advance when the change is determined by the ground surface change determination system 4.

The variation determination model 413 is a learning model generated by the learning system 2, and the variation determination model 214 stored in the learning storage unit 21 is introduced and used.

The control unit 42 is configured by using, for example, an arithmetic unit such as a CPU. Further, the arithmetic unit constituting the control unit 42 may use an MPU, GPU, or the like instead of the CPU. Further, for example, GPGPU may be used to realize each function according to the present embodiment. Specifically, the control unit 42 is a computer, which reads a program from the storage unit 41 and executes it, and functions as a component image generation means 421 and a determination means 422, which will be described later. Each means of the control unit 42 will be described later along with the processing of the control unit 42.

The output unit 43 is a user interface device such as a display or a printer that enables the user to grasp the operation of the control unit 42 and the result thereof.

FIG. 5 is a schematic diagram showing a schematic flow of the surface change determination process using the surface change determination system 4. The process of generating the interference SAR image data 411 by the interference SAR analysis process 504 from the SAR images 500 and DEM 502 of the two periods is basically the process of generating the interference SAR image data 211 described with reference to FIG. 2 for the learning system 2. Can be the same. In the present embodiment, the ground surface change determination system 4 assumes that the interference SAR image data 411 generated in this way is given as the processing target data, but the surface change determination system 4 uses the SAR image 500 to interfere SAR image data 411. May be performed.

The control unit 42 functions as the component image generation means 421, and generates n types of component images whose pixel values are defined based on the phase difference θ _T of each pixel of the interference SAR image data 411. The processing content of the component image generation means 421 is the same as the processing content of the component image generation means 221 of the learning control unit 22 that generated the variation determination model 413. That is, the conversion process (function pk (θ)) for the phase difference θ when generating the component image in the component image generation means 421 and the number _n of the types of the component image are the numbers n of the types of the component image of the learning system 2 that generated the variation determination model 413. It is common with the component image generation means 221. In the present embodiment, the component image generating means 421 corresponds to the process 306 of FIG. 2 in which the component image generating means 221 converts the phase difference θ _S into sin θ _S and cos θ _S , and the component image generating means 421 has the phase difference θ _T of the interference SAR image data 411. Is performed to convert to sin θ _T and cos θ _T , and as the converted interference image 508 which becomes a component image, a sin image whose pixel value is defined by sin θ _T and a cos image whose pixel value is defined by cos θ _T Two types are generated.

The component image generation means 421 performs a process 510 of cutting out a portion of the target region for determining the presence or absence of fluctuation from the component image (converted interference image 508) as a patch image 512. Here, when the terrain image data 212 is included in the learning data in the learning system 2, the patch image 512 for the terrain image data 412 is also generated by the cutting process 510. Regarding the designation of the target area, for example, the user can specify a specific location within the ground surface range to which the interference SAR image data 411 is given, and the control unit 42 sequentially designates the target area over the entire ground surface range. You may perform the processing to do.

The control unit 42 functions as a determination means 422 and determines whether or not there is a predetermined change in the ground surface in the target area. Specifically, the determination means 422 performs the determination process 514 using the patch image 512 of the target area as the input data of the variation determination model 413, and determines the presence or absence of landslide, which is the variation to be determined based on the output, for example. Generate a landslide candidate location map 516.

2 Learning system, 4 Ground movement judgment system, 20 Learning operation unit, 21 Learning storage unit, 22 Learning control unit, 23 Learning output unit, 40 Input unit, 41 Storage unit, 42 Control unit, 43 Output unit, 211,411 Interference SAR image data, 212,412 terrain image data, 213 interpretation results, 214,413 fluctuation determination model, 221 and 421 component image generation means, 222 learning data generation means, 223 model generation means, 422 judgment means.

Claims (4)

N types of pixel values defined based on the phase difference θ _S of each pixel of the interference SAR image obtained by interfering with each other of the SAR images of the ground surface for two periods obtained by the synthetic aperture radar mounted on the flying object. A component image generation step for generating a component image of (n is a predetermined number of 2 or more), and
A learning data generation step of acquiring a user's interpretation result for a predetermined surface change and generating learning data by combining the n types of component images and the interpretation result for a plurality of sample areas.
It has a model generation step of generating a fluctuation determination model by machine learning using the training data.
In the component image of the kth (k is an arbitrary natural number of n or less), the value of the function _pk (θ) at θ = θ _S is used as the pixel value.
Each of the functions p _k (θ) is continuous with D, with the interval D of the width 2π radian corresponding to the range of the phase difference as [α-π, α + π], and p _k (α-π) = p _k ( α + π), and if the coordinates defined by the set of pk (θ) for all _k are expressed as P (θ), then θ ₁ ≠ θ ₂ other than both ends of D, ∀ θ ₁ , θ ₂ That P (θ ₁ ) ≠ P (θ ₂ ) for ∈ D,
A machine learning method characterized by. In the machine learning method according to claim 1,
A machine learning method characterized in that n is 2, and the functions are sinθ and cosθ. In the machine learning method according to claim 1 or 2.
A machine learning method characterized in that the fluctuation determination model is composed of a convolutional neural network. A method of determining surface movements in a target area using the fluctuation determination model generated by the machine learning method according to any one of claims 1 to 3.
The phase difference θT in each pixel is extracted from the interference SAR image generated for the target region, and the value of the function p _k (θ _T ) (k is an arbitrary natural number of n or less) is used as the pixel value. And the step of generating the n kinds of component images
A step of inputting input data including the n types of component images of the target region into the fluctuation determination model and obtaining a determination result of the presence or absence of the fluctuation in the target region.
A method for determining surface change, which is characterized by having.

Images