WO2020066755A1

WO2020066755A1 - Learning model generation device, ground object change determination device, learning model generation method, ground object change determination method, and computer-readable recording medium

Info

Publication number: WO2020066755A1
Application number: PCT/JP2019/036411
Authority: WO
Inventors: 喜宏山下
Original assignee: Ｎｅｃソリューションイノベータ株式会社
Priority date: 2018-09-28
Filing date: 2019-09-17
Publication date: 2020-04-02
Also published as: JP7294678B2; JPWO2020066755A1

Abstract

A ground object change determination device 20 is provided with: an image acquisition unit 21 that acquires pair images obtained by photographing a specific area from the sky at different times; a three-dimensional data acquisition unit 22 that acquires, for each of the pair images, three-dimensional data storing, in each pixel, information about the height of a ground object which is a photographing subject; a difference data creation unit 23 that, for each pair of corresponding pixels in the pair images, obtains a difference of pixel values and a difference of heights stored in the three-dimensional data, and creates difference data including the obtained differences; and a determination unit 24 that applies the difference data to a learning model obtained by performing machine learning of a relation between the difference data of the pair images and the change of the ground object, thus determining the change of the ground object in the specific area.

Description

Learning model generation device, feature change determination device, learning model generation method, feature change determination method, and computer-readable recording medium

The present invention relates to a feature change determining device and a feature change determining method for determining a change in a feature after a certain period of time has elapsed, and furthermore, a computer readable program that records a program for realizing these features. Recording media. The present invention also relates to a learning model generation device and a learning model generation method for generating a learning model used for these.

In recent years, it has been performed to determine a change in a feature after a certain period has elapsed. The result of such determination is used, for example, for investigating land use conditions, investigating changes in topography, and the like. Examples of the feature include all things on the ground, such as buildings, grounds, forests, roads, infrastructure structures, and vehicles.

特許 Furthermore,

Patent Documents

1 and 2 disclose devices for determining such a change in a feature. Specifically, the device disclosed in Patent Literature 1 first obtains two ortho images obtained by photographing a specific area from the sky. Further, one of the ortho images is photographed at the first time point, and the other is photographed at the second time point. Further, each ortho image holds three-dimensional data for each pixel.

Subsequently, the device disclosed in Patent Literature 1 compares two ortho images and, for each corresponding pixel, calculates a difference in color or tone between the two images and a difference in height included in the three-dimensional data. Ask for. Then, the device disclosed in Patent Literature 1 determines a change of a house, which is a feature, from the obtained difference. This determination is performed by applying the difference in color or color tone and the difference in height to a rule created in advance.For example, when the difference in height is large, it is determined that the house has been lost. When the color difference increases, it is determined that the house has been renovated.

Further, the apparatus disclosed in Patent Document 2 specifies an area other than a house (a building-independent area) using an infrared photograph and map data when a change determination target is a house, and identifies the specified building. The determination of the change of the feature can be executed by excluding the object-independent area. According to the device disclosed in Patent Literature 2, it is possible to avoid a situation in which an object other than a building is erroneously detected, and to improve determination accuracy.

Japanese Patent No. 4339289 Japanese Patent No. 5366190

According to the apparatus disclosed in Patent Document 1 described above, it is possible to determine a change in a feature, but this apparatus has the following problems. The first is that the image at the first time and the image at the second time, which are the basis of the two ortho images, were photographed under different conditions such as the photographing date and time and the weather. There is a problem that a difference occurs in the color tone in a portion where there is no change, and as a result, it is determined that there is a change even though there is no change. Second, due to the above-described difference in the photographing conditions, a difference occurs between the two ortho images in terms of noise generation, an area of a feature, and the like. The problem is that it is determined that there is a change. The third problem is that a change in height and color outside the house is extracted as a change in the house, and an erroneous determination is made.

Note that the first and second problems described above may be solved to some extent by setting a stricter threshold for comparison and determination, but in this case, it is not possible to determine a portion that actually changes as a change. there is a possibility.

On the other hand, according to the device disclosed in Patent Document 2, it is considered that the third problem described above can be solved. For this purpose, in addition to an ortho image having three-dimensional data for each pixel, a photographing area , It is necessary to input information such as near-infrared photographs and map data. For this reason, there is a problem that operation of the device disclosed in Patent Document 2 is costly. Further, even with the device disclosed in Patent Literature 2, the above-described first problem and second problem are not necessarily solved. Further, in the apparatus disclosed in Patent Document 2, when information such as infrared photographs and map data is not the latest information, the possibility of erroneous detection increases rather.

An example of an object of the present invention is to solve the above-described problem, and to determine the change of a feature over time, suppress the influence of external factors, and improve the determination accuracy. An object change determination device, a learning model generation method, a feature change determination method, and a computer-readable recording medium are provided.

In order to achieve the above object, a learning model generation device according to one aspect of the present invention includes:
An image acquisition unit that acquires a pair of images obtained by shooting a specific area from the sky at different times,
A three-dimensional data acquisition unit that acquires, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature that is a subject;
A difference data creation unit for finding a difference between pixel values and a difference in height held in the three-dimensional data for each pair of corresponding pixels of the paired image, and creating difference data including the obtained difference;
For each corresponding one or two or more pairs of pixels of the pair image, a label acquisition unit configured to acquire a label indicating a change in the feature set for the pair,
A learning model generation unit that generates a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label.
It is characterized by having.

In order to achieve the above object, a feature change determination device according to one aspect of the present invention is a device for determining a change in a feature in a specific area,
Acquiring a pair image obtained by shooting the specific area at a different time from the sky, an image acquisition unit,
A three-dimensional data acquisition unit for acquiring three-dimensional data for each pixel, which holds information on the height of a feature as a subject for each of the paired images;
For each pair of corresponding pixels of the paired image, a difference data creation unit that determines a difference between pixel values and a difference between heights held in the three-dimensional data, and creates difference data including the obtained difference.
A determining unit that applies the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature to determine the change of the feature in the specific area. When,
It is characterized by having.

In order to achieve the above object, a learning model generation method according to one aspect of the present invention includes:
(A) acquiring a pair image obtained by shooting a specific area from the sky at different times,
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) obtaining, for each corresponding one or more pairs of pixels of the paired image, a label indicating a change of the feature set for the pair;
(E) generating a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label;
Characterized by having

In order to achieve the above object, a feature change determination method according to one aspect of the present invention is a method for determining a feature change in a specific area,
(A) acquiring a pair image obtained by photographing the specific area from the sky at different times, and
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) determining the change of the feature in the specific area by applying the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature; , Steps,
Characterized by having

Further, to achieve the above object, the first computer reading according to one aspect of the present invention comprises:
On the computer,
(A) acquiring a pair image obtained by shooting a specific area from the sky at different times,
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) obtaining, for each corresponding one or more pairs of pixels of the paired image, a label indicating a change of the feature set for the pair;
(E) generating a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label;
Characterized by recording a program including an instruction to execute the program.

Further, in order to achieve the above object, a second computer-readable recording medium according to one aspect of the present invention is a computer-readable recording medium storing a program for determining a change in a feature in a specific area by a computer. A medium,
On the computer,
(A) acquiring a pair image obtained by photographing the specific area from the sky at different times, and
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) determining the change of the feature in the specific area by applying the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature; , Steps,
Characterized by recording a program including an instruction to execute the program.

As described above, according to the present invention, when determining a change in a feature over time, it is possible to suppress the influence of external factors and improve the determination accuracy.

FIG. 1 is a block diagram illustrating a configuration of a learning model generation device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a pair image and three-dimensional data used in the present embodiment. FIG. 3 is a diagram showing an example of difference data created in the present embodiment. FIG. 4 is an explanatory diagram illustrating the contents of a label obtained in the embodiment of the present invention. FIG. 5 is a diagram showing Variation 1 of label setting used in the embodiment of the present invention. FIG. 6 is a diagram showing Variation 2 of setting a label used in the embodiment of the present invention. FIG. 7 is a diagram showing Variation 3 of setting a label used in the embodiment of the present invention. FIG. 8 is a flowchart showing the operation of the learning model generation device according to the embodiment of the present invention. FIG. 9 is a block diagram illustrating a configuration of the feature change determination device according to the embodiment of the present invention. FIG. 10 is a block diagram specifically illustrating a configuration of the feature change determination device according to the embodiment of the present invention. FIG. 11 is a diagram illustrating an output example 1 of the feature changing device according to the embodiment of the present invention. FIG. 12 is a diagram illustrating an output example 2 of the feature changing device according to the embodiment of the present invention. FIG. 13 is a diagram illustrating an output example 3 of the feature changing device according to the embodiment of the present invention. FIG. 14 is a flowchart showing the operation of the learning model generation device according to the embodiment of the present invention. FIG. 15 is a block diagram illustrating a configuration of a modification of the feature change determination device according to the embodiment of the present invention. FIG. 16 is a block diagram illustrating an example of a computer that implements the learning model generation device and the feature change determination device according to the embodiment of the present invention.

(Embodiment: Generation of learning model)
First, a learning model generation device, a learning model generation method, and a program according to an embodiment of the present invention will be described.

[Device configuration]
First, the configuration of the learning model generation device according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of a learning model generation device according to an embodiment of the present invention.

As shown in FIG. 1, the learning model generation device 10 according to the present embodiment is a device that generates a learning model used for determining a change in a feature. As shown in FIG. 1, the learning model generation device 10 includes an image acquisition unit 11, a three-dimensional data acquisition unit 12, a difference data creation unit 13, a label acquisition unit 14, and a learning model generation unit 15. I have.

The image acquisition unit 11 acquires a pair image obtained by photographing a specific area from the sky at different times. The three-dimensional data acquisition unit 12 acquires three-dimensional data for each pixel in the pair image, which holds information on the height of a feature as a subject for each pixel. The difference data creation unit 13 finds a difference in pixel value and a difference in height held as three-dimensional data for each pair of corresponding pixels of the paired image, and creates difference data including the found difference.

The label acquiring unit 14 acquires, for each pair of one or more pixels corresponding to the paired image, a label indicating a change of a feature set for each pair. The learning model generation unit 15 generates a learning model that specifies the relationship between the difference data and the change of the feature by machine learning the difference data and the label.

As described above, in the present embodiment, the relationship between the difference data obtained from the pair image and the change of the feature is machine-learned, and a learning model is generated. Therefore, if this learning model is used, as described later, when determining a change in a feature over time, the influence of an external factor can be suppressed to improve the determination accuracy.

Next, the functions of the learning model generation device 10 according to the present embodiment will be described more specifically with reference to FIGS. Further, in the following description, a case where the feature is a building will be described, but the type of the feature is not particularly limited in the present embodiment. In addition to the above, the ground features, forests, roads, infrastructure structures, vehicles, and all other things on the ground can be cited as the features.

FIG. 2 is a diagram showing an example of a pair image and three-dimensional data used in the present embodiment. In the present embodiment, the image obtaining unit 11 obtains, for example, the paired image shown in FIG. 2, and the three-dimensional data obtaining unit 12 obtains, for example, the three-dimensional data shown in FIG.

画像 As shown in FIG. 2, in the present embodiment, images A and B are used as a pair image. Each of the images A and B is an ortho image obtained by performing digital processing on an image obtained by shooting from the sky. The photographing time of the image A is a first time, and the photographing time of the image B is a second time after the first time.

{Circle around (3)} As shown in FIG. 2, the three-dimensional data holds the height for each pixel of the images A and B. Specifically, DSM (Digital @ Surface @ Model) is used as three-dimensional data. In FIG. 2, the vertical and horizontal axes of the matrix represent the positions of pixels. The numerical value displayed in the matrix indicates the height [m]. Pixels located at the same coordinates correspond to each other.

FIG. 3 is a diagram showing an example of difference data created in the present embodiment. As shown in FIG. 3, in the present embodiment, the difference data creating unit 13 calculates, for each pair of corresponding pixels of the paired image A and the image B, the R value of the image, The difference between each of the G value and the B value is obtained. In the example of FIG. 3, the difference between the pixel values is the difference between each of the R value, the G value, and the B value. However, the present embodiment is not limited to this example. The difference between the pixel values may be set according to the color space.

(4) The difference data creating unit 13 subtracts the height data held for each pixel in the image B from the height data held for each pixel in the image A, and obtains a difference for each corresponding pixel. In this specification, the difference between the pixels is collectively referred to as “height difference”.

Ｒ The R value difference, the G value difference, the B value difference, and the height difference obtained as described above become difference data. In this specification, the difference data is also referred to as “four-channel data”.

FIG. 4 is an explanatory diagram for explaining the contents of the label obtained in the embodiment of the present invention. In the present embodiment, for example, assuming that the feature is a building, the label acquiring unit 14 newly constructs, loses, remodels, and changes each one or two or more corresponding pixels of the pair image as a label. Gets labels that include at least one of none.

In addition, in this embodiment, the label is manually set in advance for each pixel pair. At this time, as shown in FIG. 4, the label is set in advance in accordance with changes in height and pixel value (color) from the first time point to the second time point. For example, if the height increases and the color changes from soil color to another color, it is set to “new construction”. On the other hand, if the height becomes low and the color changes from another color to earth color, it is set to “lost”. If the color has changed and the height has not changed or the difference between the heights is small, “remodeling” is set.

(5) Further, in the present embodiment, there are the following three methods for setting a label as shown in FIGS. FIG. 5 is a diagram showing Variation 1 of label setting used in the embodiment of the present invention. FIG. 6 is a diagram showing Variation 2 of setting a label used in the embodiment of the present invention. FIG. 7 is a diagram showing Variation 3 of setting a label used in the embodiment of the present invention.

In the example of FIG. 5, labels are set by text data for each area, for example, for each building. In the example of FIG. 6, a label is set for each part in the area in units of an area larger than the area unit shown in FIG. In the example of FIG. 7, a label is set for each part in the image using the entire image as a unit.

6 and 7, in order to set a label, it is necessary to specify a portion corresponding to a building in the image. The portion corresponding to the building can be specified, for example, by using the data of the peripheral line of the building included in the basic item of the basic map information provided by the Geographical Survey Institute.

The learning model generation unit 15 executes machine learning using the four-channel data created by the difference data creation unit 13 and the labels acquired by the label acquisition unit 14 as learning data. As a method of machine learning in this case, for example, deep learning can be mentioned. The learning model generation unit 15 updates the value of the parameter in the function indicating the relationship between the difference data and the change of the building by this machine learning, and builds a model. The constructed model, specifically, a set of parameters is used in a feature change determination device described later.

[Device operation]
Next, the operation of the learning model generation device 10 according to the embodiment of the present invention will be described with reference to FIG. FIG. 8 is a flowchart showing the operation of the learning model generation device according to the embodiment of the present invention. In the following description, FIGS. 1 to 7 are appropriately referred to. In the present embodiment, the learning model generation method is performed by operating the learning model generation device 10. Therefore, the description of the learning model generation method in the present embodiment is replaced with the following description of the operation of the learning model generation device 10.

As illustrated in FIG. 8, first, the image acquisition unit 11 generates a pair of images obtained by shooting a specific area from the sky at different times in order to create learning data (difference data) used in machine learning to be described later. An image is obtained (step A1).

{Next, the three-dimensional data acquisition unit 12 acquires three-dimensional data for each pixel of the paired images acquired in step A1, which holds information on the height of a feature as a subject (step A2).

Next, the difference data creation unit 13 obtains a difference in pixel value and a difference in height held in the three-dimensional data for each pair of corresponding pixels of the paired image, and outputs the four-channel data including the obtained difference. Is created as learning data (step A3).

Next, the label acquiring unit 14 acquires, for each corresponding one or more pairs of pixels of the paired image, a label indicating a change of a feature set for each pair (step A4). As described above, the label is set in advance.

Next, the learning model generation unit 15 executes machine learning such as deep learning using the difference data created in step A3 and the label acquired in step A4 as learning data, and obtains the difference data A learning model for specifying a relationship with a change of a feature is generated (step A5).

[program]
The first program in the present embodiment may be any program that causes a computer to execute steps A1 to A5 shown in FIG. By installing and executing this program on a computer, the learning model generation device 10 and the learning model generation method according to the present embodiment can be realized. In this case, the processor of the computer functions as the image acquisition unit 11, the three-dimensional data acquisition unit 12, the difference data creation unit 13, the label acquisition unit 14, and the learning model generation unit 15 and performs processing.

The first program according to the present embodiment may be executed by a computer system configured by a plurality of computers. In this case, for example, each computer may function as one of the image acquisition unit 11, the three-dimensional data acquisition unit 12, the difference data creation unit 13, the label acquisition unit 14, and the learning model generation unit 15, respectively. .

(Embodiment: Judgment of feature change)
Subsequently, a feature change determination device, a feature change determination method, and a program according to an embodiment of the present invention will be described.

[Device configuration]
First, the configuration of the feature change determination device according to the present embodiment will be described with reference to FIG. FIG. 9 is a block diagram illustrating a configuration of the feature change determination device according to the embodiment of the present invention.

地 The feature change determining device 20 according to the present embodiment shown in FIG. 9 is a device for determining a feature change in a specific area. As shown in FIG. 9, the feature change determination device 20 includes an image acquisition unit 21, a three-dimensional data acquisition unit 22, a difference data creation unit 23, and a determination unit 24.

The image acquisition unit 21 acquires a pair image obtained by photographing a specific area from the sky at different times. The three-dimensional data acquisition unit 22 acquires three-dimensional data for each pair of images, which holds information on the height of a feature as a subject for each pixel.

The difference data creation unit 23 finds a difference in pixel value and a difference in height held in three-dimensional data for each pair of corresponding pixels in the paired image, and creates difference data including the obtained difference. The determination unit 24 applies the difference data to a learning model obtained by machine learning the relationship between the difference data of the paired images and the change of the feature to determine the change of the feature in the specific area.

With this configuration, in the present embodiment, when judging a change in a feature over time, the influence of external factors such as shooting conditions and noise can be suppressed, so that the judgment accuracy can be improved. .

Next, the configuration and function of the feature change determination apparatus according to the present embodiment will be described more specifically with reference to FIG. FIG. 10 is a block diagram specifically illustrating a configuration of the feature change determination device according to the embodiment of the present invention.

As shown in FIG. 10, the feature change determination device 20 according to the present embodiment includes a model storage unit in addition to the above-described image acquisition unit 21, three-dimensional data acquisition unit 22, difference data creation unit 23, and determination unit 24. A section 25 is provided. The model storage unit 25 stores a learning model used by the determination unit 24. Further, the learning model is generated by the learning model generation device 10 shown in FIG.

The image acquisition unit 21 functions in the same manner as the image acquisition unit 21 illustrated in FIG. Each of the pair images acquired by the image acquiring unit 21 is also an ortho image obtained by performing digital processing on an image obtained by photographing from the sky.

The three-dimensional data acquisition unit 22 also functions in the same manner as the three-dimensional data acquisition unit 12 shown in FIG. The three-dimensional data acquired by the three-dimensional data acquisition unit 22 also holds a height for each pixel of each image (see FIG. 2).

(4) The difference data creation unit 23 also functions in the same manner as the difference data creation unit 13 shown in FIG. The difference data creation unit 23 obtains, for example, a difference between the R value, the G value, and the B value of the image as a difference between the pixel values for each corresponding pixel pair of each image of the paired images. Further, the difference data creating unit 13 subtracts the height data held for each pixel of the other image from the height data held for each pixel of one of the paired images, To find the difference. The R value difference, the G value difference, the B value difference, and the height difference obtained in this manner become difference data (four channel data).

Upon acquiring the four-channel data created by the difference data creating unit 23, the determination unit 24 applies the four-channel data to the learning model stored in the model storage unit 25. Then, the determination unit 24 outputs the output from the learning model as a determination result of a feature change in the specific area.

Here, an example of the determination result of the determination unit 24 will be described with reference to FIGS. FIG. 11 is a diagram illustrating an output example 1 of the feature changing device according to the embodiment of the present invention. FIG. 12 is a diagram illustrating an output example 2 of the feature changing device according to the embodiment of the present invention. FIG. 13 is a diagram illustrating an output example 3 of the feature changing device according to the embodiment of the present invention.

例 The example of FIG. 11 shows a case where a pair image of one building is input as an input image. In the example of FIG. 11, a determination result of the same content as the label is output as a text for the paired image. Further, the example of FIG. 12 shows a case where a pair image of an area larger than one building is input. In the example of FIG. 12, an image to which a determination result is added for each portion in the region is output. The example of FIG. 13 illustrates a case where a pair image of a specific area is input. In the example of FIG. 13, an image to which a determination result is added is output for each portion in the specific area.

[Device operation]
Next, the operation of the feature change determination device 20 according to the embodiment of the present invention will be described with reference to FIG. FIG. 14 is a flowchart showing the operation of the learning model generation device according to the embodiment of the present invention. In the following description, FIGS. 1 to 7 are appropriately referred to. In the present embodiment, the learning model generation method is performed by operating the learning model generation device 10. Therefore, the description of the learning model generation method in the present embodiment is replaced with the following description of the operation of the learning model generation device 10.

As shown in FIG. 14, first, the image acquisition unit 21 acquires a pair image of an area targeted for a feature change (step B1).

Next, the three-dimensional data acquisition unit 12 acquires, for each pair of images acquired in step B1, three-dimensional data that holds, for each pixel, information on the height of a feature in an area where a feature change is to be performed ( Step B2).

Next, the difference data creation unit 13 obtains a difference in pixel value and a difference in height held in the three-dimensional data for each pair of corresponding pixels of the paired image, and outputs the four-channel data including the obtained difference. It is created (step B3).

Next, when acquiring the four-channel data created in step B3, the determination unit 24 applies the four-channel data to the learning model stored in the model storage unit 25 to determine a change in the feature (step B4). ). Thereafter, the determination unit 24 outputs the determination result to an external terminal device, display device, or the like (step B5). Thereafter, the determination result is displayed on the screen of the terminal device and the screen of the display device (see FIGS. 11 to 13).

As described above, in the present embodiment, the learning model obtained by machine learning the relationship between the difference data obtained from the paired image and the change of the feature is used, and the location in the area to be determined is used. An object change is determined. For this reason, in the present embodiment, the influence of external factors such as shooting conditions and noise can be suppressed when determining a change in a feature over time, and thus the determination accuracy can be improved.

[program]
The second program in the present embodiment may be any program that causes a computer to execute steps B1 to B5 shown in FIG. By installing and executing this program on a computer, the feature change determination device 20 and the feature determination method according to the present embodiment can be realized. In this case, the processor of the computer functions as the image acquisition unit 21, the three-dimensional data acquisition unit 22, the difference data creation unit 23, and the determination unit 24, and performs processing.

The first program according to the present embodiment may be executed by a computer system configured by a plurality of computers. In this case, for example, each computer may function as any one of the image acquisition unit 21, the three-dimensional data acquisition unit 22, the difference data creation unit 23, and the determination unit 24.

[Modification]
Next, a modification of the feature change determination device 20 according to the present embodiment will be described with reference to FIG. FIG. 15 is a block diagram illustrating a configuration of a modification of the feature change determination device according to the embodiment of the present invention.

As shown in FIG. 15, in this modification, the feature change determination device 20 includes an image acquisition unit 21, a three-dimensional data acquisition unit 22, a difference data creation unit 23, a determination unit 24, and a model storage unit 25. , The label acquisition unit 14 and the learning model generation unit 15 shown in FIG. Therefore, in the present modification, the feature change determination device 20 also has a function as the learning model generation device 10 illustrated in FIG. In this modification, the feature change determination device 20 itself can generate a learning model.

(Physical configuration)
Here, a computer that realizes the learning model generation device 10 by executing the first program according to the present embodiment, and a feature change determination device 20 that executes the second program according to the present embodiment A computer that realizes the above will be described with reference to FIG. FIG. 16 is a block diagram illustrating an example of a computer that implements the learning model generation device and the feature change determination device according to the embodiment of the present invention.

As shown in FIG. 16, the computer 110 includes a CPU 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These units are connected via a bus 121 so as to be able to perform data communication with each other. Note that the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111.

The CPU 111 performs various operations by expanding the program (code) according to the present embodiment stored in the storage device 113 into the main memory 112 and executing them in a predetermined order. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Further, the program according to the present embodiment is provided in a state stored in computer-readable recording medium 120. Note that the program in the present embodiment may be distributed on the Internet connected via the communication interface 117.

具体 Specific examples of the storage device 113 include a semiconductor storage device such as a flash memory in addition to a hard disk drive. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls display on the display device 119.

The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads out a program from the recording medium 120, and writes a processing result of the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.

Specific examples of the recording medium 120 include a general-purpose semiconductor storage device such as CF (Compact @ Flash (registered trademark)) and SD (Secure Digital), a magnetic recording medium such as a flexible disk (Flexible @ Disk), or a CD-ROM. An optical recording medium such as a ROM (Compact Disk Read Only Memory) may be used.

The learning model generation device 10 and the feature change determination device 20 according to the present embodiment can also be realized by using hardware corresponding to each unit instead of a computer in which a program is installed. Further, each of the learning model generation device 10 and the feature change determination device 20 may be partially implemented by a program, and the remaining portion may be implemented by hardware.

一部 Some or all of the above-described embodiments can be expressed by the following (Appendix 1) to (Appendix 21), but are not limited to the following description.

(Appendix 1)
An image acquisition unit that acquires a pair image obtained by shooting a specific area from the sky at different times,
A three-dimensional data acquisition unit for acquiring three-dimensional data for each pixel, which holds information on the height of a feature as a subject for each of the paired images;
For each pair of corresponding pixels of the paired image, a difference data creation unit that determines a difference between pixel values and a difference between heights held in the three-dimensional data, and creates difference data including the obtained difference.
For each corresponding one or two or more pairs of pixels of the pair image, a label acquisition unit configured to acquire a label indicating a change in the feature set for the pair,
A learning model generation unit that generates a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label.
Has,
A learning model generation device, characterized in that:

(Appendix 2)
The learning model generation device according to Supplementary Note 1, wherein
The difference data creation unit obtains, for each pair of corresponding pixels of the paired image, a difference between an R value, a G value, and a B value as a difference between the pixel values.
A learning model generation device, characterized in that:

(Appendix 3)
The learning model generation device according to

claim

1 or 2, wherein
When the feature is a building,
The label acquisition unit acquires, for each pair of one or more pixels corresponding to the paired image, a label including at least one of new construction, loss, remodeling, and no change as the label,
A learning model generation device, characterized in that:

(Appendix 4)
An apparatus for determining a change in a feature in a specific area,
Acquiring a pair image obtained by shooting the specific area at a different time from the sky, an image acquisition unit,
A three-dimensional data acquisition unit for acquiring three-dimensional data for each pixel, which holds information on the height of a feature as a subject for each of the paired images;
For each pair of corresponding pixels of the paired image, a difference data creation unit that determines a difference between pixel values and a difference between heights held in the three-dimensional data, and creates difference data including the obtained difference.
A determining unit that applies the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature to determine the change of the feature in the specific area. When,
Has,
A feature change judging device characterized in that:

(Appendix 5)
The feature change determination device according to claim 4, wherein
The difference data creation unit obtains, for each pair of corresponding pixels of the paired image, a difference between an R value, a G value, and a B value as a difference between the pixel values.
A feature change judging device characterized in that:

(Appendix 6)
A feature change determination device according to claim 4 or 5, wherein
When the feature is a building,
The learning model is obtained by machine learning a relationship between the difference data of the pair image and a change in the feature including at least one of new construction, loss, renovation, and no change,
A feature change judging device characterized in that:

(Appendix 7)
7. The feature change determination device according to any one of supplementary notes 4 to 6, wherein
For each corresponding one or two or more pairs of pixels of the pair image, a label acquisition unit configured to acquire a label indicating a change in the feature set for the pair,
A learning model generation unit that generates a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label.
Further comprising
A feature change judging device characterized in that:

(Appendix 8)
(A) acquiring a pair image obtained by shooting a specific area from the sky at different times,
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) obtaining, for each corresponding one or more pairs of pixels of the paired image, a label indicating a change of the feature set for the pair;
(E) generating a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label;
Having,
A learning model generation method characterized by the following.

(Appendix 9)
A learning model generation method according to attachment 8, wherein
In the step (c), a difference between an R value, a G value, and a B value is obtained as a difference between the pixel values for each pair of corresponding pixels of the pair image.
A learning model generation method characterized by the following.

(Appendix 10)
A learning model generation method according to attachment 8 or 9, wherein
When the feature is a building,
In the step (d), the label acquiring unit includes, for each pair of one or more pixels corresponding to the paired image, a label including at least one of new construction, loss, remodeling, and no change as the label. get,
A learning model generation method characterized by the following.

(Appendix 11)
A method for determining a change in a feature in a specific area,
(A) acquiring a pair image obtained by photographing the specific area from the sky at different times, and
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) determining the change of the feature in the specific area by applying the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature; , Steps,
Having,
A feature change judging method characterized in that:

(Appendix 12)
A feature change determination method according to attachment 11, wherein
In the step (c), a difference between an R value, a G value, and a B value is obtained as a difference between the pixel values for each pair of corresponding pixels of the pair image.
A feature change judging method characterized in that:

(Appendix 13)
A feature change determination method according to claim 11 or 12, wherein
When the feature is a building,
The learning model is obtained by machine learning a relationship between the difference data of the pair image and a change in the feature including at least one of new construction, loss, renovation, and no change,
A feature change judging method characterized in that:

(Appendix 14)
The feature change determination method according to any one of supplementary notes 11 to 13, wherein
(E) obtaining, for each pair of one or more pixels corresponding to the paired image, a label indicating a change of the feature set for the pair;
(F) generating a learning model that specifies the relationship between the difference data and the change of the feature by machine learning the difference data and the label;
Further having
A feature change judging method characterized in that:

(Appendix 15)
On the computer,
(A) acquiring a pair image obtained by shooting a specific area from the sky at different times,
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) obtaining, for each corresponding one or more pairs of pixels of the paired image, a label indicating a change of the feature set for the pair;
(E) generating a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label;
And a computer-readable recording medium storing a program including instructions for executing the program.

(Appendix 16)
A computer-readable recording medium according to supplementary note 15, wherein:
In the step (c), a difference between an R value, a G value, and a B value is obtained as a difference between the pixel values for each pair of corresponding pixels of the pair image.
A computer-readable recording medium characterized by the above-mentioned.

(Appendix 17)
A computer-readable recording medium according to Supplementary Note 15 or 16, wherein
When the feature is a building,
In the step (d), the label acquiring unit includes, for each pair of one or more pixels corresponding to the paired image, a label including at least one of new construction, loss, remodeling, and no change as the label. get,
A computer-readable recording medium characterized by the above-mentioned.

(Appendix 18)
A computer-readable recording medium recording a program for determining a change of a feature in a specific area by a computer,
On the computer,
(A) acquiring a pair image obtained by photographing the specific area from the sky at different times, and
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) determining the change of the feature in the specific area by applying the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature; , Steps,
And a computer-readable recording medium storing a program including instructions for executing the program.

(Appendix 19)
A computer-readable recording medium according to supplementary note 18, wherein:
In the step (c), a difference between an R value, a G value, and a B value is obtained as a difference between the pixel values for each pair of corresponding pixels of the pair image.
A computer-readable recording medium characterized by the above-mentioned.

(Appendix 20)
A computer-readable recording medium according to claim 18 or 19,
When the feature is a building,
The learning model is obtained by machine learning a relationship between the difference data of the pair image and a change in the feature including at least one of new construction, loss, renovation, and no change,
A computer-readable recording medium characterized by the above-mentioned.

(Appendix 21)
21. The computer-readable recording medium according to any one of supplementary notes 18 to 20, wherein
The program, the computer,
(E) obtaining, for each pair of one or more pixels corresponding to the paired image, a label indicating a change of the feature set for the pair;
(F) generating a learning model that specifies the relationship between the difference data and the change of the feature by machine learning the difference data and the label;
Further comprising instructions for executing
A computer-readable recording medium characterized by the above-mentioned.

Although the present invention has been described with reference to the exemplary embodiments, the present invention is not limited to the above exemplary embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2018-185480 filed on Sep. 28, 2018, the entire disclosure of which is incorporated herein.

As described above, according to the present invention, when determining a change in a feature over time, it is possible to suppress the influence of external factors and improve the determination accuracy. INDUSTRIAL APPLICABILITY The present invention is useful in a field in which a temporal change of a feature needs to be determined, for example, an investigation of a land use situation, an investigation of a topographical change, a deterioration determination of an infrastructure structure, a management of a forest change, and the like.

Reference Signs List 10 learning model generation device 11 image acquisition unit 12 three-dimensional data acquisition unit 13 difference data creation unit 14 label acquisition unit 15 learning model generation unit 20 feature change determination device 21 image acquisition unit 22 three-dimensional data acquisition unit 23 difference data creation unit 24 determination unit 25 model storage unit 110 computer 111 CPU
112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

Image acquisition means for acquiring a pair image obtained by shooting a specific area from the sky at different times,
Three-dimensional data acquisition means for acquiring, for each of the paired images, three-dimensional data that holds information on the height of a feature as a subject for each pixel;
Difference data creating means for finding a difference in pixel value and a difference in height held in the three-dimensional data for each pair of corresponding pixels of the paired image, and creating difference data including the obtained difference;
A label acquisition unit configured to acquire, for each pair of one or more pixels corresponding to the pair image, a label set for the pair and indicating a change in the feature,
By learning the difference data and the label machine learning, to generate a learning model that specifies the relationship between the difference data and the change of the feature, learning model generating means,
Has,
A learning model generation device, characterized in that:
The learning model generation device according to claim 1,
The difference data creating unit obtains, for each pair of corresponding pixels of the paired image, a difference between an R value, a G value, and a B value as a difference between the pixel values.
A learning model generation device, characterized in that:
The learning model generation device according to claim 1 or 2,
When the feature is a building,
The label acquisition unit acquires, for each pair of one or more pixels corresponding to the paired image, a label including at least one of new construction, loss, remodeling, and no change as the label.
A learning model generation device, characterized in that:
An apparatus for determining a change in a feature in a specific area,
Acquiring a pair image obtained by shooting the specific area at a different time from the sky, image acquisition means,
Three-dimensional data acquisition means for acquiring, for each of the paired images, three-dimensional data that holds information on the height of a feature as a subject for each pixel;
Difference data creating means for finding a difference in pixel value and a difference in height held in the three-dimensional data for each pair of corresponding pixels of the paired image, and creating difference data including the obtained difference;
Determining means for applying the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature to determine the change of the feature in the specific area When,
Has,
A feature change judging device characterized in that:
The feature change determination device according to claim 4,
The difference data creating unit obtains, for each pair of corresponding pixels of the paired image, a difference between an R value, a G value, and a B value as a difference between the pixel values.
A feature change judging device characterized in that:
The feature change determination device according to claim 4 or 5,
When the feature is a building,
The learning model is obtained by machine learning a relationship between the difference data of the pair image and a change in the feature including at least one of new construction, loss, renovation, and no change,
A feature change judging device characterized in that:
The feature change determination device according to any one of claims 4 to 6,
A label acquisition unit configured to acquire, for each pair of one or more pixels corresponding to the pair image, a label set for the pair and indicating a change in the feature,
By learning the difference data and the label machine learning, to generate a learning model that specifies the relationship between the difference data and the change of the feature, learning model generating means,
Further comprising
A feature change judging device characterized in that:
(A) Obtain a pair image obtained by shooting a specific area from the sky at different times,
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) For each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data are obtained, and difference data including the obtained difference is created.
(D) acquiring, for each corresponding one or more pairs of pixels of the pair image, a label indicating the change of the feature set for the pair;
(E) generating a learning model that specifies the relationship between the difference data and the change of the feature by machine learning the difference data and the label;
A learning model generation method characterized by the following.
The learning model generation method according to claim 8, wherein
In (c), for each pair of corresponding pixels of the paired image, a difference between an R value, a G value, and a B value is determined as a difference between the pixel values.
A learning model generation method characterized by the following.
The learning model generation method according to claim 8 or 9, wherein
When the feature is a building,
In (d), for each pair of one or more pixels corresponding to the paired image, a label including at least one of new construction, loss, remodeling, and no change is obtained as the label.
A learning model generation method characterized by the following.
A method for determining a change in a feature in a specific area,
(A) obtaining a pair image obtained by shooting the specific area from the sky at different times,
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) For each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data are obtained, and difference data including the obtained difference is created.
(D) determining the change of the feature in the specific area by applying the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature; ,
A feature change judging method characterized in that:
It is a feature change determination method according to claim 11,
In (c), for each pair of corresponding pixels of the paired image, a difference between an R value, a G value, and a B value is determined as a difference between the pixel values.
A feature change judging method characterized in that:
The feature change determination method according to claim 11 or 12,
When the feature is a building,
The learning model is obtained by machine learning a relationship between the difference data of the pair image and a change in the feature including at least one of new construction, loss, renovation, and no change,
A feature change judging method characterized in that:
The feature change determination method according to any one of claims 11 to 13, further comprising:
(E) For each pair of one or more pixels corresponding to the paired image, obtain a label indicating the change of the feature set for the pair,
(F) generating a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label;
A feature change judging method characterized in that:
On the computer,
(A) acquiring a pair image obtained by shooting a specific area from the sky at different times,
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) obtaining, for each corresponding one or more pairs of pixels of the paired image, a label indicating a change of the feature set for the pair;
(E) generating a learning model that specifies a relationship between the difference data and the change of the feature by machine learning the difference data and the label;
And a computer-readable recording medium storing a program including instructions for executing the program.
A computer-readable recording medium according to claim 15, wherein
In the step (c), a difference between an R value, a G value, and a B value is obtained as a difference between the pixel values for each pair of corresponding pixels of the pair image.
A computer-readable recording medium characterized by the above-mentioned.
A computer-readable recording medium according to claim 15 or 16,
When the feature is a building,
In the step (d), for each pair of one or more corresponding pixels of the pair image, a label including at least one of a new building, a lost building, a remodeling, and no change is obtained as the label.
A computer-readable recording medium characterized by the above-mentioned.
A computer-readable recording medium recording a program for determining a change of a feature in a specific area by a computer,
On the computer,
(A) acquiring a pair image obtained by photographing the specific area from the sky at different times, and
(B) obtaining, for each of the paired images, three-dimensional data that holds, for each pixel, information on the height of a feature serving as a subject;
(C) obtaining, for each pair of corresponding pixels of the paired image, a difference in pixel value and a difference in height held in the three-dimensional data, and creating difference data including the obtained difference;
(D) determining the change of the feature in the specific area by applying the difference data to a learning model obtained by machine learning the relationship between the difference data of the pair image and the change of the feature; , Steps,
And a computer-readable recording medium storing a program including instructions for executing the program.
A computer-readable recording medium according to claim 18,
In the step (c), a difference between an R value, a G value, and a B value is obtained as a difference between the pixel values for each pair of corresponding pixels of the pair image.
A computer-readable recording medium characterized by the above-mentioned.
A computer-readable recording medium according to claim 18 or 19,
When the feature is a building,
The learning model is obtained by machine learning a relationship between the difference data of the pair image and a change in the feature including at least one of new construction, loss, renovation, and no change,
A computer-readable recording medium characterized by the above-mentioned.
A computer-readable recording medium according to any one of claims 18 to 20, wherein
The program, the computer,
(E) obtaining, for each pair of one or more pixels corresponding to the paired image, a label indicating a change of the feature set for the pair;
(F) generating a learning model that specifies the relationship between the difference data and the change of the feature by machine learning the difference data and the label;
Further comprising instructions for executing
A computer-readable recording medium characterized by the above-mentioned.