JP2019175139A - Architectural structure extraction system - Google Patents

Abstract

To prevent oversight during building extraction.SOLUTION: An architectural structure extraction system includes a first building detection device that has learned a plurality of buildings belonging to a first group using an input image for learning and teacher data of information indicating the shapes of the plurality of buildings included in the input image for learning, a second building detection device that has learned a plurality of buildings belonging to a second group different from the first group using an input image for learning and teacher data of information indicating the shapes of the plurality of buildings included in the input image for learning, an input part that inputs feature information of the input image where a region subject to learning on the ground surface is photographed from the sky to the first building detection device and the second building detection device, and an integration part that integrates the output of the first building detection device and the output of the second building detection device in response to the input feature information of the input image. The first building detection device and the second building detection device include a neural network of the type different from each other.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a building extraction system.

  A technique for extracting a building from data such as an aerial photograph or satellite image acquired from the sky has been studied. Patent Document 1 discloses a system for designating a work area including a building that an operator wants to extract on an image such as an aerial photograph, and automatically extracting the outline of the building in the work area. Patent Document 2 listed below discloses an apparatus that extracts the outline of a building using a DSM (Digital Surface Model) obtained from above using a laser scanner or the like.

  Patent Document 3 discloses an ensemble detector having three scales in an object detection device for recognizing a pedestrian, and discloses that the size of a pedestrian image to be detected differs depending on the scale. Yes.

JP 2011-76178 A JP 2013-101428 A Japanese Patent Application Laid-Open No. 2018-5520

For example, the inventors have developed a method of extracting a building using a convolutional neural network in order to reduce a work load for detecting a change (new construction or demolition) of the building.
The inventors conducted experiments using a model including a convolution layer and a pooling layer and a model using an extended convolution operation as a neural network used for building extraction. However, each model has strength and weakness in building extraction accuracy depending on attributes such as the size of the building, and it is difficult to suppress oversight in building extraction using either model.

  This invention is made | formed in view of the said subject, The objective is to provide the building extraction system which can suppress the oversight in extraction of a building.

(1) A first learning for a plurality of buildings belonging to the first group using a learning input image and teacher data of information indicating the shapes of the plurality of buildings included in the learning input image. A second that has been learned using a building detector and a plurality of buildings belonging to a second group different from the first group by using an input image for learning and teacher data of information indicating the shapes of the plurality of buildings. And the input unit for inputting the feature information of the input image in which the learning target area on the ground surface is photographed from the sky to the first building detector and the second building detector, and the input An integration unit that integrates the output of the first building detector and the output of the second building detector with respect to the feature information of the input image, the first building detector and the second building The detectors are of different types. Building extraction system, including, including the Lal network.

(2) In (1), the areas of the plurality of buildings belonging to the first group belong to the first range, and the areas of the plurality of buildings belonging to the second group are different from the first range. A building extraction system belonging to the second range.

(3) In (2), the building further includes a filter that removes the building included in the output of the first building detector and the building included in the output of the second building detector based on the area. Product extraction system.

(4) In any one of (1) to (3), the first building detector includes a convolution layer that performs an extended convolution operation, and the second building detector includes a pooling layer. Extraction system.

(5) In (2), the maximum value of the first range is smaller than the maximum value of the second range, and the first building detector includes a convolution layer that performs an extended convolution operation, The building detector of 2 includes a pooling layer.

(6) In any one of (1) to (3), a plurality of buildings belonging to any one of the first group and the second group includes a first type neural network, and the learning A first candidate detector trained using an input image and teacher data of information indicating the shape of the plurality of buildings included in the learning input image, and a second type neural network, Building shape detection accuracy of each of the second candidate detector learned using the learning input image and the teacher data of information indicating the shape of the plurality of buildings included in the learning input image One of the first candidate detector and the second candidate detector based on the evaluation accuracy evaluated by the evaluation unit and the first candidate detector and the second candidate detector. Second Further comprising a detector selection unit for selecting as the one of the building detectors, building extraction systems.

(7) In any one of (1) to (6), the integration unit outputs the output of the first building detector and the output of the second building detector with respect to the feature information of the inputted input image. The building extraction system which determines the area recognized as a building in any of the above as an area with a building.

It is a figure which shows an example of the hardware constitutions of the building extraction system concerning embodiment of this invention. It is a block diagram which shows the function structure of a building extraction system. It is a figure explaining the kind of learning detector. It is a figure explaining the difference in a scale. It is a figure which shows the outline | summary of a structure of a learning detector. It is a figure explaining the layer contained in the learning detector of a pooling model. It is a figure explaining the layer contained in the learning detector of a dilation model. It is a figure explaining an example of the layer structure in an extended convolution calculation. It is a flowchart which shows an example of the process which learns a learning detector. It is a flowchart which shows an example of the process of the learning execution part with respect to each of a window image. It is a figure which shows an example of teacher data. It is a flowchart which shows an example of the process which evaluates a learning detector. It is a figure which shows an evaluation result. It is a figure explaining the outline | summary of the process which determines the area | region of a building. It is a flowchart which shows the flow of the process which produces | generates a whole output image from a process target image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Of the constituent elements that appear, those having the same function are given the same reference numerals, and the description thereof is omitted.

In the building extraction system according to the present embodiment, an aerial photograph or a satellite image obtained by photographing a ground surface as a target region of a process for extracting a building on a building detector that is a learned model using a neural network (aerial photograph or The feature information may be an ortho image based on a satellite image (hereinafter referred to as “processing target image”), and a building region is determined and extracted based on the image output from the building detector. The building extraction system uses three building detectors when identifying a building from a processing target image. Each of the three building detectors is configured to detect a building belonging to a range of areas S, M, and L with higher accuracy. For example, the area range S is less than 45 m ² , the area range M is 45 m ² or more and less than 131 m ² , and the area range L is 131 m ² or more. Generally, buildings belonging to the area range L correspond to condominiums and large commercial facilities, the area range M corresponds to apartments and retail stores, and the area range S corresponds to ordinary houses.

  In the building extraction system according to the present embodiment, for each of the area ranges S, M, and L, a plurality of building detectors having different types of neural networks and different scales (scales) of input learning images are used. Learning is performed, and the best building detector is selected from a plurality of building detectors for each of the area ranges S, M, and L, and the selected building detector is the building region from the processing target data. Used for detection.

  FIG. 1 is a diagram illustrating a hardware configuration of a building extraction system according to an embodiment of the present invention. The building extraction system includes a learning server 1. The learning server 1 is a server computer and includes a processor 11, a storage unit 12, a communication unit 13, and an input / output unit 14.

  The processor 11 operates according to a program stored in the storage unit 12. Further, the processor 11 controls the communication unit 13 and controls a device connected to the input / output unit 14. Here, the processor 11 may include a so-called CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) used as a parallel computer. The program may be provided via the Internet or the like, or may be provided by being stored in a computer-readable storage medium such as a flash memory or a DVD-ROM. .

  The storage unit 12 includes a memory element such as a RAM and a flash memory, and a hard disk drive. The storage unit 12 stores the program. The storage unit 12 stores information input from each unit and calculation results.

  The communication unit 13 implements a function of communicating with other devices, and is configured by, for example, a wired LAN integrated circuit. The communication unit 13 transmits and receives information to and from other devices based on the control of the processor 11. The communication unit 13 inputs the received information to the processor 11 and the storage unit 12. The communication unit 13 is connected to other devices by, for example, a LAN.

  The input / output unit 14 includes a video controller that controls the display output device, a controller that acquires data from the input device, and the like. Examples of input devices include a keyboard, a mouse, and a touch panel. Based on the control of the processor 11, the input / output unit 14 outputs display data to the display output device, and acquires data input by the user operating the input device. The display output device is, for example, a display device connected to the outside.

  Next, the outline of the function of the building extraction system will be described. FIG. 2 is a block diagram showing a functional configuration of the building extraction system. The building extraction system functionally includes a learning data acquisition unit 51, a learning execution unit 52, a learning detector set 53, an evaluation data acquisition unit 56, an evaluation execution unit 57, a detector selection unit 58, An execution detector set 61, a target data input unit 65, an output acquisition unit 66, a filter unit 67, an integration unit 68, and an image output unit 69 are included. These functions are mainly realized by the processor 11 executing a program stored in the storage unit 12 and accessing data stored in the storage unit 12. All these functions may be executed by the learning server 1, or a part of the functions may be executed by another server. For example, the functions of the target data input unit 65, the execution detector set 61, the output acquisition unit 66, the filter unit 67, the integration unit 68, and the image output unit 69 are the processor 11, the storage unit 12, the communication unit 13, and the input / output unit 14. It may be realized by another server having

  The learning detector set 53 includes a plurality of learning detectors 54. In the present embodiment, the number of learning detectors 54 is 6, and each of the learning detectors 54 has a common unit 540 that performs common learning regardless of the area ranges S, M, and L, and each area range. And individual units 541, 542, and 543 that perform learning according to S, M, and L. Each of the learning detectors 54 learns about a combination of different neural network types and input learning image scales.

  The learning data acquisition unit 51 acquires a learning input image and teacher data indicating the shape of a building included in the learning input image. The learning execution unit 52 causes the learning detector 54 to learn using the learning input image and the teacher data.

  The evaluation data acquisition unit 56 acquires an evaluation input image and correct data indicating the shape of the building included in the evaluation input image. The evaluation input image and the correct answer data may be the same as the learning input image and the teacher data. The evaluation execution unit 57 evaluates the detection accuracy of the shape of the building for each of the individual detectors 541, 542, and 543 for each of the learning detectors 54 using the input image for evaluation and the correct answer data.

  Based on the detection accuracy evaluated by the evaluation execution unit 57, the detector selection unit 58 selects the learning detector 54 that detects the building for the input target data for each of the area ranges S, M, and L. At least a part of the selected learning detector 54 is used as the execution detectors 62, 63, 64 constituting the execution detector set 61. More specifically, the combination of the common unit 540 and the individual unit 541 included in the learning detector 54 selected for the area range S is included in the execution detector 62 corresponding to the area range S. And the individual unit 621. The combination of the common unit 540 and the individual unit 542 included in the learning detector 54 selected for the area range M becomes the common unit 630 and the individual unit 631 included in the execution detector 63 corresponding to the area range M. . The combination of the common unit 540 and the individual unit 543 included in the learning detector 54 selected for the area range L becomes the common unit 640 and the individual unit 641 included in the execution detector 64 corresponding to the area range L. .

  The target data input unit 65 acquires an input target image, processes the input target image as necessary, and inputs the input target image to the execution detectors 62, 63, 64. The output acquisition unit 66 acquires output images output from the execution detectors 62, 63, and 64.

  The filter unit 67 removes the buildings included in the output images of the execution detectors 62, 63, and 64 based on the area, and generates a filtered output image.

  The integration unit 68 integrates the filtered output images of the execution detectors 62, 63, and 64. The integration unit 68 generates an image in which an area recognized as a building in any of the output images of the execution detectors 62, 63, and 64 is determined as an area with a building.

  The image output unit 69 outputs the image integrated by the integration unit 68 to the storage unit 12 and the display output device.

  Next, details of the learning detector set 53 and the learning detector 54 included therein will be described. FIG. 3 is a diagram for explaining the types of the learning detector 54. “No” in the table shown in FIG. 3 indicates numbers assigned to the six learning detectors 54. “Scale” indicates the scale of the learning input image to be input to the learning detector 54 of that number, and the learning input image prepared first is adjusted (enlarged or reduced as necessary) at the magnification indicated on the scale. The learning input image cut out so as to have the same number of pixels regardless of the scale (hereinafter, the cut out learning input image is referred to as a “window image”) is input to the learning detector 54. “Model type” indicates the type of the neural network that constitutes the learning detector 54 of that number. “Pooling” indicates that the convolutional layer and the pooling layer are combined in the CNN (Convolutional Neural Network) (hereinafter referred to as “pooling model”), and “Dilation” indicates a convolutional layer that performs an extended convolution operation. This is a model that uses (hereinafter referred to as “dilation model”).

  FIG. 4 is a diagram for explaining a difference in scale. 4A is an example of a window image when the scale is 0.5 times, and FIGS. 4B and 4C are window images when the scale is 1 time and 2 times, respectively. It is an example. The window images shown in FIGS. 4A to 4C include the same region. The number of pixels in the window image is Px × Py. The values of Px and Py may be 32 or 64, for example. The learning input image when the scale is 0.5 times is obtained by reducing (thinning out) the learning input image when the scale is 1.0 so that the number of vertical and horizontal dots becomes 1/2 times. The learning input image when the scale is 2.0 times is enlarged so that the number of vertical and horizontal dots is doubled (such as linear interpolation between dots). It is obtained by placing a dot. The learning data acquisition unit 51 enlarges or reduces the learning input image.

  FIG. 5 is a diagram showing an outline of the configuration of the learning detector 54. As described above, the learning detector 54 includes the common unit 540 and the individual units 541, 542, and 543. The common unit 540 has a plurality of layers, and the individual units 541, 542, and 543 have the same number of layers. The adjusted learning input image is input to the first layer of the common unit 540, and the feature information that is the output of the last layer is input to the first layers of the individual units 541, 542, and 543, respectively. . The outputs of the individual units 541, 542, and 543 are, for example, 16 × 16 dot images, and each dot indicates the existence probability of the building at the position of the dot.

  FIG. 6 is a diagram for explaining the layers included in the learning detector 54 of the pooling model. In FIG. 6, the layers are described in the order of processing. In the column of affiliation, a layer described as “common” exists in the common part 540, and a layer described as “individual” exists in the individual parts 541, 542, and 543. Here, the layers described as “individual” exist in each of the individual units 541, 542, and 543. The processing type indicates the type of each layer, “input” indicates the input layer, “convolution” indicates the convolution layer, and “pooling (s2)” indicates the pooling layer whose stride (kernel application interval) is 2. Yes. The kernel size is a parameter representing the size of the convolution filter. Here, corresponding to the processing target being an image, the kernel is two-dimensional, and the kernel size value “k” means a “k × k” filter. The “number of feature maps” of each layer is the number of feature maps extracted in the layer, and is also called a channel. The stride is 1 unless otherwise specified, and the description for each layer is omitted.

  FIG. 7 is a diagram illustrating layers included in the learning detector of the dilation model. Although the description of FIG. 7 is similar to the description of FIG. 6, the “convolution” layer in the dilation model indicates an extension convolution layer, and the setting of the extension convolution layer is shown in the column of the expansion coefficient.

  The extended convolution operation will be further described. FIG. 8 is a diagram for explaining an example of a layer structure in the extended convolution operation. An input image such as a learning input image is spatially two-dimensional data. Here, for simplification of illustration and description, the input data to the learning detector 54 is simplified to one-dimensional data. Specifically, a plurality of “◯” marks arranged in the horizontal direction in the input layer located at the bottom in FIG. 8 constitute input data. The input data element 30 indicated by “◯” corresponds to a pixel (or pixel value) in the input image. The convolution layer shown in FIG. 8 is a so-called feature extraction layer, and the description of the layer following the feature extraction layer is omitted.

  The neural network shown in FIG. 8 has seven convolution layers as feature extraction layers, and each convolution layer performs an extended convolution operation. The first convolution layer located above the input layer performs an expansion convolution operation with an expansion coefficient d = 1. Specifically, a convolution operation is performed in each of a plurality of units 31 indicated by “◯” in the first layer, and each unit 31 multiplies the values of two adjacent elements 30 in the input layer by adding a weight. Output the value.

  The second convolution layer performs an expansion convolution operation with an expansion coefficient d = 2. Specifically, a convolution operation is performed in each of the plurality of units 32 indicated by “◯” in the second layer, and each unit 32 multiplies the output value of every other unit 31 in the first layer by a weight. Output the sum.

  Further, the convolution layer of the third layer performs an expansion convolution operation with the expansion coefficient d = 3, and each unit 33 represented by “◯” mark of the third layer becomes the output value of every third unit 32 in the second layer. The value obtained by multiplying by the weight is output, the convolution layer of the fourth layer performs the expansion convolution operation with the expansion coefficient d = 4, and each unit 34 represented by “◯” mark of the fourth layer is in the third layer. A value obtained by multiplying the output value of every seventh unit 33 by the weight is added. Each unit 35 in the fifth layer performs an extension convolution operation with d = 3, and each unit 36 in the sixth layer and each unit 37 in the seventh layer respectively perform an extension convolution operation with d = 2 and d = 1. I do.

  Here, in the structure of the feature extraction layer shown in FIG. 8, the portion consisting of the first layer to the fourth layer is referred to as a front end portion, and the subsequent portion consisting of the fifth layer to the seventh layer is referred to as a local feature extraction portion. I will call it. The front end part is a plurality of convolution layers following the input layer. In the front end part, the expansion coefficient d increases to the maximum value in the feature extraction layer according to the arrangement order of the convolution layers. On the other hand, the local feature extraction unit is a plurality of convolution layers following the front end unit. In the local feature extraction unit, the expansion coefficient decreases according to the arrangement order of the convolution layers.

FIG. 8 illustrates, as a line, the connection relationship between the units of the first to sixth layers and the input layer that are folded into the output of one unit 37 of the seventh layer. In the extended convolution operation, the application range of the kernel is expanded exponentially according to the expansion coefficient d. For example, the kernel of the convolution operation of d = 1 to 4 in FIG. 8 is a filter that convolves two inputs, that is, a filter of size 2, but one of the two inputs convolved by the kernel of d = 1. The interval in the array of dimensional data is 1, whereas the interval between two inputs convolved by the kernel with d = 2 is 2, and when d = 3, the interval is 4, and when d = 4, the interval is The interval is 8. That is, the interval is set to 2 ^d−1 .

  As can be seen from the connection relationship between the unit and the input layer in the front end unit, in the extended convolution operation, the receptive field can be expanded with a small number of layers by extending the application range of the kernel. And since a receptive field is expanded only by convolution, the pooling layer used by general CNN becomes unnecessary, and the resolution fall by a pooling layer can be avoided. Further, while expanding the application range, only a part of the remaining elements after thinning out the elements in the range is convoluted, thereby suppressing an increase in the weight parameter.

  On the other hand, the structure in which layers are stacked so that the expansion coefficient d increases in order as in the front-end portion has a problem that the correlation between neighboring units in the uppermost layer is weakened and a problem that it is difficult to pick up local features of input data Have The local feature extraction unit is provided in order to solve this problem. By combining the front end unit and the local feature extraction unit, there is a problem that the correlation between neighboring units is weakened in a unit of the seventh layer, The problem that local information that the units 31a and 31b of the first layer are adjacent to each other cannot be grasped is solved.

  In other words, the local feature extraction unit is provided after the front end unit, so that the front end unit actively uses the extended convolution operation to obtain the context without reducing any resolution, and the local feature extraction unit. Then, local features distributed by the front-end part are aggregated. As a result, context information and local feature information can be used effectively, and small and dense objects can be recognized.

  Next, the details of the process of causing the learning detector 54 described so far to learn using the learning input image corresponding to the scale and the teacher data indicating the shape of the building included in the learning image will be described. .

  FIG. 9 is a flowchart showing an example of a process for causing the learning detector 54 to learn. FIG. 9 shows the processing of the learning data acquisition unit 51 and the learning execution unit 52, and the learning detector 54 is learned by this processing. Further, the process shown in FIG. 9 is performed for each learning detector 54 by the number of repetitions.

  The learning data acquisition unit 51 acquires the learning image stored in the storage unit 12 (step S101). The learning image is an aerial photograph, a satellite image, or the like (which may be an ortho image based on the aerial photograph or the satellite image) obtained by photographing the ground surface as a processing target region for extracting a building. Next, the learning data acquisition unit 51 sets the size of the learning image to match the scale of the learning detector 54 (step S102). For example, if the scale of the learning detector 54 is 0.5 times, the learning image is reduced to 0.5 times, and if the scale is 2 times, the learning image is enlarged twice. Instead of performing the process of step S102, a plurality of learning images corresponding to each type of scale are prepared in advance, and the learning data acquisition unit 51 reads an image corresponding to the scale of the learning detector 54. Good.

  And the learning execution part 52 cuts out the window image input into the learning detector 54 from the image for learning set so that it might match | combine with a scale (step S103). The window image has a size of Px × Py, a position is randomly selected from one learning image, and the window image is cut out from the learning image based on the selected position.

  The learning execution unit 52 inputs the window image cut out from the learning image, and causes the learning detector 54 to learn by comparing the output with the teacher data (step S104).

  FIG. 10 is a flowchart showing an example of processing of the learning execution unit 52 for each of the window images, and is a diagram for explaining the processing of step S104 in more detail. In step S104, first, the learning execution unit 52 inputs a window image cut out from the learning image to the common unit 540 of the learning detector 54 (step S121). Accordingly, the common unit 540 of the learning detector 54 processes the window image, and the individual units 541, 542, and 543 process the output of the common unit 540. And the learning execution part 52 acquires each output image of the separate parts 541,542,543 of the learning detector 54 (step S122). Here, in the following, the output image of the individual unit 541 corresponding to the area range S is the output image (S), the output image of the individual unit 542 corresponding to the area range M is the output image (M), and the area range L An output image of the individual unit 543 corresponding to is described as an output image (L). The output images of the individual units 541, 542, and 543 are collectively referred to as output images (S, M, L). Here, the value of each dot in the output image (S, M, L) indicates the existence probability of the building area.

  Next, the learning execution unit 52 calculates an error between the output image (S, M, L) of the learning detector 54 and the teacher data (step S123). Here, the teacher data is information indicating the shape of the building included in the learning image data.

  FIG. 11 is a diagram illustrating an example of teacher data. The teacher data shown in FIG. 11 is a bitmap image corresponding to the learning image including the window image shown in FIG. The teacher data shown in FIG. 11 includes a building area (eg, A) whose area belongs to the range S, a building area (eg, B) that belongs to the range M, and a building area (eg, C) that belongs to the range L. It is distinguished. The teacher data may be, for example, an image in which the dot value of a region without a building is set to 0, and the dot values of building regions whose areas are ranges S, M, and L are set to 1, 2, and 3, respectively. . The teacher data includes an image having a dot value of 1 in a building region whose area belongs to the range S, an image having a dot value of 1 in a building region belonging to the area M, and an area having a range L. It may be an image corresponding to a plurality of layers with an image having a dot value of 1 in a building area belonging to.

  In calculating the error, the learning execution unit 52 cuts out an image at a position corresponding to 16 × 16 dots in the center of the window image of the learning image from the teacher data, and outputs each of the output images (S, M, L). Compare with teacher data. Here, the learning execution unit 52 calculates an error from the output image (S) for the area without buildings and the area of buildings belonging to the range S in the teacher data, but for the areas of buildings belonging to the ranges M and L. Does not calculate the error. Similarly, the learning execution unit 52 does not calculate an error from the output image (M) for the area of the building belonging to the ranges S and L, and the error from the output image (L) for the area of the building belonging to the ranges S and M. Is not calculated. As a result, learning proceeds so that each of the individual units 541, 542, and 543 is suitable for detection of buildings having the area ranges S, M, and L.

  Next, the learning execution unit 52 changes the values of parameters such as weights in the individual units 541, 542, and 543 by the error back-propagation method (back propagation) based on the calculated error (step S124). . Further, the learning execution unit 52 integrates errors to be propagated from the uppermost layer of each of the individual units 541, 542, and 543 to the lowermost layer of the common unit (step S125), and based on the accumulated error, The value of a parameter such as a weight in the common unit 540 is changed by a back propagation method or the like (step S126).

  The learning process shown in step S103 and step S104 (FIG. 9) is repeated until all window images used for learning are acquired from a certain learning image. This set of processes is repeated for each of the learning detectors 54, whereby each learning detector 54 is learned. Here, instead of the process of step S103, a process of cutting out a plurality of window images used for learning may be performed. In this case, the process of step S104 may be repeatedly performed so that the process of inputting the window image and learning the learning detector 54 is performed for each of the extracted window images.

  Next, the learning detector 54 for evaluating the learned learning detector 54 and evaluating the learning detector 54 for executing the processing of actually extracting the building region from the processing target image is selected as the execution detectors 62, 63, 64. Details will be described.

  FIG. 12 is a flowchart showing an example of processing for evaluating the learning detector 54. In this process, first, the evaluation data acquisition unit 56 acquires an evaluation image and correct answer data from the storage unit 12 (step S201). The evaluation image may be the same as or different from the learning image. The scale of the evaluation image is the same as the learning image. The correct answer data is an image indicating the area of the building belonging to each of the area ranges S, M, and L in the evaluation image. When the evaluation image and the learning image are the same, the correct answer data is teacher data. Good. Although not illustrated in FIG. 12, the evaluation data acquisition unit 56 sets the size of the evaluation image so as to match the scale of the learning detector 54 as in the learning data acquisition unit 51.

  Next, the evaluation execution unit 57 cuts out a window image to be input to the learning detector 54 from the evaluation image (step S202). More specifically, the evaluation execution unit 57 cuts out the window image so that a predetermined number of dots are shifted from the cut out area of the window area cut out so far. The predetermined number of dots can be any size from 1 dot to 16 dots. The predetermined upper limit of 16 corresponds to the output of the learning detector 54 being a 16 × 16 dot image. The predetermined number is less than the vertical or horizontal size of the output of the learning detector 54. The evaluation execution unit 57 inputs the window image cut out from the evaluation image to the learning detector 54 (step S203), and outputs the output images (S, M, and S) of the individual units 541, 542, and 543 of the learning detector 54, respectively. L) is acquired (step S204). Here, the evaluation execution unit 57 binarizes the acquired output image based on whether the existence probability value of each dot is larger or smaller than the threshold value, and stores the binarized output image in the storage unit 12. To do. In the following processing, the output image indicates a binarized output image. Then, the processing from step S202 to S204 is repeated until the processing of the learning detector 54 is performed for all window images (see step S205).

  When the output images (S, M, L) for all the window images are obtained, the evaluation execution unit 57 and the entire image (S) in which the output images (S) are arranged at positions corresponding to the window images. The whole image (M) in which the output image (M) is arranged at a position corresponding to the window image, and the whole image (L) in which the output image (L) is arranged at a position corresponding to the window image. Are generated (step S206). More specifically, the evaluation execution unit 57 arranges the output images (S, M, L) so as to be shifted from each other by a predetermined number of dots so as to correspond to the arrangement of the window images. L). Here, when the predetermined number of dots, which is the magnitude of the shift when the window image is cut out, is smaller than 16 dots, at least some of the output images (S, M, L) obtained from each window image are It overlaps with output images (S, M, L) for other window images. For the dots whose positions overlap in the output of a plurality of window images, the evaluation execution unit 57 uses the average value obtained by averaging the dot values of the output image as the dot value in the entire image (S, M, L). Thereby, even when the boundary between adjacent output images (S, M, L) is not smoothly connected, it is possible to prevent inconsistencies caused by the boundary from appearing in the entire image.

  Then, the evaluation execution unit 57 compares the entire image and the correct answer data, and evaluates the accuracy of each of the individual units 541, 542, and 543 of the learning detector 54 (step S207). For example, in the evaluation of accuracy, the evaluation execution unit 57 obtains a ratio (Recall) in which the area determined to be a building in the output image (S) is present in the area where the building belonging to the area range S exists in the correct answer data. Do that. The evaluation execution unit 57 similarly evaluates the accuracy of the building regions belonging to the area ranges M and L of the correct answer data and the building regions existing in the output image (M) and the output image (L). .

  Through the processing from step S202 to step S207, the accuracy of one learning detector 54 is evaluated. If the accuracy is not evaluated for all the learning detectors 54, the evaluation executing unit 57 repeats the process from step S202 (step S208), whereby the evaluation executing unit 57 is all the learning detectors 54. Assess the accuracy of

  FIG. 13 is a diagram illustrating an evaluation result by the evaluation execution unit 57. “No” in FIG. 13 indicates the number assigned to the learning detector 54 as in FIG. In the example of FIG. 13, with respect to the output of the individual unit 541 whose area range is S, the learning detector 54 having a scale of 1.0 times and a dilation model has the highest accuracy. Regarding the output of the individual unit 542 whose area range is M, the learning detector 54 of the pooling model with the scale of 1.0 times has the highest accuracy, and the output of the individual unit 543 whose area range is L. The learning detector 54 having a scale of 0.5 times and a pooling model is most accurate.

  When the accuracy of the learning detector 54 is evaluated, the detector selection unit 58 sets the learning detector 54 having the highest accuracy as the execution detectors 62, 63, and 64 for each of the area ranges S, M, and L. Select (step S209). The execution detector 62 is a combination of a common unit 540 (hereinafter referred to as a common unit 620) and an individual unit 541 (hereinafter referred to as an individual unit 621) included in the learning detector 54 having the highest accuracy in the area range S. is there. The execution detector 63 is a combination of the common unit 540 (hereinafter referred to as the common unit 630) and the individual unit 542 (hereinafter referred to as the individual unit 631) included in the learning detector 54 having the highest accuracy in the area range M. is there. The execution detector 64 is a combination of the common unit 540 (hereinafter referred to as the common unit 640) and the individual unit 543 (hereinafter referred to as the individual unit 641) included in the learning detector 54 that is also highly accurate with respect to the area range L. It is.

  Here, as can be seen from the description of FIG. 13, the dilation model tends to catch small changes compared to the pooling model, and therefore the dilation model is advantageous when the area range (the maximum value) is small. Therefore, the pooling model is advantageous when the area range is large. In addition, when the scale is small, detailed information is reduced, while it tends to be easy to determine the shape of a large-scale building. Therefore, when the area range (the maximum value) is small, the larger scale is advantageous, and when the area range is large, the smaller scale is advantageous.

  Therefore, also in the example of FIG. 13, the learning detector 54 that is 1.0 times larger than the large scale and the dilation model is selected as the execution detector 62 corresponding to the one having the smallest maximum area range. The learning detector 54, which is a pooling model, is selected as the execution detector 64 corresponding to the one having the largest maximum area range.

  The detector selection unit 58 simply stores in the storage unit 12 information indicating the learning detector 54 to which the target data input unit 65 (to be described later) inputs a processing target image and obtains an output image. 54 may be selected, or the learning detector 54 may be selected by copying the common unit 540, the individual unit 541, and the like of the selected learning detector 54 as an entity of the execution detectors 62, 63, and 64. You may select as 62,63,64.

  Next, processing for determining a building area from the processing target image using the execution detectors 62, 63, and 64 will be described. FIG. 14 is a diagram illustrating an outline of processing for determining a building area.

  First, the target data input unit 65 inputs the processing target image to the execution detector 62 suitable for the area range S, and the output acquisition unit 66 outputs the entire output image (S) based on the output of the execution detector 62. Obtain (step S301). The entire output image (S) is an image showing an area in which the execution detector 62 determines that a building exists for the entire processing target image. Similarly, an overall output image (M) and an overall output image (L), which will be described later, are images indicating regions where it is determined by the execution detectors 63 and 64 that a building exists, respectively.

  FIG. 15 is a flowchart showing the flow of processing for generating the entire output image from the processing target image, and is a diagram for explaining the processing in step S301 in detail. First, the target data input unit 65 matches the scale of the processing target image with the scale set in the execution detector 62 (step S321). If the scale of the processing target image is different from the scale of the execution detector 62, the target data input unit 65 adjusts the scale by enlarging or reducing the processing target image. Next, the target data input unit 65 cuts out a window image from the processing target image with the scale adjusted (step S322). Since the window image size and the method for cutting out the window image from the processing target image are the same as the method for cutting out the window image from the evaluation image, description thereof will be omitted. Next, the target data input unit 65 inputs a window image to the execution detector 62 (step S323). Then, the execution detector 62 performs a process of detecting a building area for the input window image, and the output acquisition unit 66 acquires the output image of the execution detector 62 (step S324). Here, although not shown, the output acquisition unit 66 binarizes the acquired output image based on whether the value of the probability of existence of each dot is larger or smaller than the threshold, and binarized output image Is stored in the storage unit 12. In the following processing, the output image indicates a binarized output image. Then, the processes of steps S322 to S324 are repeated until the process of the learning detector 54 is performed for all window images (see step S325).

  The building detector corresponds to the individual units 621, 631, and 641 of the execution detectors 62, 63, and 64, and the feature information of the processing target image input to the building detector is the common unit 620, 630, and 640, respectively. May be output. Note that the learning detector 54 and the execution detectors 62, 63, and 64 may not include the common units 540, 620, 630, and 640. In this case, the learning input image and the processing target image are input for each of the area ranges S, M, and L, and the feature information of the processing target image input to the building detector is simply the processing target image or its window image. It may be.

  When output images for all window images are obtained, the evaluation execution unit 57 generates an entire output image (S) in which the output images are arranged at positions corresponding to the window images (step S326).

Next, the filter unit 67 filters the entire output image (S) based on the area (step S302). More specifically, in this process, the filter unit 67 calculates the area of an area determined to have a building in the entire output image (S) (obtained from the number of dots and the scale of the area), and the area is A region that is not within the allowable range corresponding to the area range S is deleted from the entire output image (S). Specifically, the allowable range is less than 89.2 m ² . Note that the processing of the filter unit 67 may not be performed.

  The target data input unit 65 inputs the processing target image to the execution detector 63 suitable for the area range M, and the output acquisition unit 66 outputs the entire output image (M) based on the output of the execution detector 63. Obtain (step S303). The details of this processing are the same as the processing for acquiring the entire output image (S) from the execution detector 62, and thus detailed description thereof is omitted.

Next, the filter unit 67 applies a filter based on the area to the entire output image (M) (step S304). More specifically, in this process, the filter unit 67 calculates the area of an area determined to have a building in the entire output image (M) (obtained from the number of dots and the scale of the area), and the area is A region that is not within the allowable range corresponding to the area range M is deleted from the entire output image (M). Specifically tolerance is less than 22.3 m ² or more 89.2m ^2.

  The target data input unit 65 inputs the processing target image to the execution detector 64 suitable for the area range L, and the output acquisition unit 66 outputs the entire output image (L) based on the output of the execution detector 64. Obtain (step S305). The details of this processing are the same as the processing for acquiring the entire output image (L) from the execution detector 62, and thus detailed description thereof is omitted.

Next, the filter unit 67 applies a filter based on the area to the entire output image (L) (step S306). More specifically, in this process, the filter unit 67 calculates the area of an area determined to have a building in the entire output image (L) (obtained from the number of dots and the scale of the area), and the area is A region that is not within the allowable range corresponding to the area range L is deleted from the entire output image (M). Specifically, the allowable range is 65.4 m ² or more.

  Then, the integration unit 68 executes a process of enlarging or reducing at least one of these so that the scales of the overall output image (S), the overall output image (M), and the overall output image (L) match. (Step S307). This process may be performed before the process of the filter unit 67.

  The integration unit 68 integrates the processed overall output image (S), overall output image (M), and overall output image (L) (step S308). In other words, the integration unit 68 determines an area recognized as a building in any one of the overall output image (S), the overall output image (M), and the overall output image (L) as an area with a building, and the determination. An integrated image showing the marked area is generated. More specifically, the integration unit 68 obtains the integrated image by taking the logical sum of each dot of the filtered overall output image (S), overall output image (M), and overall output image (L). Generate. Here, it is assumed that each dot of the overall output image (S), the overall output image (M), and the overall output image (L) is 1 in an area where it is determined that a building exists, and 0 in an area other than that. .

  Then, the image output unit 69 outputs the image generated by the integration unit 68 to the storage unit 12 and the display output device.

  The execution detectors 62, 63, and 64 having scales and model types suitable for the area ranges S, M, and L are used to acquire the images in which the building areas are determined, and the integration unit 68 acquires the images. By integrating the images, it is possible to improve the accuracy of the building determined from the processing target image, and particularly to reduce oversight.

  For example, based on the evaluation result shown in FIG. 13, the detector selection unit 58 sets the execution detectors 62, 63, 64 as a scale of 1.0 times, a dilation model, and a scale of 1.0 times, respectively. When the pooling model and the learning detector 54 of the pooling model are selected with a scale of 0.5, in some experiments, the value of Recall, which is an overlooked index, is 87.0%, and the execution detectors 62, 63, 64 82.0% which is the value when the pooling model is 1.0 times the scale, and the execution detectors 62, 63 and 64 are all the dilation model with the scale 1.0 times. The value in this case is 83.8%. Here, the value of “Recall” is a number obtained by dividing the number of areas determined as having a building out of the area of the building given as a correct answer by the number of building areas given as a correct answer. This effect is very significant because it is generally not easy to reduce oversights in the determination of building areas.

  By using the building extraction system that combines the execution detectors 62, 63, and 64 described so far, it is possible to more accurately detect structures and buildings of various sizes from remote sensing images such as aerial photographs and satellite images. Become able to recognize. Then, the building extraction system can be used for grasping a new construction or a loss of a building, and it is possible to obtain basic information on statistics regarding a house change. In addition, since it is possible to extract building areas with high accuracy, it is possible to more easily grasp the temporal transition of individual buildings, and the detailed attributes of buildings (for example, detached houses, It is also easier to determine the type of building such as a condominium or a factory.

  Then, by automating these information extraction operations relating to buildings from images, it is possible to perform the operations for processing a wide range of the ground surface at low cost and at high speed.

  Although the embodiments of the present invention have been described so far, various modifications can be made within the scope of the spirit of the present invention. For example, the area range is not three, but may be two or four or more. Also, the number of model types and the number of scale types may be different. The individual unit may not be optimized according to the range of the area of the building. For example, the individual unit may be optimized according to a group classified by another method such as a height of a building.

  1 learning server, 11 processor, 12 storage unit, 13 communication unit, 14 input / output unit, 30 elements, 31, 32, 33, 34, 35, 36, 37 unit, 51 learning data acquisition unit, 52 learning execution unit, 53 Learning detector set, 54 Learning detector, 540 Common unit, 541, 542, 543 Individual unit, 56 Evaluation data acquisition unit, 57 Evaluation execution unit, 58 Detector selection unit, 61 Execution detector set, 62, 63, 64 Execution detector, 620, 630, 640 common unit, 621, 631, 641 individual unit, 65 target data input unit, 66 output acquisition unit, 67 filter unit, 68 integration unit, 69 image output unit.

Claims (7)

1st building detector learned about a plurality of buildings belonging to the first group using learning input images and teacher data of information indicating the shapes of the plurality of buildings included in the learning input images When,
A second building detector that has learned a plurality of buildings belonging to a second group different from the first group using a learning input image and teacher data of information indicating the shapes of the plurality of buildings; ,
An input unit for inputting feature information of an input image obtained by photographing a learning target region on the ground surface from the sky to the first building detector and the second building detector;
An integration unit that integrates the output of the first building detector and the output of the second building detector with respect to the feature information of the input image;
Including
The first building detector and the second building detector include different types of neural networks.
Building extraction system. In the building extraction system according to claim 1,
The areas of the plurality of buildings belonging to the first group belong to the first range,
The areas of the plurality of buildings belonging to the second group belong to a second range different from the first range,
Building extraction system. In the building extraction system according to claim 2,
A filter for removing a building included in the output of the first building detector and a building included in the output of the second building detector based on an area;
Building extraction system. In the building extraction system according to any one of claims 1 to 3,
The first building detector includes a convolution layer that performs an extended convolution operation;
The second building detector includes a pooling layer;
Building extraction system. In the building extraction system according to claim 2,
The maximum value of the first range is smaller than the maximum value of the second range,
The first building detector includes a convolution layer that performs an extended convolution operation;
The second building detector includes a pooling layer;
Building extraction system. In the building extraction system according to any one of claims 1 to 3,
For a plurality of buildings belonging to either the first group or the second group, the first type of neural network is included, and the learning input image and the learning input image A first candidate detector trained using teacher data of information indicating the shape of the building, and a second type of neural network, the learning input image and the learning input image An evaluation unit that evaluates the detection accuracy of the shape of each of the second candidate detectors learned using the teacher data of information indicating the shape of a plurality of buildings;
Based on the detection accuracy evaluated by the evaluation unit, one of the first candidate detector and the second candidate detector is replaced with one of the first building detector and the second building detector. A detector selection unit to select one of them,
Further including a building extraction system. In the building extraction system according to any one of claims 1 to 6,
The integration unit, for the feature information of the input image input, an area recognized as a building in either the output of the first building detector or the output of the second building detector, To determine a certain area,
Building extraction system.

Images