CN108776772B - Cross-time building change detection modeling method, detection device, method and storage medium - Google Patents
Cross-time building change detection modeling method, detection device, method and storage medium Download PDFInfo
- Publication number
- CN108776772B CN108776772B CN201810411612.8A CN201810411612A CN108776772B CN 108776772 B CN108776772 B CN 108776772B CN 201810411612 A CN201810411612 A CN 201810411612A CN 108776772 B CN108776772 B CN 108776772B
- Authority
- CN
- China
- Prior art keywords
- image
- cutting
- semantic segmentation
- images
- size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/182—Network patterns, e.g. roads or rivers
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of remote sensing satellite image analysis, and discloses a semantic segmentation model modeling method, a semantic segmentation model modeling device, semantic segmentation model modeling equipment and a storage medium for image contrast analysis. The device specifically comprises: and setting a label unit, an image cutting unit, an input and output corresponding unit and a detection model obtaining unit so as to obtain an automatic detection model. The invention uses the semantic segmentation model based on the pixel level, realizes the combination of the deep learning algorithm and the high-resolution satellite image technology, and can realize the automatic detection of the building change spanning a long time in the same area.
Description
Technical Field
The invention relates to the technical field of remote sensing satellite image analysis, and discloses a device, a method and a storage medium for cross-time building change detection modeling and detection based on remote sensing satellite image analysis.
Background
The description of the background of the invention pertaining to the related art to which this invention pertains is given for the purpose of illustration and understanding only of the summary of the invention and is not to be construed as an admission that the applicant(s) explicitly or putatively admitted the prior art of the filing date of the first-filed application of the present invention.
Labeling of terrestrial objects (roads, buildings, etc.) in satellite images is an important issue in the field of satellite images. In the territorial surveillance business, an important work is to supervise the building, dismantling, modifying and expanding of buildings on the ground. For example, if a parcel is present in a building without approval, a person needs to be sent in the field to investigate whether illegal floor occupation behavior occurs. If the parcel is sold to a developer but not actually constructed, it is necessary to investigate whether problems occur in the country or in the funding chain of the developer. If residential housing/business use is abnormally enlarged, it is necessary to investigate whether there is a violation of a building, and thus a land office officer needs a lot of patrol every day.
When the traditional building detection method is used for processing the problem, the texture or shape features of the target are usually extracted, and building detection and change analysis are performed on the satellite image by using some typical operators.
With the gradual maturity of high-resolution remote sensing satellite technology, many countries in the world gradually become aware of the great practical value of high-resolution remote sensing satellites in the fields of territorial inspection, disaster reduction and prevention, map drawing and the like. For a large amount of high-quality remote sensing satellite image data, the traditional building detection method depends on manual extraction of features and cannot fully mine the association between the data, so that the detection precision is not high. The typical characteristic operator may be disabled due to the style difference of buildings on different land parcels, and a universal expert rule cannot be found, so that the traditional machine learning algorithm has obvious limitation on the problem.
Disclosure of Invention
In order to solve the technical problem of automatic detection of building changes spanning 20 years of time scale in the same region, a first aspect of the invention relates to a semantic segmentation model modeling device for image contrast analysis, which comprises: a label unit is arranged, remote sensing image groups shot at the front time and the back time of the same area are selected from the satellite remote sensing images, manual identification and labeling are carried out, and labels are arranged in the corresponding areas; the image cutting unit is used for cutting the image of the artificial labeling area constructed in the step of setting the label into a sufficient number of small-size images to form a small-size image data set for training a semantic segmentation model; the input and output corresponding unit is used for stacking multi-channel images of the front time and the rear time in the small-size picture data set obtained in the image cutting step as input, and the corresponding change label is used as a true value label in a target; and obtaining a detection model unit, and continuously optimizing FCN network parameters until convergence by using the small-size picture data set in the input and output corresponding step through a deep learning frame, thereby obtaining an automatic detection model. The method combines a deep learning algorithm with a high-resolution satellite image technology, and uses a semantic segmentation model based on a pixel level to realize the algorithm.
Preferably, the image segmentation unit generates coordinate point pairs by using uniformly distributed random numbers and performs random segmentation in the image of the artificial labeling area.
Preferably, the image segmentation unit randomly segments the image in the manual labeling area according to a multi-angle placing mode of 0 °, 30 °, 60 °, 90 ° and the like.
Preferably, the semantic segmentation model modeling apparatus for image contrast analysis further includes: and an amplification data set unit for further amplifying the scale of the training data set for the small-size pictures obtained by cutting.
Preferably, the semantic segmentation model modeling apparatus for image contrast analysis further includes: and the image preprocessing unit is used for carrying out color homogenizing treatment on the satellite remote sensing image group to be processed.
Preferably, the input/output correspondence unit first normalizes the pictures to eliminate interference caused by color difference in each input picture.
The second aspect of the invention relates to a semantic segmentation model modeling method for image contrast analysis, and the specific steps of the method correspond to the unit functions of the device one by one.
For automatic analysis of remote sensing satellite images, a third aspect of the invention relates to a method of detecting terrain variations, comprising: an image cutting step, wherein front and rear images of a region to be detected in the remote sensing image are cut by small-size windows respectively; an image stacking step of stacking multi-channel images of front and rear time in a small-size picture data set obtained in the image cutting step; transmitting the stacked images into a model established by the semantic segmentation model modeling method for the images, and detecting and outputting, wherein the model outputs the ground feature change judgment results of each small-size area with the same quantity; and a result splicing step, namely splicing all the change detection results into a complete large-size picture according to the corresponding sequence to obtain the detection result of the change condition of the ground object in the region. Through the scheme, the land monitoring efficiency of at least more than 100 times can be realized on the spatial scale, namely, only a small land block needs to be manually marked for deep learning algorithm training, the algorithm can be directly deployed to a land area of more than 100 times to finish automatic identification of building change, the current state and soil examination mode can be improved, and the efficiency is greatly improved.
Preferably, a redundant cutting method is adopted in the image cutting step.
A fourth aspect of the invention relates to a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of any of the above-mentioned methods when executing the program.
A fifth aspect of the invention relates to a computer-readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, carries out the steps of any of the methods described above.
Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the embodiments will be briefly described as follows:
FIG. 1 is a logic block diagram of an apparatus for detecting a change of a ground object across time based on a deep learning semantic segmentation model according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an image structure of the input and output of a model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a method of an apparatus for cross-time feature change detection based on a deep learning semantic segmentation model according to a first embodiment of the present invention;
FIG. 4 is a logic block diagram of an apparatus for detecting a change of a feature across time based on a deep learning semantic segmentation model according to a second embodiment of the present invention;
FIG. 5 is a schematic diagram of a method for cross-time feature change detection based on a deep learning semantic segmentation model according to a second embodiment of the present invention;
fig. 6 is a schematic view of a device for detecting a feature change according to a third embodiment of the present invention;
FIG. 7 is a diagram (redundancy cutting strategy) of an identification result with the size of 256 × 256 obtained by cutting out a small block region to be identified from an original remote sensing image by using a blue rectangular frame with the size of 256 × 256 and then performing network prediction;
fig. 8 is a schematic diagram of a method for detecting a feature change according to a third embodiment of the present invention;
9a-c, taking the remote sensing image of 2015 and the remote sensing image of 2017 of a local area in Fushan City in Guangdong province as an example, are diagrams illustrating the effect of the steps of the method for detecting the land feature change provided by the third embodiment of the invention;
FIG. 10 shows a prior art FCN schematic;
FIG. 11 illustrates the process of FCN convolution and deconvolution upsampling;
fig. 12 illustrates a comparison of results obtained using 32-fold, 16-fold and 8-fold upsampling, respectively, after using the FCN full convolution neural network.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and thus the scope of the present invention is not limited by the specific embodiments disclosed below.
The following discussion provides multiple embodiments of the invention. While each embodiment represents a single combination of the inventions, different embodiments of the inventions may be combined or substituted and, thus, the inventions are intended to include all possible combinations of the same and/or different embodiments which are described. Thus, if one embodiment comprises A, B, C and another embodiment comprises a combination of B and D, then the invention should also be construed as including embodiments that comprise A, B, C, D in all other possible combinations, although this embodiment may not be explicitly recited in the following text.
Convolutional Neural Networks (CNN) have achieved great success and wide application in image classification, image detection, and the like since 2012. The conventional CNN-based segmentation method generally works as follows: to classify a pixel, an image block around the pixel is used as input to the CNN for training and prediction. These abstract features help in classification and make a good decision as to what classes of objects are contained in an image. CNN is powerful in that its multi-layer structure can learn features automatically, and can learn features of multiple levels: the sensing domain of the shallower convolutional layer is smaller, and the characteristics of some local regions are learned; deeper convolutional layers have larger perceptual domains and can learn more abstract features. These abstract features are less sensitive to the size, position, orientation, etc. of the object, thereby contributing to an improvement in recognition performance.
This approach has several disadvantages: one is that the storage overhead is large. For example, the size of the image block used for each pixel is 15x15, the required storage space is 225 times that of the original image. Secondly, the calculation efficiency is low. The adjacent pixel blocks are substantially repetitive, and the convolution is calculated for each pixel block one by one, and this calculation is also largely repetitive. Thirdly, the size of the pixel block limits the size of the sensing region. The size of the pixel block is usually much smaller than the size of the whole image, and only some local features can be extracted, so that the performance of classification is limited. That is, because details of some objects are lost, the specific outline of the object cannot be given well, and it is difficult to accurately segment the object to which each pixel belongs.
To address this problem, Jonathan Long et al of UC Berkeley proposed a full volumetric Networks (FCN) for segmentation of images. The network attempts to recover from the abstract features the class to which each pixel belongs. I.e. from image level classification to pixel level classification.
FCNs convert the fully-connected layers in traditional CNNs into convolutional layers one by one. In the conventional CNN structure, the first 5 layers are convolutional layers, the 6 th and 7 th layers are respectively a one-dimensional vector with a length of 4096, and the 8 th layer is a one-dimensional vector with a length of 1000, which respectively corresponds to a probability of 1000 categories. FCN represents these 3 layers as convolutional layers, and the sizes (number of channels, width, height) of the convolutional cores are (4096, 1, 1), (1000, 1, 1), respectively. All layers are convolutional layers, so it is called a full convolutional network, as shown in fig. 10.
It can be seen that after a plurality of convolutions (and posing), the resulting image becomes smaller and lower in resolution (coarse image), and in order to restore the resolution of the original image from this low-resolution coarse image, the FCN uses upsampling. For example, after 5 times of convolution (and pooling), the resolution of the image is reduced by 2, 4, 8, 16, and 32 times. For the output image of the last layer, up-sampling by 32 times is needed to obtain the size as the original image.
This upsampling is performed by deconvolution (deconvolution). Deconvolution of the output of layer 5 (32 x magnification) to the original size still yields results that are not accurate enough and some details are not recoverable. Jonathan then deconvolves the output of layer 4 and layer 3 in turn, requiring 16 and 8 times upsampling, respectively, with a somewhat finer result. The process of this convolution and deconvolution up-sampling is illustrated in fig. 11.
Fig. 12 illustrates a comparison of the results obtained with 32 x, 16 x and 8 x upsampling, which can be seen to be increasingly accurate.
The deep learning model backbone network used in the present invention is a full convolution semantic segmentation network (FCN). The FCN changes the full-connection structure in the later layers in the traditional Convolutional Neural Network (CNN) structure into a convolutional structure, so that the network can accept the input of pictures with any size, and the CNN can complete the extraction of high-layer semantic information and simultaneously fuse the previous layers to obtain more complete contour information, and finally obtain a semantic segmentation result graph with the size consistent with that of an original image.
The technical solution adopted by the present invention is specifically described below.
Example one
Fig. 1 is a logic block diagram of an apparatus for cross-time feature change detection based on a deep learning semantic segmentation model according to an embodiment of the present invention.
A semantic segmentation model modeling apparatus 10 for image contrast analysis, comprising:
the tag unit 101 is provided to select a remote sensing image group photographed at a time before and after the same area from satellite remote sensing images, perform manual identification and labeling, and set a tag in a corresponding area.
For example, remote sensing images (RGB three-channel images) corresponding to the same region of two different years but with the same geographical position aligned are screened from satellite image data, and then manually marked as learning samples. The size of the label is the same as the size of the image, but the number of channels is 1 (i.e., grayscale).
Taking the mark "new building" as an example, the building regions that do not appear in the later year but in the earlier year are marked, the label of the corresponding region is set to 1, and the other regions are set to 0.
Taking the mark "disappearing" water body as an example, the water wave area which is present in the earlier year and does not appear in the later year is marked, the label of the corresponding area is set to be 1, and the other areas are 0.
Preferably, the size of the area to be manually marked is 1/100 or less for the total area size to be automatically identified by the machine, so as to reduce the manual workload while the sample meets the identification requirement.
Preferably, small sample areas are respectively cut at a plurality of positions of the total area automatically identified by the machine, and manual marking is carried out to be used as a learning sample so as to ensure that the earth surface sample is rich in earth surface object types.
The image segmentation unit 102 segments the image of the artificial labeling area constructed in the step of setting labels into a sufficient number of small-sized pictures to form a small-sized picture data set for training a semantic segmentation model.
For example, a sufficient number of small-sized pictures are cut from the image of the artificial labeling area constructed by the labeling unit 101 to construct a data set for training the network model.
For example, on the image of the manually labeled starfish region, a sufficient number of cuts are made with the small size of pixels 256 × 256 to generate a sufficient number of small size pictures of pixels 256 × 256 to form a data set for training of the model. A sufficient number, for example of the order of tens or hundreds of thousands, to ensure that the manually marked regions are all cut.
Preferably, the image segmentation unit 102 includes: and generating coordinate point pairs by using uniformly distributed random numbers, and randomly cutting in the image of the artificial labeling area to provide samples with more angles, thereby further enhancing the training effect. For example, the multi-angle samples are obtained according to the multi-angle arrangement modes such as 0 degrees, 30 degrees, 60 degrees, 90 degrees and the like, and the problem of inaccurate later-stage identification caused by that part of the earth surface is always positioned at the cutting edge due to single-angle cutting is avoided.
Preferably, the image segmentation unit 102 includes: and further amplifying the scale of the training data set by using a mode of turning the small-size picture obtained by cutting up and down and turning the small-size picture left and right.
It should be noted that all the operations of cutting and turning over should be simultaneously executed on the remote sensing images of the early year and the late year and the corresponding manual labeling labels, that is, each time a position coordinate of a region to be cut with a size of 256 × 256 is generated, cutting is executed once on the same positions of the images of the early year, the images of the late year and the manual labeling images, and a picture triple (year 1, year 2, change mark 0 or 1) is extracted.
An input/output corresponding unit 103 for inputting the multi-channel image stack of the small-size image data set obtained in the image cutting step in the previous and subsequent time, and using the corresponding change label as the true label in the target;
for example, in the picture triplet obtained by the image segmentation unit 102, the small-sized pictures of year 1 and year 2 are stacked in multiple channels as actual input, and the corresponding change label (0 or 1) is used as the true value label of the network optimization target.
For example, in the data set constructed by the image segmentation unit 102, 256 × 256 multi-channel images (e.g., RGB3 channel images) corresponding to year 1 and year 2 are stacked as the input of the actual network, e.g., two RGB3 channel images are stacked as 6 channel input, and the corresponding change label is used as the true value (value 1) label used in the network optimization target. See fig. 2 for a schematic.
And the detection model obtaining unit 104 is used for continuously optimizing the FCN network parameters until convergence by using the small-size picture data sets in the input and output corresponding steps through a deep learning framework, so that an automatic detection model is obtained.
And continuously optimizing FCN network parameters (such as global parameters defined by the Tensorflow) by using the data set constructed in the second step by means of a deep learning framework (such as the Tensorflow and the like) until the network performance tends to converge and stops training, and finally obtaining a model which can be used for automatically detecting the change of the building.
TensorFlow is a computing framework that Google officially opened on 11/9/2015. The TensorFlow computation framework may well support various algorithms for deep learning, but its application is not limited to deep learning either. TensorFlow is a general computing framework developed by the brain team of Google leading by Jeff Dean based on the DistBeief, the first generation deep learning system inside Google. Distebrief is an internal deep learning tool developed in google 2011 that has had great success inside google. The brain team in google improved distalief and formally published a computing framework tensrflow based on the Apache 2.0 open source protocol in 11 months 2015. Compared with DistBeief, the TensorFlow calculation model is more universal, the calculation speed is higher, more calculation platforms are supported, the supported deep learning algorithm is wider, and the stability of the system is higher. For technical details of the TensorFlow platform itself, reference may be made to the Google paper TensorFlow, Large-Scale Machine Learning on Heterogeneous Distributed Systems. And 6, 16.2.2017, the Google formally releases the version 1.0 of the Google TensorFlow outwards, and the API interface of the released version completely meets the requirement of the stability of the production environment.
In this embodiment, in consideration of the phenomenon that the color distributions of different areas in a picture of an acquired remote sensing satellite image are not consistent and the color distributions of multiple scenes are not consistent, the method may further include, for example, a remote sensing image preprocessing unit, and before the global image is cut, the image with color distortion or obvious stitching traces is subjected to color equalization by using a preprocessing method such as histogram matching for the global satellite remote sensing image, so that the entire data set maintains a consistent distribution state.
The scheme establishes a deep learning semantic analysis segmentation model based on the remote sensing satellite image, and can be used for solving the problem of automatic detection of building changes spanning 20 years of time scale in the same region.
Fig. 3 is a schematic diagram of a method for detecting a change of a feature across time based on a deep learning semantic segmentation model according to an embodiment of the present invention. The contents of each step of the method are completely in one-to-one correspondence with the functions of each unit of the device described in this embodiment, and are not described herein again.
Example two
It is considered that before the remote sensing images are transmitted into the FCN backbone network, there may be color distribution tendencies in different remote sensing images (as mentioned in the first embodiment, different areas of the same image may have different color distribution states, and there may also be such color distribution differences between different images). Based on this, we perform normalization processing on each picture, and further eliminate the interference on network training caused by the difference of color distribution among different images.
The apparatus, functions, and method steps not described in detail in this embodiment are the same as those in the above embodiments, and are not described again here.
Fig. 4 is a logic block diagram of an apparatus 20 for detecting a change of a feature across time based on a deep learning semantic segmentation model according to a second embodiment of the present invention.
On the basis of the first embodiment, as shown in fig. 4, the apparatus 20 for detecting a change of a feature across time based on a deep learning semantic segmentation model further includes a picture normalization unit 1041, configured to eliminate a color distribution difference between different images, for example, a mean value and a standard deviation of each input picture are calculated. For example, for a 256 × 3 image, we calculate the mean and standard deviation for a dimension of 1 × 3, and then subtract the mean from the original and divide by the standard deviation.
Fig. 5 is a schematic diagram of a method for detecting a change of a feature across time based on a deep learning semantic segmentation model according to a second embodiment of the present invention. The contents of each step of the method are completely in one-to-one correspondence with the functions of each unit of the device described in this embodiment, and are not described herein again.
EXAMPLE III
This embodiment describes an apparatus and method for detecting terrain variations using the method of cross-time terrain variation detection based on a deep-learning semantic segmentation model described above.
The apparatus, functions, and method steps not described in detail in this embodiment are the same as those in the above embodiments, and are not described again here.
Fig. 6 shows a schematic diagram of a surface feature change detection device 30 according to a third embodiment of the present invention.
The feature change detection device 30 specifically includes:
the image cutting unit 301 cuts the front and rear images of the region to be detected in the remote sensing image with small-size windows respectively.
And (3) performing sliding cutting on the large-size satellite remote sensing images (such as image 1 and image 2) of the area to be detected for two years in a 256 × 256 window in a row-column scanning manner.
In order to further improve the problem of inaccurate detection caused by interference of factors such as incomplete buildings in the edge area and the like, so that the detection effect is further improved, the image segmentation unit 301 of the embodiment preferably provides a more optimized "redundant window detection strategy". As shown in fig. 7, a small area to be identified can be cut out from an original remote sensing image by using a blue rectangular frame of 256 × 256, and then an identification result map with the same size of 256 × 256 is obtained through network prediction, but only a yellow area with the size of 128 × 128 at the center is reserved as an effective result, and then the areas are spliced back to a large size change detection map according to a certain overlapping rate. Since the recognition result of the central area is finer than that of the edge area, a better recognition result can be obtained using such a redundant manner.
Based on the fixed model performance capability, the invention further provides a redundant window-dividing detection strategy, which solves the problem of inaccurate detection caused by interference of factors such as incomplete buildings in edge areas and the like, and further improves the detection effect.
The image stacking unit 302 stacks the multi-channel images at the front and rear time in the small-size picture data set obtained in the image segmentation step.
And images of the corresponding region for two years are stacked in the manner described above when processing the training set.
Using the model unit 303, the stacked image is passed into a model built by the semantic segmentation model modeling method for images as claimed in claim 7.
Then, the pictures are all transmitted into the trained model (semantic segmentation model) for automatically detecting the ground feature change,
the detection output unit 304 outputs the same number of ground feature change determination results for each small-sized region.
The time-span feature change detection model based on the deep learning semantic segmentation model outputs the same number of building change judgments corresponding to each 256 × 256 region, where the output value is 1 when there is a change and 0 when there is no change.
And the result splicing unit 305 splices all the change detection results into a complete large-size picture according to the corresponding sequence to obtain the detection result of the change condition of the ground object in the area.
And finally, splicing all the change detection result graphs into a complete large-size graph according to the corresponding sequence to obtain the change condition of the buildings in the region within two years.
Fig. 8 is a schematic diagram illustrating a method for detecting a feature change. The contents of each step of the method are completely in one-to-one correspondence with the functions of each unit of the device described in this embodiment, and are not described herein again.
The invention directly stacks the remote sensing images of the buildings for two years to form a multi-channel composite image to be transmitted into the network, so that the network directly learns the output change instead of detecting the buildings of each year independently and then comparing and calculating the difference. The implementation mode can enable the output detection result to be purer, can effectively reduce false detection and noise generation, is more concise and clearer by using the framework, and greatly reduces the complexity of operation.
As shown in fig. 9a-c, the analysis using the method of detecting the feature change is shown by comparing the remote sensing image of 2015 and the remote sensing image of 2017 in a local area of the mountain of foshan city, guangdong. Fig. 9a is a schematic diagram of the image segmentation step, fig. 9b is a schematic diagram of the edge position mirroring process (the left side and the lower side are mirrored), fig. 9c is a detection effect diagram, wherein the left side is a 2015-year remote sensing diagram, the middle is a 2017-year remote sensing diagram, and the right diagram is a change detection result.
Therefore, the device or the method for detecting the change of the ground object can train and learn to obtain a model with the detection capability of the change of the building in the cross-time only by constructing a certain number of labeled data samples for each region, and can detect the building construction, dismantling, changing and expanding conditions of the building in one step by using the model for any position in the region with the similar ground or building style. The workers in the national and local authorities only need to pay attention to the warning area given by the model for manual recheck confirmation, so that the working intensity and difficulty are greatly reduced. Particularly, the invention provides a building change detection algorithm spanning a large time scale based on a deep learning semantic segmentation model, which can realize automatic detection of building changes with automation efficiency more than 100 times over a time span range of 20 years.
It is clear to a person skilled in the art that the solution according to the invention can be implemented by means of software and/or hardware. The "module" and "unit" in this specification refer to software and/or hardware that can perform a specific function independently or in cooperation with other components, where the hardware may be, for example, an FPGA (Field-Programmable Gate Array), an IC (Integrated Circuit), or the like.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the above-mentioned method for controlling the playing of a mind map. The computer-readable storage medium may include, but is not limited to, any type of disk including floppy disks, optical disks, DVDs, CD-ROMs, microdrive, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method for controlling the playing of a mind map when executing the program. In the embodiment of the present invention, the processor is a control center of a computer system, and may be a processor of an entity machine or a processor of a virtual machine.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (16)
1. A semantic segmentation model modeling apparatus for image contrast analysis, comprising:
a label unit is arranged, remote sensing image groups shot in the front and back time of the same area are selected from the satellite remote sensing images, manual identification and labeling are carried out, and labels are arranged in the corresponding areas;
the image cutting unit is used for cutting the image of the artificial labeling area constructed in the step of setting the label into a sufficient number of small-size images to form a small-size image data set for training a semantic segmentation model;
the input and output corresponding unit is used for stacking multi-channel images of the front time and the rear time in the small-size picture data set obtained in the image cutting step as input, and the corresponding change label is used as a true value label in a target;
a detection model unit is obtained, and FCN network parameters are continuously optimized until convergence by using the small-size picture data set in the input and output corresponding step through a deep learning framework, so that an automatic detection model is obtained;
the image cutting unit generates coordinate point pairs by using uniformly distributed random numbers and performs random cutting in the image of the manual labeling area.
2. The semantic segmentation model modeling apparatus for image contrast analysis according to claim 1, characterized in that:
the image cutting unit randomly cuts the images in the manual labeling area according to the placing modes of multiple angles such as 0 degree, 30 degrees, 60 degrees, 90 degrees and the like.
3. The semantic segmentation model modeling apparatus for image contrast analysis according to claim 1, further comprising:
and an amplification data set unit for further amplifying the scale of the training data set for the small-size pictures obtained by cutting.
4. The semantic segmentation model modeling apparatus for image contrast analysis according to claim 1, further comprising:
and the image preprocessing unit is used for carrying out color homogenizing treatment on the satellite remote sensing image group to be processed.
5. The semantic segmentation model modeling apparatus for image contrast analysis according to claim 1, characterized in that:
the input and output corresponding unit firstly standardizes the pictures to eliminate the interference caused by the color difference in each input picture.
6. A semantic segmentation model modeling method for image contrast analysis, comprising:
a label setting step, namely selecting a remote sensing image group shot in the same area at the front and back time from the satellite remote sensing images, carrying out manual identification and labeling, and setting labels in corresponding areas;
an image cutting step, namely cutting the image of the artificial labeling area constructed in the label setting step into a sufficient number of small-size images to form a small-size image data set for training a semantic segmentation model;
an input/output corresponding step of stacking multi-channel images of the front time and the rear time in the small-size picture data set obtained in the image cutting step as input, wherein the corresponding change label is used as a true value label in a target;
obtaining an automatic detection model, namely continuously optimizing FCN network parameters until convergence by using the small-size picture data set in the input and output corresponding step through a deep learning framework, so as to obtain the automatic detection model;
and in the image cutting step, coordinate point pairs are generated by using uniformly distributed random numbers, and random cutting is carried out in the image of the manual labeling area.
7. The method of modeling a semantic segmentation model for image contrast analysis according to claim 6, wherein:
and in the image cutting step, random cutting is carried out in the images of the artificial labeling area according to the placing modes of multiple angles such as 0 degree, 30 degrees, 60 degrees, 90 degrees and the like.
8. The method of modeling a semantic segmentation model for image contrast analysis according to claim 6, further comprising:
and a step of amplifying the data set, wherein the scale of the training data set is further amplified for the small-size pictures obtained by cutting.
9. The method of modeling a semantic segmentation model for image contrast analysis according to claim 6, further comprising:
and an image preprocessing step, namely performing color homogenizing treatment on the satellite remote sensing image group to be processed.
10. The method of modeling a semantic segmentation model for image contrast analysis according to claim 6, wherein:
the input and output corresponding step is to firstly standardize the pictures to eliminate the interference caused by the chromatic aberration in each input picture.
11. An apparatus for detecting a feature change, comprising:
the image cutting unit is used for cutting front and rear images of the to-be-detected region in the remote sensing image by small-size windows respectively;
the image stacking unit is used for stacking the multi-channel images of the front time and the rear time in a small-size picture data set obtained in the image cutting step;
using a model unit, the stacked images are passed into a model built by the method for modeling a semantic segmentation model for images as claimed in claim 6,
the detection output unit outputs the ground feature change judgment results of each small-size area with the same quantity through the model;
the result splicing unit is used for splicing all the change detection results into a complete large-size picture according to the corresponding sequence to obtain the detection result of the change condition of the ground object in the region;
and in the image cutting step, coordinate point pairs are generated by using uniformly distributed random numbers, and random cutting is carried out in the image of the manual labeling area.
12. The apparatus for detecting a change in a feature of claim 11, wherein: the image cutting unit adopts a redundant cutting mode.
13. A method of detecting a feature change, comprising:
an image cutting step, wherein front and rear images of a region to be detected in the remote sensing image are cut by small-size windows respectively;
an image stacking step of stacking multi-channel images of front and rear time in a small-size picture data set obtained in the image cutting step;
applying a model step, introducing the stacked images into a model established by a semantic segmentation model modeling method for images as claimed in claims 6-10,
a detection output step, wherein the model outputs the ground feature change judgment results of each small-size area with the same quantity;
and a result splicing step, namely splicing all the change detection results into a complete large-size picture according to the corresponding sequence to obtain the detection result of the change condition of the ground object in the region.
14. The method of claim 13, wherein the step of detecting the feature change comprises: the image cutting step adopts a redundant cutting mode.
15. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to any of the claims 6-10, 13, 14 are implemented by the processor when executing the program.
16. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the steps of the method according to any one of the claims 6-10, 13, 14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810411612.8A CN108776772B (en) | 2018-05-02 | 2018-05-02 | Cross-time building change detection modeling method, detection device, method and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810411612.8A CN108776772B (en) | 2018-05-02 | 2018-05-02 | Cross-time building change detection modeling method, detection device, method and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108776772A CN108776772A (en) | 2018-11-09 |
| CN108776772B true CN108776772B (en) | 2022-02-08 |
Family
ID=64026822
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810411612.8A Active CN108776772B (en) | 2018-05-02 | 2018-05-02 | Cross-time building change detection modeling method, detection device, method and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108776772B (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382645B (en) * | 2018-12-29 | 2023-11-28 | 顺丰科技有限公司 | Method and system for identifying overdue building in electronic map |
| CN109753928B (en) * | 2019-01-03 | 2022-03-29 | 北京百度网讯科技有限公司 | Method and device for identifying illegal buildings |
| CN110119744A (en) * | 2019-05-07 | 2019-08-13 | 上海交通大学 | The remote sensing image semantic segmentation method of multi-scale division constraint |
| CN110288602B (en) * | 2019-05-10 | 2021-07-09 | 香港理工大学深圳研究院 | Landslide extraction method, landslide extraction system and terminal |
| CN112241659B (en) * | 2019-07-17 | 2024-05-17 | 杭州海康威视数字技术股份有限公司 | Detection device and detection method for illegal building and terminal equipment |
| CN110956182B (en) * | 2019-09-23 | 2023-04-07 | 四创科技有限公司 | Method for detecting water area shoreline change based on deep learning |
| CN110827269B (en) * | 2019-11-11 | 2024-03-05 | 深圳前海微众银行股份有限公司 | Crop growth change condition detection method, device, equipment and medium |
| CN111860204A (en) * | 2020-06-29 | 2020-10-30 | 成都数之联科技有限公司 | Multi-temporal remote sensing image change detection method and medium based on semantic segmentation technology |
| KR102244678B1 (en) * | 2020-12-28 | 2021-04-26 | (주)컨텍 | Method and apparatus for providing education service using satellite imagery based on Artificial Intelligence |
| CN112819792A (en) * | 2021-02-03 | 2021-05-18 | 杭州高斯洪堡科技有限公司 | DualNet-based urban area change detection method |
| CN112801929A (en) * | 2021-04-09 | 2021-05-14 | 宝略科技(浙江)有限公司 | Local background semantic information enhancement method for building change detection |
| CN112967286B (en) * | 2021-05-19 | 2021-09-17 | 航天宏图信息技术股份有限公司 | Method and device for detecting newly added construction land |
| CN113469898A (en) * | 2021-06-02 | 2021-10-01 | 北京邮电大学 | Image distortion removal method based on deep learning and related equipment |
| CN114120148B (en) * | 2022-01-25 | 2022-04-26 | 武汉易米景科技有限公司 | Method for detecting changing area of remote sensing image building |
| CN114937204B (en) * | 2022-04-29 | 2023-07-25 | 南京信息工程大学 | Neural network remote sensing change detection method for lightweight multi-feature aggregation |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221239A (en) * | 2008-01-25 | 2008-07-16 | 电子科技大学 | Synthetic aperture radar image segmentation method based on level set |
| CN102708167A (en) * | 2012-04-27 | 2012-10-03 | 上海交通大学 | Web-based semantic annotation system and Web-based semantic annotation method for high resolution SAR (synthetic aperture radar) image interpretation |
| CN104680542A (en) * | 2015-03-15 | 2015-06-03 | 西安电子科技大学 | Online learning based detection method for change of remote-sensing image |
| CN105513079A (en) * | 2015-12-16 | 2016-04-20 | 中国科学院电子学研究所 | Detection method for large-scale time sequence remote sensing image change area |
| CN106778814A (en) * | 2016-11-24 | 2017-05-31 | 郑州航空工业管理学院 | A kind of method of the removal SAR image spot based on projection spectral clustering |
| CN106875395A (en) * | 2017-01-12 | 2017-06-20 | 西安电子科技大学 | Super-pixel level SAR image change detection based on deep neural network |
| CN107239797A (en) * | 2017-05-23 | 2017-10-10 | 西安电子科技大学 | Polarization SAR terrain classification method based on full convolutional neural networks |
| CN107730496A (en) * | 2017-10-26 | 2018-02-23 | 武汉大学 | A kind of multi-temporal remote sensing image building change detecting method based on image blocks |
-
2018
- 2018-05-02 CN CN201810411612.8A patent/CN108776772B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221239A (en) * | 2008-01-25 | 2008-07-16 | 电子科技大学 | Synthetic aperture radar image segmentation method based on level set |
| CN102708167A (en) * | 2012-04-27 | 2012-10-03 | 上海交通大学 | Web-based semantic annotation system and Web-based semantic annotation method for high resolution SAR (synthetic aperture radar) image interpretation |
| CN104680542A (en) * | 2015-03-15 | 2015-06-03 | 西安电子科技大学 | Online learning based detection method for change of remote-sensing image |
| CN105513079A (en) * | 2015-12-16 | 2016-04-20 | 中国科学院电子学研究所 | Detection method for large-scale time sequence remote sensing image change area |
| CN106778814A (en) * | 2016-11-24 | 2017-05-31 | 郑州航空工业管理学院 | A kind of method of the removal SAR image spot based on projection spectral clustering |
| CN106875395A (en) * | 2017-01-12 | 2017-06-20 | 西安电子科技大学 | Super-pixel level SAR image change detection based on deep neural network |
| CN107239797A (en) * | 2017-05-23 | 2017-10-10 | 西安电子科技大学 | Polarization SAR terrain classification method based on full convolutional neural networks |
| CN107730496A (en) * | 2017-10-26 | 2018-02-23 | 武汉大学 | A kind of multi-temporal remote sensing image building change detecting method based on image blocks |
Non-Patent Citations (1)
| Title |
|---|
| Recurrent Fully Convolutional Networks for Video Segmentation;Sepehr Valipour 等;《arXiv:1606.00487v3[cs.CV]》;20161031;正文第1-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108776772A (en) | 2018-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108776772B (en) | Cross-time building change detection modeling method, detection device, method and storage medium | |
| CN109800736B (en) | Road extraction method based on remote sensing image and deep learning | |
| CN109977812B (en) | Vehicle-mounted video target detection method based on deep learning | |
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| Prathap et al. | Deep learning approach for building detection in satellite multispectral imagery | |
| CN109086668B (en) | Unmanned aerial vehicle remote sensing image road information extraction method based on multi-scale generation countermeasure network | |
| CN111553397B (en) | Cross-domain target detection method based on regional full convolution network and self-adaption | |
| US11308714B1 (en) | Artificial intelligence system for identifying and assessing attributes of a property shown in aerial imagery | |
| US11854244B2 (en) | Labeling techniques for a modified panoptic labeling neural network | |
| CN106897681B (en) | Remote sensing image contrast analysis method and system | |
| EP3690741A2 (en) | Method for automatically evaluating labeling reliability of training images for use in deep learning network to analyze images, and reliability-evaluating device using the same | |
| CN112560675B (en) | Bird visual target detection method combining YOLO and rotation-fusion strategy | |
| Ren et al. | YOLOv5s-M: A deep learning network model for road pavement damage detection from urban street-view imagery | |
| CN110349167A (en) | A kind of image instance dividing method and device | |
| US11367278B2 (en) | Field segmentation and classification | |
| CN112052845A (en) | Image recognition method, device, equipment and storage medium | |
| CN113723377A (en) | Traffic sign detection method based on LD-SSD network | |
| CN112488229A (en) | Domain self-adaptive unsupervised target detection method based on feature separation and alignment | |
| CN112215190A (en) | Illegal building detection method based on YOLOV4 model | |
| CN112784834A (en) | Automatic license plate identification method in natural scene | |
| CN115909079A (en) | Crack detection method combining depth feature and self-attention model and related equipment | |
| CN114519819B (en) | Remote sensing image target detection method based on global context awareness | |
| CN115512222A (en) | Method for evaluating damage of ground objects in disaster scene of offline training and online learning | |
| CN116071294A (en) | Optical fiber surface defect detection method and device | |
| CN113971764B (en) | Remote sensing image small target detection method based on improvement YOLOv3 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |