CN115424017B - Building inner and outer contour segmentation method, device and storage medium - Google Patents
Building inner and outer contour segmentation method, device and storage medium Download PDFInfo
- Publication number
- CN115424017B CN115424017B CN202211010367.2A CN202211010367A CN115424017B CN 115424017 B CN115424017 B CN 115424017B CN 202211010367 A CN202211010367 A CN 202211010367A CN 115424017 B CN115424017 B CN 115424017B
- Authority
- CN
- China
- Prior art keywords
- building
- segmentation
- training
- image
- outline
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
Abstract
The invention discloses a method, a device and a storage medium for segmenting inner and outer contours of a building, wherein the method comprises the steps of collecting an image dataset of the upper air of the building, cutting images in the image dataset into fixed-size dimensions, and taking the fixed-size dimensions as a test dataset; inputting the test data set into a pre-constructed and trained building internal and external contour segmentation network, and segmenting the building internal and external contour by using the weight information with the best training effect; performing masking operation to obtain a final mask picture; performing binarization processing on the mask picture to obtain a region of the photovoltaic map; the invention can efficiently and accurately divide the outline of the building and calculate the actual area of the building for arranging the photovoltaic panel, and has small calculated amount and strong universality.
Description
Technical Field
The invention relates to a building inner and outer contour segmentation method, a device and a storage medium, and belongs to the technical field of photovoltaic artificial intelligence.
Background
A photovoltaic map, i.e. an area that can be used to arrange photovoltaic panels for photovoltaic power generation. At present, under the background of energy shortage, photovoltaic power generation is widely applied to industry and individual houses, the arrangement of photovoltaic panels directly influences the efficiency of photovoltaic power generation, the arrangement of the photovoltaic panels still has low calculation efficiency because of manual measurement or manual consulting of related building manuals to calculate the building area, and related records for the roof area of a building are also few in the building manuals, so that the area of a photovoltaic map in the building is difficult to calculate due to the above reasons, and a large area of the area which can be used for arranging the photovoltaic panels cannot be directly obtained, and the area at the top of the building is not fully utilized to a great extent, so that serious resource waste is caused;
in recent years, with the rapid development of deep learning, building contour segmentation based on deep learning is also gradually rising. At present, a building contour segmentation method based on deep learning is based on a satellite remote sensing image as an original image, features are extracted through a simple convolutional neural network, and then the contour of a building is segmented, but only the outer contour of the building is segmented, the contour inside the building is ignored, and the contour inside the building directly influences the area of a region where a photovoltaic panel can be arranged. So the current method for building contour segmentation based on deep learning is difficult to obtain accurate building interior areas which can be used for arranging photovoltaic panels, and only external contours can be obtained; meanwhile, as the source image is a picture obtained based on satellite remote sensing, the resolution is very low, and the building contour segmentation method based on deep learning is in principle pixel level segmentation, the method can cause false recognition due to lower resolution in actual use;
although the method based on the pure image can also divide the outer contour of a building, even some image algorithms based on the image can divide the inner contour more accurately, a large number of image algorithms, such as a method for graying, binarizing and thresholding the image, and some image dividing algorithms, are needed in the process, but the universality of the method is drastically reduced along with the increase of the number of the image algorithms, and once the scene of the image is changed greatly, the original algorithm is not applicable any more; in practical application, the method has extremely large calculation amount and calculation time due to the fact that a large amount of image processing algorithms are needed, hardware is needed to have high calculation force, and instantaneity is difficult to guarantee;
currently, both deep learning-based and pure image processing-based methods have respective drawbacks: the building contour segmentation method based on deep learning only stays in the segmentation of the external contour of the building, and adopts a satellite remote sensing image as a source image, so that the resolution is low, and the erroneous segmentation can be caused; the segmentation method based on the pure image can accurately segment the outline of the building, but has high calculation force requirement on hardware, low universality and difficult application to actual engineering.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provide a method, a device and a storage medium for dividing the inner and outer contours of a building, which can effectively and accurately divide the contour of the building and calculate the actual area of the building for arranging the photovoltaic panel, and has small calculated amount and strong universality.
In order to achieve the above purpose, the invention is realized by adopting the following technical scheme:
in a first aspect, the present invention provides a method for dividing an inner contour and an outer contour of a building, including:
acquiring a pre-acquired image dataset of the building, cutting images in the image dataset into fixed sizes, and eliminating pictures with poor acquisition effects to be used as a test dataset;
inputting the test data set into a pre-constructed and trained building outer contour segmentation network and a building inner contour segmentation network, and segmenting the building inner contour and the building outer contour by using weight information with the best training effect to obtain a segmentation result of the building inner contour and the building outer contour;
masking the segmentation result of the inner and outer contours of the building to obtain a final mask picture;
performing binarization processing on the mask picture to obtain a region of the photovoltaic map;
and calculating the pixel area of the photovoltaic map by using a connected domain image algorithm, and obtaining the actual photovoltaic map area through the internal and external parameters of the camera.
Further, the training method of the building outer contour segmentation network or the building inner contour segmentation network comprises the following steps:
acquiring a pre-acquired picture sample of the building overhead as a source image, preprocessing the image, cutting the image into a fixed size, marking the fixed size, constructing a data set for building outer contour/inner contour segmentation, and dividing the data set according to a proportion;
a deep convolution neural network for building outline/inner outline segmentation is built, wherein the deep convolution neural network comprises a feature extraction module, a feature connection module, a feature recovery module, a attention module and a multi-scale fusion module;
the divided building outer contour/inner contour segmentation training data set is input into a deep convolutional neural network, the training is carried out by using a gradient back propagation algorithm and a gradient descent algorithm, and the weight parameters with the best effect in the training are stored.
Further, the preprocessing the image, cutting the image into a size with a fixed size, and marking the image, including:
cutting all images in the image data set into preset fixed sizes, screening, and removing useless parts;
dividing the outer contour of the building from the cut and screened image, and then dividing the inner contour from the obtained outer contour;
marking the external outline of the building and the internal outline of the building which need to be segmented respectively, and distinguishing the external outline and the internal outline of the building by different labels.
Further, the data set partitioning includes: dividing the marked data set into a training set and a testing set, wherein the training set is used for being placed into a deep convolutional neural network to train so as to enable the deep convolutional neural network to learn weight parameters, and the testing set is used for testing how the weight parameters obtained by the deep convolutional neural network learning are effective.
Further, the inputting the divided building outer contour/inner contour segmentation training data set into the deep convolutional neural network and training by using a gradient back propagation algorithm and a gradient descent algorithm comprises the following steps:
respectively inputting divided building outer contour/inner contour segmentation training data sets into a deep convolutional neural network, wherein the training data sets comprise cut and screened pictures and marked pictures;
during training, the cut original image and the corresponding label image are read at the same time, and are input into a loss function and an accuracy evaluation function in the deep convolutional neural network for calculation, a loss value and an accuracy value are obtained through calculation, gradients of different layers in the deep convolutional neural network are calculated according to the obtained loss value, the calculated gradient parameters are input into an optimizer for optimization, and parameters of each layer are adjusted in the next training round number according to the optimization result until an optimal result is achieved.
Further, in the training process, if the divided test set exists in the data set, the verification set is read at a certain time interval in the training process, test prediction is performed according to the previous training result, the test prediction is compared with the label graph in the test set, the label graph is input into a loss function and an accuracy evaluation function in the deep convolutional neural network for calculation, a loss value and an accuracy value are obtained through calculation, the loss value and the accuracy value are input into an optimizer for parameter optimization, and the parameters are adjusted in the next round of training.
Further, the feature extraction module includes, but is not limited to, VGG series network, residual series network, denseNet series network, and MobileNet series network, and is composed of convolution operation, regularization operation, activation operation, pooling operation, where the convolution operation includes, but is not limited to, two-dimensional convolution, hole convolution, separable convolution, transpose convolution, and other operations; regularization operations are used for pattern overfitting; the activation functions in the activation operation include, but are not limited to Sigmoid, relu, tanh, leak Relu; the network structure is provided with a deep feature extraction module, and compared with the input feature vector, the feature vector output by the feature extraction module has the advantages that the space number is reduced by half, and the channel number is doubled;
the feature connection module is used for connecting or stacking pictures with different scales or pictures with the same scale, and for the pictures with the same scale, the pictures are simply stacked; for pictures with different scales, the pictures are required to be up-sampled to be of the same scale and then stacked;
the feature recovery module is realized by virtue of an up-sampling module and a deconvolution module and is used for increasing the dimension of the picture, and the picture is subjected to up-sampling operation after multiple convolutions, regularization and activation functions, wherein the up-sampling operation comprises but is not limited to nearest neighbor interpolation, linear interpolation and bilinear interpolation; a deep feature recovery module is provided in the network structure, and corresponds to the deep feature extraction module, and the scale of the image after deep feature extraction is recovered; compared with the input feature vector, the size of the feature vector output after feature recovery is doubled, and the number of channels is reduced to half of the original number;
the attention module includes but is not limited to: SE module, CBAM module, EMA module, OC module;
the multi-scale fusion module is used for extracting features of pictures with different scales, and then fusing the pictures to obtain multi-scale feature information, and the feature information among different scales is fused in the deep convolutional neural network by carrying out up-sampling operation on the feature information with a lower scale and stacking the feature information with a final larger scale after copying the feature information.
Further, the binarizing processing is performed on the mask picture to obtain a region of the photovoltaic map, including:
the outer contour of the building is segmented from the image, and rough segmentation is carried out;
using masking operation in an image algorithm to only reserve building outline areas in the source pictures;
and (3) dividing the inner outline of the building to obtain the outer outline edge and the inner ridge of the building, removing the outer outline edge and the inner ridge of the building by using masking operation, and obtaining the photovoltaic map area by dividing the rest.
In a second aspect, the present invention provides a building interior-exterior profile segmentation apparatus comprising:
the test data set acquisition unit is used for acquiring a pre-acquired image data set of the building, cutting images in the image data set into fixed sizes, and eliminating pictures with poor acquisition effects to be used as a test data set;
the segmentation unit is used for inputting the test data set into a pre-constructed and trained building outer contour segmentation network and a building inner contour segmentation network, and segmenting the building outer contour and the building inner contour by using weight information with the best training effect to obtain segmentation results of the building inner contour and the building outer contour;
the mask picture obtaining unit is used for masking the segmentation result of the inner outline and the outer outline of the building to obtain a final mask picture;
the photovoltaic map region acquisition unit is used for carrying out binarization processing on the mask picture to obtain a region of the photovoltaic map;
and the calculating unit is used for calculating the pixel area of the photovoltaic map by using the connected domain image algorithm, and obtaining the actual photovoltaic map area through the internal and external parameters of the camera.
In a third aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of any of the methods described in the preceding claims.
Compared with the prior art, the invention has the beneficial effects that:
(1) The technical scheme provides a building inner and outer contour extraction method based on deep learning semantic segmentation and a traditional visual image algorithm, and calculates the actual area of a building photovoltaic map according to the extraction result, so that the method can replace manual measurement of the actual layout photovoltaic panel area of the building, and greatly reduces the workload;
(2) According to the technical scheme, the neural network for dividing the inner and outer contours of the building is constructed, and the attention module and the multi-scale fusion module are introduced, so that the characteristic information in the image can be fully acquired, and the dividing effect of the image is effectively improved;
(3) According to the technical scheme, the deep learning semantic segmentation is used for building inner and outer contour segmentation, the traditional image vision algorithm is used for calculating the pixel area of the photovoltaic map, the inner and outer parameters of the camera are introduced to convert the pixel area into the actual area, and compared with a pure deep learning method and a pure vision image algorithm, the scheme has universality and robustness when coping with different scenes;
(4) The method for processing the fusion deep learning semantic segmentation in the traditional visual image provided by the technical scheme aims at the building, can effectively segment the inner contour and the outer contour, calculates the actual area of the photovoltaic map, and has the advantages of smaller calculated amount, lower hardware calculation occupancy rate and higher processing speed in the practical stage.
Drawings
FIG. 1 is a flow chart of a method for dividing inner and outer contours of a building according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a neural network model for dividing internal and external contours of a building according to an embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
Example 1
The embodiment introduces a building inner and outer contour segmentation method, which comprises the following steps:
acquiring a pre-acquired image dataset of the building, cutting images in the image dataset into fixed sizes, and eliminating pictures with poor acquisition effects to be used as a test dataset;
inputting the test data set into a pre-constructed and trained building outer contour segmentation network and a building inner contour segmentation network, and segmenting the building inner contour and the building outer contour by using weight information with the best training effect to obtain a segmentation result of the building inner contour and the building outer contour;
masking the segmentation result of the inner and outer contours of the building to obtain a final mask picture;
performing binarization processing on the mask picture to obtain a region of the photovoltaic map;
and calculating the pixel area of the photovoltaic map by using a connected domain image algorithm, and obtaining the actual photovoltaic map area through the internal and external parameters of the camera.
As shown in fig. 1, an example of the embodiment of the present embodiment is as follows:
the method comprises the steps of (1) image preprocessing, (2) network model training, (3) internal and external contour segmentation, (4) photovoltaic map calculation and (5) practical application; the following process of dividing the inner and outer contours of the building by using the neural network is two-stage, wherein the first stage is to divide the outer contour of the building, and the outer contour of the building is divided from the original image by the neural network; the second stage is to divide the inner contour of the building, wherein the inner contour of the building is divided by the neural network from the outer contour of the previous stage, the inner contour comprises an inner ridge and an outer contour edge, the inner contour and the outer contour edge are all areas which cannot be used for arranging the photovoltaic panel, and then the image processing is carried out on the division results of the two stages to obtain the area which can be used for arranging the photovoltaic panel finally.
(1) The data set preparation phase includes the steps of:
1.1 Data acquisition: the unmanned aerial vehicle is used for carrying the camera to collect the building pictures above the industrial park, and the unmanned aerial vehicle is arranged to fly at a fixed height, so that parameters such as the focal length of the camera are kept unchanged, and the internal and external parameters of the camera are kept unchanged;
1.2 Image cropping and screening: screening the pictures acquired in the step 1.1), removing the pictures with poor imaging effect, avoiding increasing the calculation amount of hardware, and then cutting the obtained pictures into fixed sizes (the size is 512 x 512 in the embodiment) so as to facilitate the subsequent input of the pictures into a neural network model for reading;
1.3 Image annotation: labeling the picture data set obtained in the step 1.2) by using Labelme software, in the labeling of the external contour of the building, labeling the external contour of the building as 1, taking the rest part as a background, labeling 0 for distinguishing, generating a labeling data set corresponding to the external contour of the building after the labeling is finished, generating a mask picture by using the labeled picture and an original picture, wherein the mask picture only comprises the external contour of the building and is used for labeling the internal contour data set of the building; in labeling the building interior contour dataset, the building exterior contour is labeled 1, the building interior ridge is labeled 2, and the remaining background portion is labeled 0 for distinction. Generating a corresponding labeling data set of the inner outline of the building after labeling;
1.4 Data set expansion and partitioning: dividing the marked data set obtained in 1.3) according to a certain proportion, and according to 9: the proportion of 1 divides the training set into two parts, namely a training set and a testing set; if the number of the data sets obtained in 1.3) is small, performing operations such as rotation, scaling, noise adding and the like on the original image and the corresponding label image to expand the data sets, and then dividing the data sets;
(2) The steps of the network model training phase include:
2.1 Building an inside and outside contour segmentation neural network model: as shown in fig. 2, in the building outer contour segmentation network and the building inner contour segmentation network, the following modules are included: the device comprises a feature extraction module, a feature connection module, a feature recovery module, an attention module and a multi-scale fusion module;
2.2 Training of neural network model):
2.2.1 Overview of training procedure: all pictures in a training set of the inner contour and the outer contour of a building are input into the neural network for training, the output value of the neural network and the label graph corresponding to the source image are input into a loss function and an accuracy evaluation function to calculate a loss value and an accuracy value, gradients of different layers in the neural network are calculated according to the obtained loss value, the calculated gradient parameters are input into an optimizer for optimization, and the parameters of each layer are adjusted in the next training round number according to the optimized result until the optimal result is reached, namely, the predicted value and the corresponding label value are approximated infinitely. In the learning process, two parameters, namely a batch number batch and a training round number epoch, are involved, the batch refers to the number of pictures read in from a data set each time, the epoch refers to the number of rounds of training, and each round refers to the fact that all data in the training set are read into a neural network and training is completed;
2.2.2 During training, the test set is read after each round of training is finished, test prediction is carried out according to the previous training result, the test set is compared with a label graph in the test set, the label graph is input into a loss function and an accuracy evaluation function in a neural network for calculation, a loss value of the test set and an accuracy value of the test set are obtained through calculation, the loss value and the accuracy value of the test set are input into an optimizer for parameter optimization, and parameters are adjusted in the next round of training;
2.2.3 The neural network loss function calculates the difference between the prediction graph of the original graph after training and the corresponding label value, and the neural network loss function can be divided into a loss function of a training set and a loss function of a test set according to different data sets; the neural network precision evaluation function calculates the same probability between the prediction graph of the original graph after training and the corresponding label value, and can be divided into a precision evaluation function of a training set and a precision evaluation function of a testing machine through the same model; the optimizer is a class of optimization functions including, but not limited to, BGD, SGD, MBGD, adam, adagrad, rmsprop, adaDelta and the like; the super parameters in the optimization process of the optimizer include, but are not limited to, learning rate, momentum, weight attenuation rate and the like, are usually adjusted according to the super parameters in the training process, the super parameters are selected through continuous experiments, the initial super parameters suitable for the model are determined through continuous repetition of 2.2.1), and the parameters are dynamically adjusted according to loss functions of different numbers of rounds and the like in the training process until an optimal set of super parameters is found;
2.2.4 The training set can be expanded and enhanced in the training process, and the training set can be expanded through operations such as rotation, scaling and the like of a data set enhancement function, so that the defect of low accuracy caused by fewer data sets is overcome, and the overall accuracy of the neural network is further improved;
(3) The steps of the internal and external contour segmentation stage comprise:
3.1 Building outline segmentation): predicting a building picture according to the weight parameters obtained after the neural network training in the step (2), inputting the weight parameters obtained after the building outline segmentation network training into a corresponding prediction network to obtain an outline segmentation result, wherein the segmentation result is that two colors are used for respectively representing the background and the outline of the building, a mask is manufactured according to the prediction picture, all pixels of the segmented building outline area are set as 1, the background is set as 0 to obtain a mask picture, and then the mask picture is multiplied with an original picture to obtain a mask picture, namely, only the picture of the outline of the building is reserved for the segmentation of the inner outline of the building;
3.2 Building inner contour segmentation): inputting the mask picture obtained by the outer contour of the building in 3.1) and the weight parameters obtained by the inner contour segmentation network training into the corresponding prediction network together to obtain the segmentation result of the inner contour, wherein the result comprises three types: the three types of results are endowed with different pixel values to generate corresponding mask pictures for calculation of the photovoltaic map in the next stage;
(4) The photovoltaic map calculation stage comprises the following steps:
4.1 Acquisition of a photovoltaic map: from the definition of the photovoltaic map we need to obtain the area in the building where the photovoltaic panels can be arranged, i.e. the area left after subtracting the building outline edges and the internal ridge from the building outline. Calculating through the mask pictures of the inner outline and the outer outline of the building in the step (3), firstly, assigning 0 to pixel values of the outer outline edge and the inner ridge area in the mask picture of the inner outline of the building, and assigning 1 to the rest, and then multiplying the mask pictures in the step (3.1) by the assigned pixels to form a new mask picture, wherein the area in the obtained picture is a photovoltaic map (the background is black, and the foreground is the photovoltaic map);
4.2 Photovoltaic map pixel area calculation: firstly binarizing the picture in 4.1), selecting a proper threshold value, setting all the backgrounds to be 0 by using a threshold function, marking all the areas of the photovoltaic map to be 1, then carrying out area calculation on the areas in the photovoltaic map obtained in 4.1) by using a connected domain algorithm in a traditional visual image algorithm, wherein the obtained areas are pixel areas, namely the number of pixel points contained in the areas, setting the number of pixels which are larger than a certain value according to actual conditions, outputting, eliminating the range of small areas, and drawing the minimum external rectangle of the areas meeting the conditions by using the image algorithm for displaying and outputting;
4.3 Photovoltaic map actual area calculation: 4.2 The area of the photovoltaic map obtained is only the pixel area in the picture, in practical application, the area needs to be converted into the practical area, the practical area represented by each pixel in the picture can be converted through the internal and external parameters of the camera, and the practical area of the photovoltaic map can be calculated by multiplying the area of each area calculated in 4.2), so that the photovoltaic map is used for reference in practical arrangement of the photovoltaic panel;
(5) The actual application stage:
5.1 Selecting a sensor, and calculating internal and external parameters for use in the final calculation of the actual area of the photovoltaic map;
5.2 The sensor is used for collecting data, which can be in various forms such as pictures, videos, point clouds and the like, and finally the data are converted into the picture form;
5.3 Data preparation and processing, cutting the acquired data set into specified sizes, and eliminating pictures with poor acquisition effects;
5.4 Building a predicted neural network for internal and external contour segmentation, wherein weight parameters after internal and external contour segmentation network training are built and obtained before the predicted neural network is built, and image data in 5.3) and pre-trained weight parameters are read into the network to obtain corresponding internal and external contour segmentation results;
5.5 Performing mask operation according to the internal and external contour segmentation result obtained in 5.4), obtaining a final mask picture, binarizing the mask picture to obtain a region of the photovoltaic map, calculating the pixel area of the photovoltaic map by using a connected domain algorithm, and combining the internal and external parameters of the camera in 5.1) to obtain the actual area of the photovoltaic map, wherein the following information is finally obtained: the number of photovoltaic panel areas that can be arranged within a single picture, the individual area of the photovoltaic panel areas that can be arranged within a single picture, and the total area of the photovoltaic map (photovoltaic panel areas that can be arranged) in the entire data set.
The photovoltaic map comprises: the method is characterized in that a region which can be used for photovoltaic power generation in a building is mainly concentrated on the roof of the building, the required region is flat, when a photovoltaic map is calculated, the outline (roof) of the building is firstly required to be segmented from an image, rough segmentation is carried out, masking operation in an image algorithm is used for only keeping the outline region of the building in a source picture, then segmentation of the outline of the building is further carried out, the edge of the outline of the building and the internal ridge of the building are obtained, the two regions are regions where a photovoltaic plate cannot be arranged, the two regions are removed by the masking operation, the rest is the segmented photovoltaic map region, the pixel area of the photovoltaic map is calculated by a connected domain algorithm in the image processing, and the actual photovoltaic map area is obtained by introducing internal and external parameters of a sensor.
The feature extraction module is as follows: the feature extraction module is similar to a common image feature extraction network, including but not limited to a VGG series network, a residual series network, a DenseNet series network, and a MobileNet series network; this is a consistent portion of an image-based neural network, typically consisting of convolution operations, regularization operations, activation operations, pooling operations, including but not limited to two-dimensional convolution, hole convolution, separable convolution, transpose convolution, etc.; regularization is mainly aimed at mode overfitting; the activation functions in the activation operation include, but are not limited to Sigmoid, relu, tanh, leak Relu, etc.; the network structure is provided with a deep feature extraction module which comprises feature extraction operation with lower scale, and the number of layers of feature extraction can be increased and reduced according to actual requirements in actual application; compared with the input feature vector, the feature vector output by the feature extraction module has the advantages that the space number is reduced by half, and the channel number is doubled;
the characteristic connection module is as follows: connecting or stacking pictures with different scales or pictures with the same scale, and simply stacking the pictures with the same scale; for pictures with different scales, the pictures are required to be up-sampled to be of the same scale and then stacked;
the above feature recovery module: the method is mainly realized by an up-sampling module and a deconvolution module, and mainly aims to increase the dimension of the picture, and the picture is subjected to up-sampling operation after multiple convolutions, regularization and activation functions, wherein the up-sampling operation comprises but is not limited to nearest neighbor interpolation, linear interpolation, bilinear interpolation and the like; a deep feature recovery module is provided in the network structure, and corresponds to the deep feature extraction module, and the scale of the image after deep feature extraction is recovered; compared with the input feature vector, the size of the feature vector output after feature recovery is doubled, and the number of channels is reduced to half of the original number;
the attention module is added between different layers, can help the network locate the interested information, inhibit useless information and effectively improve the segmentation effect of the image, and comprises but is not limited to: SE module, CBAM module, EMA module, OC module, etc.;
the multi-scale fusion module comprises: the images with different scales are subjected to feature extraction, and then the images are fused to obtain multi-scale feature information, so that compared with the feature extraction with a single scale, the obtained feature information is more abundant. And in the neural network, feature information of different scales is fused by carrying out upsampling operation on the feature information of a lower scale and stacking the feature information with the final feature information of a larger scale after copying.
The above-mentioned loss function: to represent the gap between the predicted value and the tag value. When the neural network is trained, the loss function is continuously reduced by continuously changing all parameters in the neural network, so that a neural network model with higher accuracy is trained. In practice, the loss functions include, but are not limited to Cross entropy loss, focal loss, dice Soft loss, etc
The above precision evaluation function: used to represent the similarity of the predicted value to the tag value. When the neural network is trained, the accuracy evaluation function value is continuously improved by continuously changing all parameters in the neural network, so that a neural network model with higher accuracy is trained. In practical applications, the precision evaluation function includes, but is not limited to, MIoU, fwIoU, etc.
Example 2
The present embodiment provides a building inside and outside contour segmentation apparatus, including:
the test data set acquisition unit is used for acquiring a pre-acquired image data set of the building, cutting images in the image data set into fixed sizes, and eliminating pictures with poor acquisition effects to be used as a test data set;
the segmentation unit is used for inputting the test data set into a pre-constructed and trained building outer contour segmentation network and a building inner contour segmentation network, and segmenting the building outer contour and the building inner contour by using weight information with the best training effect to obtain segmentation results of the building inner contour and the building outer contour;
the mask picture obtaining unit is used for masking the segmentation result of the inner outline and the outer outline of the building to obtain a final mask picture;
the photovoltaic map region acquisition unit is used for carrying out binarization processing on the mask picture to obtain a region of the photovoltaic map;
and the calculating unit is used for calculating the pixel area of the photovoltaic map by using the connected domain image algorithm, and obtaining the actual photovoltaic map area through the internal and external parameters of the camera.
Example 3
The present embodiment provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of any of embodiment 1.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and variations could be made by those skilled in the art without departing from the technical principles of the present invention, and such modifications and variations should also be regarded as being within the scope of the invention.
Claims (7)
1. A method for dividing an inner contour and an outer contour of a building, comprising:
acquiring a pre-acquired image dataset of the building, cutting images in the image dataset into fixed sizes, and eliminating pictures with poor acquisition effects to be used as a test dataset;
inputting the test data set into a pre-constructed and trained building outer contour segmentation network and a building inner contour segmentation network, and segmenting the building inner contour and the building outer contour by using weight information with the best training effect to obtain a segmentation result of the building inner contour and the building outer contour;
masking the segmentation result of the inner and outer contours of the building to obtain a final mask picture;
performing binarization processing on the mask picture to obtain a region of the photovoltaic map;
calculating the pixel area of the photovoltaic map by using a connected domain image algorithm, and obtaining the actual photovoltaic map area through the internal and external parameters of the camera;
the training method of the building outline segmentation network or the building inner outline segmentation network comprises the following steps:
acquiring a pre-acquired picture sample of the building overhead as a source image, preprocessing the image, cutting the image into a fixed size, marking the fixed size, constructing a data set for building outer contour/inner contour segmentation, and dividing the data set according to a proportion;
a deep convolution neural network for building outline/inner outline segmentation is built, wherein the deep convolution neural network comprises a feature extraction module, a feature connection module, a feature recovery module, a attention module and a multi-scale fusion module;
inputting the divided building outer contour/inner contour segmentation training data set into a deep convolutional neural network, training by using a gradient back propagation algorithm and a gradient descent algorithm, and storing the weight parameters with the best effect in training;
the method for inputting the divided building outer contour/inner contour segmentation training data set into the deep convolutional neural network and training by using a gradient back propagation algorithm and a gradient descent algorithm comprises the following steps:
respectively inputting divided building outer contour/inner contour segmentation training data sets into a deep convolutional neural network, wherein the training data sets comprise cut and screened pictures and marked pictures;
during training, the original pictures and the corresponding label pictures after cutting are read at the same time, and are input into a loss function and an accuracy evaluation function in the deep convolutional neural network for calculation to obtain a loss value and an accuracy value, gradients of different layers in the deep convolutional neural network are calculated according to the obtained loss value, gradient parameters obtained through calculation are input into an optimizer for optimization, and parameters of each layer are adjusted in the next training round number according to the optimization result until an optimal result is achieved;
the binarizing processing is carried out on the mask picture to obtain a region of the photovoltaic map, which comprises the following steps:
the outer contour of the building is segmented from the image, and rough segmentation is carried out;
using masking operation in an image algorithm to only reserve building outline areas in the source pictures;
and (3) dividing the inner outline of the building to obtain the outer outline edge and the inner ridge of the building, removing the outer outline edge and the inner ridge of the building by using masking operation, and obtaining the photovoltaic map area by dividing the rest.
2. The method for segmenting the inner and outer contours of the building according to claim 1, wherein the preprocessing the image, cutting the image into a fixed size and marking the image, comprises the following steps:
cutting all images in the image data set into preset fixed sizes, screening, and removing useless parts;
dividing the outer contour of the building from the cut and screened image, and then dividing the inner contour from the obtained outer contour;
marking the external outline of the building and the internal outline of the building which need to be segmented respectively, and distinguishing the external outline and the internal outline of the building by different labels.
3. The building interior-exterior contour segmentation method according to claim 1, wherein the data set segmentation comprises: dividing the marked data set into a training set and a testing set, wherein the training set is used for being placed into a deep convolutional neural network to train so as to enable the deep convolutional neural network to learn weight parameters, and the testing set is used for testing how the weight parameters obtained by the deep convolutional neural network learning are effective.
4. The method for building contour segmentation according to claim 1, wherein in the training process, if there is a already divided test set in the data set, the verification set is read at a certain time interval in the training process, test prediction is performed according to the previous training result, and comparison is performed with a label graph in the test set, and the label graph is input into a loss function and an accuracy evaluation function in the deep convolutional neural network for calculation, so that a loss value and an accuracy value are obtained, and are input into an optimizer for parameter optimization, and the parameters are adjusted in the next round of training.
5. The building interior-exterior contour segmentation method according to claim 1, wherein:
the feature extraction module comprises a VGG series network, a residual series network, a DenseNet series network and a MobileNet series network, and consists of convolution operation, regularization operation, activation operation and pooling operation, wherein the convolution operation comprises but is not limited to two-dimensional convolution, hole convolution, separable convolution and transpose convolution operation; regularization operations are used for pattern overfitting; the activation functions in the activation operation include, but are not limited to Sigmoid, relu, tanh, leak Relu; the network structure is provided with a deep feature extraction module, and compared with the input feature vector, the feature vector output by the feature extraction module has the advantages that the space number is reduced by half, and the channel number is doubled;
the feature connection module is used for connecting or stacking pictures with different scales or pictures with the same scale, and for the pictures with the same scale, the pictures are simply stacked; for pictures with different scales, the pictures are required to be up-sampled to be of the same scale and then stacked;
the feature recovery module is realized by virtue of an up-sampling module and a deconvolution module and is used for increasing the dimension of the picture, and the picture is subjected to up-sampling operation after multiple convolutions, regularization and activation functions, wherein the up-sampling operation comprises but is not limited to nearest neighbor interpolation, linear interpolation and bilinear interpolation; a deep feature recovery module is provided in the network structure, and corresponds to the deep feature extraction module, and the scale of the image after deep feature extraction is recovered; compared with the input feature vector, the size of the feature vector output after feature recovery is doubled, and the number of channels is reduced to half of the original number;
the attention module includes but is not limited to: SE module, CBAM module, EMA module, OC module;
the multi-scale fusion module is used for extracting features of pictures with different scales, and then fusing the pictures to obtain multi-scale feature information, and the feature information among different scales is fused in the deep convolutional neural network by carrying out up-sampling operation on the feature information with a lower scale and stacking the feature information with a final larger scale after copying the feature information.
6. A building interior-exterior contour segmentation apparatus, comprising:
the test data set acquisition unit is used for acquiring a pre-acquired image data set of the building, cutting images in the image data set into fixed sizes, and eliminating pictures with poor acquisition effects to be used as a test data set;
the segmentation unit is used for inputting the test data set into a pre-constructed and trained building outer contour segmentation network and a building inner contour segmentation network, and segmenting the building outer contour and the building inner contour by using weight information with the best training effect to obtain segmentation results of the building inner contour and the building outer contour;
the mask picture obtaining unit is used for masking the segmentation result of the inner outline and the outer outline of the building to obtain a final mask picture;
the photovoltaic map region acquisition unit is used for carrying out binarization processing on the mask picture to obtain a region of the photovoltaic map;
the calculation unit is used for calculating the pixel area of the photovoltaic map by using a connected domain image algorithm, and obtaining the actual photovoltaic map area through the internal and external parameters of the camera;
in the segmentation unit, the training method of the building outline segmentation network or the building outline segmentation network comprises the following steps:
acquiring a pre-acquired picture sample of the building overhead as a source image, preprocessing the image, cutting the image into a fixed size, marking the fixed size, constructing a data set for building outer contour/inner contour segmentation, and dividing the data set according to a proportion;
a deep convolution neural network for building outline/inner outline segmentation is built, wherein the deep convolution neural network comprises a feature extraction module, a feature connection module, a feature recovery module, a attention module and a multi-scale fusion module;
inputting the divided building outer contour/inner contour segmentation training data set into a deep convolutional neural network, training by using a gradient back propagation algorithm and a gradient descent algorithm, and storing the weight parameters with the best effect in training;
the method for inputting the divided building outer contour/inner contour segmentation training data set into the deep convolutional neural network and training by using a gradient back propagation algorithm and a gradient descent algorithm comprises the following steps:
respectively inputting divided building outer contour/inner contour segmentation training data sets into a deep convolutional neural network, wherein the training data sets comprise cut and screened pictures and marked pictures;
during training, the original pictures and the corresponding label pictures after cutting are read at the same time, and are input into a loss function and an accuracy evaluation function in the deep convolutional neural network for calculation to obtain a loss value and an accuracy value, gradients of different layers in the deep convolutional neural network are calculated according to the obtained loss value, gradient parameters obtained through calculation are input into an optimizer for optimization, and parameters of each layer are adjusted in the next training round number according to the optimization result until an optimal result is achieved;
in the photovoltaic map region obtaining unit, performing binarization processing on the mask picture to obtain a region of a photovoltaic map, including:
the outer contour of the building is segmented from the image, and rough segmentation is carried out;
using masking operation in an image algorithm to only reserve building outline areas in the source pictures;
and (3) dividing the inner outline of the building to obtain the outer outline edge and the inner ridge of the building, removing the outer outline edge and the inner ridge of the building by using masking operation, and obtaining the photovoltaic map area by dividing the rest.
7. A computer-readable storage medium having stored thereon a computer program, characterized by: the program, when executed by a processor, implements the steps of the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211010367.2A CN115424017B (en) | 2022-08-23 | 2022-08-23 | Building inner and outer contour segmentation method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211010367.2A CN115424017B (en) | 2022-08-23 | 2022-08-23 | Building inner and outer contour segmentation method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115424017A CN115424017A (en) | 2022-12-02 |
| CN115424017B true CN115424017B (en) | 2023-10-17 |
Family
ID=84197667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211010367.2A Active CN115424017B (en) | 2022-08-23 | 2022-08-23 | Building inner and outer contour segmentation method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115424017B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115661144B (en) * | 2022-12-15 | 2023-06-13 | 湖南工商大学 | Adaptive medical image segmentation method based on deformable U-Net |
| CN116132824B (en) * | 2023-04-13 | 2023-06-16 | 四川新视创伟超高清科技有限公司 | Image partition conversion method, system, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104412165A (en) * | 2012-05-02 | 2015-03-11 | 罗利诗公司 | Cylindrical polymer mask and method of fabrication |
| CN109614871A (en) * | 2018-11-13 | 2019-04-12 | 远景能源（南京）软件技术有限公司 | Photovoltaic roof and photovoltaic barrier automatic identification algorithm |
| CN111191500A (en) * | 2019-11-12 | 2020-05-22 | 广东融合通信股份有限公司 | Photovoltaic roof resource identification method based on deep learning image segmentation |
| US10754999B1 (en) * | 2019-04-05 | 2020-08-25 | Sunpower Corporation | Generating a photovoltaic system design for a building |
| CN113221355A (en) * | 2021-05-10 | 2021-08-06 | 中国计量大学上虞高等研究院有限公司 | Arrangement method of irregular roof photovoltaic panels of complex building |
| CN113963055A (en) * | 2021-11-19 | 2022-01-21 | 国网福建省电力有限公司经济技术研究院 | Roof photovoltaic theoretical installed capacity calculation method based on multispectral map data |
| CN114332020A (en) * | 2021-12-29 | 2022-04-12 | 中国科学技术大学 | Photovoltaic panel positioning and defect detection method and system based on visible light image |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2558776B1 (en) * | 2010-04-16 | 2022-09-14 | Azumo, Inc. | Front illumination device comprising a film-based lightguide |
| US10127670B2 (en) * | 2016-09-27 | 2018-11-13 | Xactware Solutions, Inc. | Computer vision systems and methods for detecting and modeling features of structures in images |
| US11854208B2 (en) * | 2020-01-15 | 2023-12-26 | The Regents Of The University Of California | Systems and methods for trainable deep active contours for image segmentation |
| US20220215622A1 (en) * | 2020-02-28 | 2022-07-07 | Aurora Solar Inc. | Automated three-dimensional building model estimation |
-
2022
- 2022-08-23 CN CN202211010367.2A patent/CN115424017B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104412165A (en) * | 2012-05-02 | 2015-03-11 | 罗利诗公司 | Cylindrical polymer mask and method of fabrication |
| CN109614871A (en) * | 2018-11-13 | 2019-04-12 | 远景能源（南京）软件技术有限公司 | Photovoltaic roof and photovoltaic barrier automatic identification algorithm |
| US10754999B1 (en) * | 2019-04-05 | 2020-08-25 | Sunpower Corporation | Generating a photovoltaic system design for a building |
| CN111191500A (en) * | 2019-11-12 | 2020-05-22 | 广东融合通信股份有限公司 | Photovoltaic roof resource identification method based on deep learning image segmentation |
| CN113221355A (en) * | 2021-05-10 | 2021-08-06 | 中国计量大学上虞高等研究院有限公司 | Arrangement method of irregular roof photovoltaic panels of complex building |
| CN113963055A (en) * | 2021-11-19 | 2022-01-21 | 国网福建省电力有限公司经济技术研究院 | Roof photovoltaic theoretical installed capacity calculation method based on multispectral map data |
| CN114332020A (en) * | 2021-12-29 | 2022-04-12 | 中国科学技术大学 | Photovoltaic panel positioning and defect detection method and system based on visible light image |
Non-Patent Citations (2)
| Title |
|---|
| 基于自适应全局阈值融合标记的遥感图像建筑群分割;李丽;柴文婷;梅树立;;农业机械学报(第07期);全文 * |
| 建筑物表面光伏组件最佳铺设方案的研究;杨若涵;文艺;范镭耀;周羚君;;西安航空学院学报(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115424017A (en) | 2022-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110322495B (en) | Scene text segmentation method based on weak supervised deep learning | |
| CN110287849B (en) | Lightweight depth network image target detection method suitable for raspberry pi | |
| CN110728200B (en) | Real-time pedestrian detection method and system based on deep learning | |
| CN115424017B (en) | Building inner and outer contour segmentation method, device and storage medium | |
| CN111145174B (en) | 3D target detection method for point cloud screening based on image semantic features | |
| Biasutti et al. | Lu-net: An efficient network for 3d lidar point cloud semantic segmentation based on end-to-end-learned 3d features and u-net | |
| CN111414923B (en) | Indoor scene three-dimensional reconstruction method and system based on single RGB image | |
| CN112287941B (en) | License plate recognition method based on automatic character region perception | |
| CN112183203A (en) | Real-time traffic sign detection method based on multi-scale pixel feature fusion | |
| CN110348435B (en) | Target detection method and system based on regional candidate network | |
| CN111753682B (en) | Hoisting area dynamic monitoring method based on target detection algorithm | |
| CN110188802B (en) | SSD target detection algorithm based on multi-layer feature map fusion | |
| CN111462140B (en) | Real-time image instance segmentation method based on block stitching | |
| CN115131797B (en) | Scene text detection method based on feature enhancement pyramid network | |
| CN113609896A (en) | Object-level remote sensing change detection method and system based on dual-correlation attention | |
| CN110852199A (en) | Foreground extraction method based on double-frame coding and decoding model | |
| CN116129291A (en) | Unmanned aerial vehicle animal husbandry-oriented image target recognition method and device | |
| CN115100652A (en) | Electronic map automatic generation method based on high-resolution remote sensing image | |
| CN111640116A (en) | Aerial photography graph building segmentation method and device based on deep convolutional residual error network | |
| CN112668662B (en) | Outdoor mountain forest environment target detection method based on improved YOLOv3 network | |
| CN116797787B (en) | Remote sensing image semantic segmentation method based on cross-modal fusion and graph neural network | |
| CN116596966A (en) | Segmentation and tracking method based on attention and feature fusion | |
| CN113192018B (en) | Water-cooled wall surface defect video identification method based on fast segmentation convolutional neural network | |
| CN113657225B (en) | Target detection method | |
| CN114708591A (en) | Document image Chinese character detection method based on single character connection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |