CN114494283A - Automatic farmland dividing method and system - Google Patents
Automatic farmland dividing method and system Download PDFInfo
- Publication number
- CN114494283A CN114494283A CN202111602281.4A CN202111602281A CN114494283A CN 114494283 A CN114494283 A CN 114494283A CN 202111602281 A CN202111602281 A CN 202111602281A CN 114494283 A CN114494283 A CN 114494283A
- Authority
- CN
- China
- Prior art keywords
- farmland
- segmentation
- image
- network
- remote sensing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000011218 segmentation Effects 0.000 claims abstract description 167
- 238000005520 cutting process Methods 0.000 claims abstract description 19
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 17
- 230000006870 function Effects 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 11
- 238000011176 pooling Methods 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 7
- 239000011800 void material Substances 0.000 claims description 7
- 238000003860 storage Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000003709 image segmentation Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000003892 spreading Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 15
- 238000004590 computer program Methods 0.000 description 7
- 239000000284 extract Substances 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000012800 visualization Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a farmland automatic segmentation method and a farmland automatic segmentation system, wherein the farmland automatic segmentation method comprises the following steps: acquiring an original remote sensing image, cutting the original remote sensing image into image blocks, and sequentially inputting the image blocks into an encoder module of a farmland rough segmentation network to obtain farmland characteristics under different scales; decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a rough segmentation result; positioning pixels belonging to a farmland in the original remote sensing image by using a rough segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and eliminating pixels wrongly divided into the farmland in the rough segmentation result to obtain a fine segmentation result; and splicing the input fine segmentation results of all the image blocks in a mean value superposition mode to finish the segmentation of the original large image. The advantages are that: the method can solve the problem of inconsistent farmland dimensions and avoid the mistaken segmentation of road information such as ridges and the like into farmlands.
Description
Technical Field
The invention relates to a farmland automatic segmentation method and a farmland automatic segmentation system, and belongs to the technical field of image information processing.
Background
With the rapid development of the aerospace technology, the spatial resolution of the remote sensing satellite images becomes higher and higher. For example, the spatial resolution of the sentinel second image reaches 10 meters. Thanks to such abundant spatial information, the remote sensing satellite technology can play a very important role in the application fields of land survey, environmental monitoring, crop estimation, construction planning and the like. If the invention focuses on the application of the remote sensing image in farmland area statistics, farmland information in the image needs to be accurately identified. In order to identify the information required by the application in the image, various identification methods including semantic segmentation are provided, and important significance is brought to the realization of agricultural refinement and the improvement of agricultural productivity.
Semantic segmentation is a typical task in the field of computer vision, and aims to allocate a corresponding category label to each pixel point in an input image. Early semantic segmentation methods mainly rely on manual design features, lack robustness and discriminability, and are difficult to obtain ideal segmentation results. With the breakthrough progress of the convolutional neural network on the computer vision task, the semantic segmentation model based on the full convolutional network has achieved great success. The method adaptively learns the image characteristics related to the task through the neural network, converts the final full-connection layer of the network into a convolution layer, and then performs up-sampling by using deconvolution to obtain a segmentation result consistent with the size of an input image. However, the step size convolution and pooling operations in the full convolution network easily cause the problem of loss of spatial semantic information, resulting in fuzzy object boundaries in the segmentation result. In addition, the local receptive field of the convolution operation (experimentally found to be much smaller than the theoretical receptive field of the convolutional neural network, especially at the deep level of the network) also limits the contextual information extraction capability of the model.
To capture richer contextual information, there have been efforts to perform multi-scale fusion of network features. For example, multi-scale features are fused using an image pyramid structure. Compared with the original model based on the full convolution network, the methods can obtain richer context information, enhance the identification capability of the object boundary information and improve the segmentation performance of the model. However, the problems of large variation of the same kind of objects, small difference of different types of objects, and the like in the remote sensing image still cannot be solved well. For example, in some seasons, the farmland in the remote sensing image and the forest in the mountain range are green. And the farmlands in different types and different areas show different sizes and appearances. When the method identifies the information, the characteristics cannot be well distinguished, and the segmentation is mistaken. In addition, how to identify the tiny roads in the image is also a difficult point of semantic segmentation. Compared with a natural image, the problem that the road characteristics such as ridges and the like in the high-resolution remote sensing image are not obvious is more serious.
Disclosure of Invention
The technical problem to be solved by the invention is to overcome the defects of the prior art and provide a farmland automatic segmentation method and system.
In order to solve the technical problem, the invention provides a farmland automatic segmentation method, which comprises the following steps:
acquiring an original remote sensing image, cutting the original remote sensing image into image blocks, and sequentially inputting the image blocks into an encoder module of a farmland rough segmentation network to obtain farmland characteristics under different scales;
decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a rough segmentation result;
positioning pixels belonging to a farmland in the original remote sensing image by using a rough segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and eliminating pixels wrongly divided into the farmland in the rough segmentation result to obtain a fine segmentation result;
and splicing the input fine segmentation results of all the image blocks in a mean value superposition mode to finish the segmentation of the original large image.
Further, tailor original remote sensing image into the image piece, input into the encoder module of farmland rough segmentation network in proper order, obtain the farmland characteristic under the different yards, include:
cutting the original remote sensing image X to obtain a cut image block XiI belongs to {1, 2, …, n }, wherein n represents the number of image blocks obtained by clipping;
image block xiAs the input of the encoder, extracting the features through an Xceptance network with cavity convolution to obtain shallow features f in the networkNAnd output characteristic fXThen, f is mapped to a 3 × 3 hole convolution layer having an expansion rate r of 6, a 3 × 3 hole convolution layer having an expansion rate r of 12, a 3 × 3 hole convolution layer having an expansion rate r of 18, and an image pooling operation, respectivelyXExtracting multi-scale features, capturing farmland features f under different scalesMExpressed as:
wherein, F, P and C respectively represent convolution, image pooling and feature combining operation, and the upper and lower labels of the convolution kernel omega respectively represent the expansion ratio r and the size of the convolution kernel.
Further, the decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a coarse segmentation result includes:
the obtained farmland characteristics f under different scalesMShallow feature f in Xconcept with backbone networkNPerforming fusion to obtain a fusion characteristic fMNExpressed as:
fMN＝C(F(ω1×1，U(fM))，F(ω1×1，fN))(3)
wherein, U represents 4 times up-sampling by using bilinear difference;
fusing features f using convolution layers of size 1 × 1MNThe number of channels of (a) is mapped into a number of categories;
mapping the channels into category numbers, performing up-sampling to the size same as that of the input image block by utilizing bilinear interpolation to obtain a rough segmentation result y of the farmland rough segmentation networkFiExpressed as:
yFi＝U(F(ω1×1，fMN))(4)。
further, the method for locating the pixels belonging to the farmland in the remote sensing image by using the rough segmentation result comprises the following steps of carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and rejecting the pixels wrongly divided into the farmland in the rough segmentation result to obtain a fine segmentation result, wherein the method comprises the following steps:
using the result y of the rough segmentationFiPositioning xiJudging each pixel point predicted as the farmland through a farmland subdivision network to obtain each judgment result dRBased on each judgment result dRObtaining a judgment matrix DRThen according to the judgment matrix DRAnd the coarse segmentation result yFiObtaining a segmentation result yiExpressed as:
yi＝DR·yFi (7)
obtaining each judgment result dRThe method comprises the following steps:
cutting out a fixed area by taking the pixel point as a center
As the input of the fine division network, the road information is extracted by the 3-layer convolution layer to obtain the road characteristic f of the areaR：
Will f isRSpreading into one-dimensional vector, inputting into a full-connection layer with hidden node number of 2, and adding fRMapping into a two-dimensional vector, obtaining the probability that the central pixel point belongs to the road and the farmland by using a Sigmoid activation function on the two-dimensional vector, and obtaining a judgment result d through the following formulaR：
dR＝A(σ(L(f(fR)，2)))(6)
Wherein, A, sigma, L and f respectively represent argmax function, Sigmoid function, full connection layer and flatten operation.
Further, the step of splicing the input fine segmentation results of all the image blocks in a mean value superposition manner to complete the segmentation of the original large image includes:
subdivision of image blocksiSplicing is carried out by adopting a single-row splicing mode, a single-column splicing mode and a multi-row and multi-column splicing mode; wherein, the image blocks at the edge position in the original image are spliced in a single line in a left-right overlapping area when the segmentation result is segmented; an upper overlapping area and a lower overlapping area exist during the splicing of the segmentation results, and single-row splicing is carried out; the image blocks in the original image at non-edge positions have overlapped areas at the upper, lower, left and right sides when the segmentation result is spliced, and multi-row and multi-column splicing is carried out; and replacing the predicted values of the overlapped areas by using the mean values to obtain a remote sensing image segmentation result Y.
An automated farmland segmentation system comprising:
the cutting module is used for obtaining an original remote sensing image, cutting the original remote sensing image into image blocks, and sequentially inputting the image blocks into the encoder module of the farmland rough segmentation network to obtain farmland characteristics under different scales;
the rough segmentation module is used for decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a rough segmentation result;
the fine segmentation module is used for positioning pixels belonging to a farmland in the original remote sensing image by using the coarse segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and eliminating pixels which are mistakenly divided into the farmland in the coarse segmentation result to obtain a fine segmentation result;
and the splicing module is used for splicing the input fine segmentation results of all the image blocks in a mean value superposition mode to finish the segmentation of the original large image.
A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform any of the methods.
A computing device, comprising, in combination,
one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods.
The invention achieves the following beneficial effects:
the invention extracts farmland and road information by using a farmland rough segmentation network and a farmland fine segmentation network respectively. Firstly, an encoder of a farmland rough segmentation network extracts image features by using a convolutional neural network, and extracts multi-scale information by using cavity convolutions of different scales on the features, so that the problem of different farmland scales is solved; secondly, in order to keep the detail information such as the edge of the farmland, the decoder of the farmland rough segmentation network simultaneously uses the output characteristics of the encoder and the shallow layer characteristics in the convolutional neural network for decoding to obtain the segmentation result; and finally, the farmland fine segmentation network carries out secondary judgment on the result of the farmland rough segmentation network, so that the phenomenon that road information such as ridges is segmented into farmlands by mistake is avoided.
Drawings
FIG. 1 is a schematic flow chart of an automatic dividing method of farmland according to the invention;
FIG. 2 is a basic diagram of the automatic dividing method of farmland according to the invention;
FIG. 3 is a diagram of an encoder structure of a sentinel remote sensing image farmland rough segmentation network of the present invention;
FIG. 4 is a decoder structure diagram of a sentinel remote sensing image farmland rough segmentation network of the present invention;
FIG. 5 is a network structure diagram of the sentinel remote sensing image farmland subdivision method of the present invention;
fig. 6(a), fig. 6(b), and fig. 6(c) are examples of a single-row splicing method, a single-column splicing method, and a multi-row and multi-column splicing method for image blocks, respectively, according to the present invention;
fig. 7(a) and 7(b) are schematic diagrams of remote sensing images of sentinels in jin hu county, Huai' an city, Jiangsu province and the road information labeling method, respectively;
fig. 8(a), 8(b) and 8(c) are respectively a sentinel remote sensing image, a corresponding rough segmentation visualization result graph and a corresponding fine segmentation visualization result graph of Huaian city, Jinhui county, Jiangsu province;
fig. 9(a), 9(b), and 9(c) are respectively a sentinel remote sensing image, a corresponding rough segmentation visualization result diagram, and a corresponding fine segmentation visualization result diagram in Jiangsu province according to the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
As shown in fig. 1 and 2, an automatic farmland dividing method comprises the following steps:
step 1), cutting the original remote sensing image into image blocks, and sequentially inputting the image blocks into an encoder module of a farmland rough segmentation network. The module uses the void convolutions of different scales to extract the multi-scale features of the image, thereby solving the problem of inconsistent farm field scales in the same scene;
firstly, because the single remote sensing image has a large scale, the direct processing requires higher computing resources to be configured. Therefore, the invention cuts the original image, and divides the original image into image blocks for processing. Assuming that the original image is X, cutting X according to a sliding window with the size of 256 multiplied by 256 and with the step length of 64 to obtain a cut image block XiI ∈ {1, 2, …, n }, where n denotes the number of video blocks obtained by clipping.
In order to capture rich space context information of a remote sensing image, the encoder of the farmland rough segmentation network carries out feature extraction by using hole convolution, and the problem of space information loss caused by step convolution or pooling operation is solved. The output characteristic U can be obtained from the input characteristic V by the following equation:
U[j]＝∑kV[j+r·k]ω[k](1)
where j represents the spatial position of the input image and ω is the convolution kernel. r represents the expansion rate of the hole convolution, determines the sampling step length of the input signal, and can expand the reception field of k x k into keK + (k-1) (r-1). When r is 1, the convolution is standard.
The encoder of the field rough-cut network is shown in fig. 3. Image block xiAs the input of the encoder, extracting the features through an Xceptance network with cavity convolution to obtain shallow features f in the networkNAnd output characteristic fX. Then, f was mapped using a 1 × 1 convolutional layer, a 3 × 3 void convolutional layer with r equal to 6, a 3 × 3 convolutional void layer with r equal to 12, a 3 × 3 void convolutional layer with r equal to 18, and an image pooling operationXAnd performing multi-scale feature extraction. Sampling the multi-scale features to the same scale, thereby capturing farmland features f under different scalesM. For fMThe following table shows:
where F, P and C represent convolution, image pooling and feature combining operations, respectively. The upper and lower labels of the convolution kernel ω represent the expansion ratio r and the convolution kernel size, respectively.
Step 2), a decoder of the farmland rough segmentation network decodes the output characteristics of the encoder and the shallow characteristics in the convolutional network to obtain a segmentation result so as to retain the detail information of the farmland;
in order to relieve the problem of loss of detail information caused by the down-sampling operation, the decoder designed by the invention can combine deep features containing abundant spatial context information with shallow features containing more detail information, and retain the detail information of farmland in a step-by-step up-sampling mode.
The decoder of the field rough segmentation network is shown in fig. 4. The decoder will get the multi-scale feature fMShallow feature f in Xconcept with backbone networkNAnd (4) carrying out fusion. Fusion feature fMNObtained by the following formula:
fMN＝C(F(ω1×1，U(fM))，F(ω1×1，fN))(3)
where U denotes 4-fold upsampling with bilinear difference.
Then, the convolution layer with size of 1 × 1 is used to fuse the features fMNThe channel number is mapped into a category number, and then the channel number is up-sampled to the same size as the input image block by utilizing bilinear interpolation after being mapped into the category number to obtain a segmentation result y of the farmland rough segmentation networkFi。
yFi＝U(F(ω3×3，fMN))(4)
Step 3), positioning pixels belonging to a farmland in the remote sensing image by using the rough segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and thus eliminating pixels which divide roads such as ridges and the like into the farmland by mistake in the rough segmentation result;
by observing the remote sensing image, adjacent farmlands are usually separated by ridges and other roads. Since these roads are very fine, it is difficult to completely recognize the roughly divided network, and thus it is erroneously divided into farmlands. In order to solve the problem, the invention designs a farmland subdivision network. The network eliminates road pixels wrongly divided into farmlands by secondarily judging the farmland rough division result.
The field subdivision network is shown in fig. 5. First using the coarse segmentation result yFiPositioning xiAnd is predicted as a pixel point of the farmland. The invention cuts out a 9 multiplied by 9 area by taking the pixel point as the center
As input to the subdivided network. By 3 layers of convolution layer, the road can be coveredExtracting the information to obtain the road characteristics f of the areaR：
Will f isRExpanding into one-dimensional vector, inputting into a fully-connected layer with 2 hidden nodes, and converting into fRMapped into a two-dimensional vector. And obtaining the probability that the central pixel point belongs to the road and the farmland by using a Sigmoid activation function for the vector. Obtaining a judgment result d by the following formulaR：
dR＝A(σ(L(f(fR)，2)))(6)
Wherein, A, sigma, L and f respectively represent argmax function, Sigmoid function, full connection layer and flatten operation.
For the coarse segmentation result yFiAll the pixel points marked as farmland in the farmland are judged through a farmland fine segmentation network to obtain a judgment matrix DRThereby obtaining a fine segmentation result yi。yiThe expression of (c) is as follows:
yi＝DR·yFi (7)
and 4) splicing the fine segmentation results of the input image blocks in a mean value superposition mode to finish the segmentation of the original large image.
Subdivision of image blocksiSplicing is carried out by adopting a single-row splicing mode, a single-column splicing mode and a multi-row and multi-column splicing mode; wherein, the image blocks at the edge position in the original image are spliced in a single line in a left-right overlapping area when the segmentation result is segmented; an upper overlapping area and a lower overlapping area exist during the splicing of the segmentation results, and single-row splicing is carried out; and (3) carrying out multi-row and multi-column splicing on the image blocks which are positioned at the non-edge positions in the original image when the segmentation result is spliced, wherein the upper part, the lower part, the left part and the right part of the image blocks have overlapped areas, so as to obtain a remote sensing image segmentation result Y.
The overlap condition is mainly classified into the following three types: single row splicing, single column splicing and multi-row and multi-column splicing. The calculation of the overlapping portions in the three cases will be described below with reference to fig. 6(a), 6(b), and 6(c)And (5) clearing. In fig. 6(a), 6(b), and 6(c), the shaded portion indicates a portion where the video blocks overlap during the stitching process, the darker the color, the greater the number of overlaps, and the number indicates the number of overlaps. Fig. 6(a) shows a single-line stitching of three image blocks a, B, and C, where the image block B has two overlapping regions with the image blocks a and C, and the number of overlapping times l is 2. For example, the overlap region L1The length of the stack is 192, and the final probability value is:
the upper and lower labels of A and B are used for representing the horizontal coordinate and vertical coordinate range of the image block respectively.
The same single-row splicing of three image blocks as shown in FIG. 6 (b); the nine image blocks shown in fig. 6(c) are spliced in multiple rows and multiple columns, the number of times of overlapping is different in different overlapping areas, and the final value is obtained by means of mean value superposition in the same manner.
The embodiment applies the technical scheme provided by the invention to remote sensing image data of sentinels in Jinhu county in Huaian city, Jiangsu province. The spatial resolution of the image is 10 meters, and the size is 4600 × 4436. The image is composed of 4 channels, the order of the channels is: red, green, blue, near infrared. The image will be divided into 2 classes of objects: farmland and background (non-farmland).
The method solves the problem of inconsistent farmland scales by using multi-scale characteristics, and obtains road detail information such as ridges and the like by using a multi-stage segmentation strategy so as to obtain an accurate farmland segmentation result.
Firstly, an encoder of a farmland rough segmentation network extracts sentinel image characteristics by using a convolutional neural network, and extracts multi-scale characteristics by using cavity convolutions of different scales on the characteristics, so that the problem of different farmland scales is solved; then, in order to keep the detail information of the farmland, a decoder of the farmland rough segmentation network simultaneously uses the output characteristics of the encoder and shallow layer characteristics in the convolutional neural network for decoding to obtain a segmentation result; and finally, the farmland fine segmentation network carries out secondary judgment on the result of the farmland rough segmentation network, so that the road is prevented from being segmented into the farmland by mistake.
The classification process of this embodiment is specifically as follows:
1. cutting the original sentinel image into image blocks, and sequentially inputting the image blocks into an encoder of a farmland rough segmentation network. The encoder uses the convolutional neural network to extract the sentinel image characteristics, and extracts multi-scale characteristics by using the cavity convolutions of different scales for the characteristics, so that the problem of different farmland scales is solved:
because the sentinel image has a very large size, the current GPU resource is difficult to process. For this purpose, the invention uses a sliding window with size of 256 × 256 for the original sentinel image, and cuts it into image blocks with step length of 64.
The farmland rough segmentation network encoder uses the Xceptance with the cavity convolution as a backbone network of a model to preliminarily extract the characteristics; extracting multi-scale information by using a 1 × 1 convolutional layer, a 3 × 3 cavity convolutional layer with an expansion rate of 6, a 3 × 3 cavity convolutional layer with an expansion rate of 12, a 3 × 3 cavity convolutional layer with an expansion rate of 18 and image pooling operation on the extracted features respectively, thereby capturing farmland features under different scales;
2. the decoder of the farmland rough segmentation network uses the output characteristics of the encoder and the shallow layer characteristics in the convolutional network to decode to obtain a segmentation result so as to retain the detail information of the farmland:
the farmland rough-segmentation network decoder disclosed by the invention fuses the obtained multi-scale features with shallow-layer features in a convolutional network, maps the number of channels of output features into class numbers by using a 1 x 1 convolutional layer, and performs 4-time upsampling by using a bilinear difference value to obtain a segmentation result of the farmland rough-segmentation network.
In order to effectively train the rough farmland segmentation network, a Gaofen Image Dataset (GID) Dataset was selected as experimental training data. The GID is a large data set for land use and land cover classification. It contains 150 high quality second-grade (GF-2) images from 60 different cities in China, with image size of 6908 x 7300 pixels. Multispectral provides images in the blue, green, red and near-infrared bands.
3. The method comprises the following steps of positioning pixel points of a farmland in a sentinel image by using a rough segmentation result, carrying out secondary judgment on the pixel points through a farmland fine segmentation network formed by a convolutional neural network, and thus eliminating the pixel points which are obtained by wrongly segmenting roads such as ridges and the like into the farmland in the rough segmentation result:
the farmland fine segmentation network consists of three convolutional layers and a full connecting layer, wherein each convolutional layer consists of a convolution kernel of 3 multiplied by 3 and is used for carrying out feature extraction on the image. The number of hidden nodes of the full-link layer is set to be 2, the convolution features are mapped into two-dimensional vectors, and the vector values represent the probability that the input image belongs to roads and farmlands. The farmland fine segmentation network carries out secondary judgment on the result of the farmland rough segmentation network, and avoids the mistaken segmentation of the road into the farmland.
As shown in fig. 7(a) and 7(b), in order to more accurately segment the road, the present invention manually labels the road information of the image of the golden lake sentinel to obtain the road information of the whole image, and uses the information as the training data of the farmland subdivision network. And cutting out a small 9 x 9 image block by taking a certain pixel point in the road training data as a center. The label of the center pixel point is approximately represented as the label of the whole small image block. And defining the image block with the central pixel point as the road as a positive sample, and defining the image block with the central pixel point as the non-road as a negative sample, and obtaining training data according to the ratio of the positive sample to the negative sample of 1:1 to train the farmland subdivision network.
4. Splicing the fine segmentation results of the input image blocks in a mean value superposition mode to finish the segmentation of the original large image:
and (4) splicing all the image blocks with the sub-segmentation results according to the cutting method in the step 1, and replacing the predicted values of the overlapped parts by using a mean value. And obtaining the sentinel image segmentation result.
As shown in FIGS. 8(a), 8(b) and 8(c), after image preprocessing operation is performed, firstly, the image is subjected to primary farmland segmentation on the preprocessed image through a farmland rough segmentation network, and on the basis, a farmland fine segmentation network is used for carrying out secondary judgment on the primary result and refining the segmentation result. According to the experimental result, the result accuracy after the coarse segmentation network and the fine segmentation network are fused is obviously improved.
In order to further verify the generalization performance of the model provided by the invention, a test experiment is carried out on the whole image of the sentinel in Jiangsu province. As shown in fig. 9(a), 9(b), and 9(c), the results of the image experiments in jiangsu province are similar to those in jin hu county, and the accuracy of the results obtained by merging the rough-divided network and the fine-divided network is significantly improved.
Correspondingly, the invention also provides a farmland automatic segmentation system, which comprises:
the cutting module is used for obtaining an original remote sensing image, cutting the original remote sensing image into image blocks, and sequentially inputting the image blocks into the encoder module of the farmland rough segmentation network to obtain farmland characteristics under different scales;
the rough segmentation module is used for decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a rough segmentation result;
the fine segmentation module is used for positioning pixels belonging to a farmland in the original remote sensing image by using the coarse segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and eliminating pixels which are mistakenly divided into the farmland in the coarse segmentation result to obtain a fine segmentation result;
and the splicing module is used for splicing the input fine segmentation results of all the image blocks in a mean value superposition mode to finish the segmentation of the original large image.
The present invention accordingly also provides a computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform any of the methods described.
The invention also provides a computing device, comprising,
one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (8)
1. An automatic farmland dividing method is characterized by comprising the following steps:
acquiring an original remote sensing image, cutting the original remote sensing image into image blocks, and sequentially inputting the image blocks into an encoder module of a farmland rough segmentation network to obtain farmland characteristics under different scales;
decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a rough segmentation result;
positioning pixels belonging to a farmland in the original remote sensing image by using a rough segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and eliminating pixels wrongly divided into the farmland in the rough segmentation result to obtain a fine segmentation result;
and splicing the input fine segmentation results of all the image blocks in a mean value superposition mode to finish the segmentation of the original large image.
2. The farmland automatic segmentation method according to claim 1, wherein the cutting of the original remote sensing image into image blocks is sequentially input into an encoder module of a farmland rough segmentation network to obtain farmland characteristics under different scales, and comprises the following steps:
cutting the original remote sensing image X to obtain a cut image block XiI belongs to {1, 2, …, n }, wherein n represents the number of image blocks obtained by clipping;
image block xiAs the input of the encoder, extracting the features through an Xceptance network with cavity convolution to obtain shallow features f in the networkNAnd output characteristic fXThen, 1 × 1 convolutional layers, 3 × 3 void convolutional layers having an expansion ratio r of 6, 3 × 3 void convolutional layers having an expansion ratio r of 12, and 3 × 3 void convolutional layers having an expansion ratio r of 18 were used, respectivelyAnd image pooling operation, on fXExtracting multi-scale features, capturing farmland features f under different scalesMExpressed as:
wherein, F, P and C respectively represent convolution, image pooling and feature combining operation, and the upper and lower labels of the convolution kernel omega respectively represent the expansion ratio r and the size of the convolution kernel.
3. The method for automatically segmenting the farmland according to claim 2, wherein the decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a coarse segmentation result comprises the following steps:
the obtained farmland characteristics f under different scalesMShallow feature f in Xconcept with backbone networkNPerforming fusion to obtain a fusion characteristic fMNExpressed as:
fMN＝C(F(ω1×1，U(fM))，F(ω1×1，fN))(3)
wherein, U represents 4 times up-sampling by using bilinear difference;
fusing features f using convolution layers of size 1 × 1MNThe number of channels of (a) is mapped into a number of categories;
mapping the channels into category numbers, performing up-sampling to the size same as that of the input image block by utilizing bilinear interpolation to obtain a rough segmentation result y of the farmland rough segmentation networkFiExpressed as:
yFi＝U(F(ω1×1，fMN))(4)。
4. the automatic farmland segmentation method according to claim 3, wherein the method for locating the pixels belonging to the farmland in the remote sensing image by using the rough segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and eliminating the pixels wrongly divided into the farmland in the rough segmentation result to obtain the fine segmentation result comprises the following steps:
using the result y of the rough segmentationFiPositioning xiJudging each pixel point predicted as the farmland through a farmland subdivision network to obtain each judgment result dRBased on each judgment result dRObtaining a judgment matrix DRThen according to the judgment matrix DRAnd the coarse segmentation result yFiObtaining a segmentation result yiExpressed as:
yi＝DR·yFi (7)
obtaining each judgment result dRThe method comprises the following steps:
cutting out a fixed area by taking the pixel point as a center
As the input of the fine division network, the road information is extracted by the 3-layer convolution layer to obtain the road characteristic f of the areaR：
Will f isRSpreading into one-dimensional vector, inputting into a full-connection layer with hidden node number of 2, and adding fRMapping into a two-dimensional vector, obtaining the probability that the central pixel point belongs to the road and the farmland by using a Sigmoid activation function on the two-dimensional vector, and obtaining a judgment result d through the following formulaR：
dR＝A(σ(L(f(fR)，2)))(6)
Wherein, A, sigma, L and f respectively represent argmax function, Sigmoid function, full connection layer and flatten operation.
5. The farmland automatic segmentation method according to claim 4, wherein the step of splicing the input fine segmentation results of all image blocks in a mean value superposition manner to complete segmentation of the original large image comprises the following steps:
subdivision of image blocksiSplicing is carried out by adopting a single-row splicing mode, a single-column splicing mode and a multi-row and multi-column splicing mode; wherein, the image blocks at the edge position in the original image are spliced in a single line in a left-right overlapping area when the segmentation result is segmented; an upper overlapping area and a lower overlapping area exist during the splicing of the segmentation results, and single-row splicing is carried out; the image blocks in the original image at non-edge positions have overlapped areas at the upper, lower, left and right sides when the segmentation result is spliced, and multi-row and multi-column splicing is carried out; and replacing the predicted values of the overlapped areas by using the mean values to obtain a remote sensing image segmentation result Y.
6. An automatic farmland dividing system, comprising:
the cutting module is used for acquiring an original remote sensing image, cutting the original remote sensing image into image blocks, and sequentially inputting the image blocks into the encoder module of the farmland rough segmentation network to obtain farmland characteristics under different scales;
the rough segmentation module is used for decoding according to the farmland characteristics under different scales and the shallow layer characteristics in the convolutional network to obtain a rough segmentation result;
the fine segmentation module is used for positioning pixels belonging to a farmland in the original remote sensing image by using the coarse segmentation result, carrying out secondary judgment on the pixels through a farmland fine segmentation network formed by a convolutional neural network, and eliminating pixels which are mistakenly divided into the farmland in the coarse segmentation result to obtain a fine segmentation result;
and the splicing module is used for splicing the input fine segmentation results of all the image blocks in a mean value superposition mode to finish the segmentation of the original large image.
7. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform any of the methods of claims 1-5.
8. A computing device, comprising,
one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111602281.4A CN114494283A (en) | 2021-12-24 | 2021-12-24 | Automatic farmland dividing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111602281.4A CN114494283A (en) | 2021-12-24 | 2021-12-24 | Automatic farmland dividing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114494283A true CN114494283A (en) | 2022-05-13 |
Family
ID=81496026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111602281.4A Pending CN114494283A (en) | 2021-12-24 | 2021-12-24 | Automatic farmland dividing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494283A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115272667A (en) * | 2022-06-24 | 2022-11-01 | 中科星睿科技(北京)有限公司 | Farmland image segmentation model training method and device, electronic equipment and medium |
| CN115810020A (en) * | 2022-12-02 | 2023-03-17 | 中国科学院空间应用工程与技术中心 | Remote sensing image segmentation method and system from coarse to fine based on semantic guidance |
-
2021
- 2021-12-24 CN CN202111602281.4A patent/CN114494283A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115272667A (en) * | 2022-06-24 | 2022-11-01 | 中科星睿科技(北京)有限公司 | Farmland image segmentation model training method and device, electronic equipment and medium |
| CN115272667B (en) * | 2022-06-24 | 2023-08-29 | 中科星睿科技(北京)有限公司 | Farmland image segmentation model training method and device, electronic equipment and medium |
| CN115810020A (en) * | 2022-12-02 | 2023-03-17 | 中国科学院空间应用工程与技术中心 | Remote sensing image segmentation method and system from coarse to fine based on semantic guidance |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| US20210256258A1 (en) | Method, apparatus, and computer program for extracting representative characteristics of object in image | |
| CN111428781A (en) | Remote sensing image ground object classification method and system | |
| CN114494283A (en) | Automatic farmland dividing method and system | |
| CN108629286B (en) | Remote sensing airport target detection method based on subjective perception significance model | |
| CN113609896A (en) | Object-level remote sensing change detection method and system based on dual-correlation attention | |
| CN114627052A (en) | Infrared image air leakage and liquid leakage detection method and system based on deep learning | |
| CN110503052A (en) | A kind of image, semantic dividing method based on improvement U-NET network | |
| CN116645592B (en) | Crack detection method based on image processing and storage medium | |
| CN111680690A (en) | Character recognition method and device | |
| CN114724155A (en) | Scene text detection method, system and equipment based on deep convolutional neural network | |
| CN112883926A (en) | Identification method and device for table medical images | |
| CN114299383A (en) | Remote sensing image target detection method based on integration of density map and attention mechanism | |
| CN114241256A (en) | Training sample image enhancement method and system based on neural network | |
| CN113989744A (en) | Pedestrian target detection method and system based on oversized high-resolution image | |
| CN115100469A (en) | Target attribute identification method, training method and device based on segmentation algorithm | |
| Dong et al. | A deep learning based framework for remote sensing image ground object segmentation | |
| CN114445620A (en) | Target segmentation method for improving Mask R-CNN | |
| CN112132164B (en) | Target detection method, system, computer device and storage medium | |
| CN116403132B (en) | Ground object identification method for generating symptom ground removal table based on image and machine algorithm | |
| Patel et al. | A novel approach for detecting number plate based on overlapping window and region clustering for Indian conditions | |
| CN113657225B (en) | Target detection method | |
| CN112330659B (en) | Geometric tolerance symbol segmentation method combining LSD (least squares) linear detection and connected domain marking method | |
| CN117372437B (en) | Intelligent detection and quantification method and system for facial paralysis | |
| CN113888758B (en) | Curved character recognition method and system based on complex scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |