CN112215096A - Remote sensing image town extraction method and device based on scene and pixel information - Google Patents
Remote sensing image town extraction method and device based on scene and pixel information Download PDFInfo
- Publication number
- CN112215096A CN112215096A CN202011024302.4A CN202011024302A CN112215096A CN 112215096 A CN112215096 A CN 112215096A CN 202011024302 A CN202011024302 A CN 202011024302A CN 112215096 A CN112215096 A CN 112215096A
- Authority
- CN
- China
- Prior art keywords
- image
- town
- grid
- remote sensing
- classified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 152
- 238000000034 method Methods 0.000 claims abstract description 34
- 238000012805 post-processing Methods 0.000 claims abstract description 19
- 238000002372 labelling Methods 0.000 claims description 42
- 238000005520 cutting process Methods 0.000 claims description 30
- 238000012549 training Methods 0.000 claims description 26
- 238000013145 classification model Methods 0.000 claims description 6
- 238000003708 edge detection Methods 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 6
- 238000009826 distribution Methods 0.000 description 13
- 238000013527 convolutional neural network Methods 0.000 description 12
- 238000011161 development Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A30/00—Adapting or protecting infrastructure or their operation
- Y02A30/60—Planning or developing urban green infrastructure
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for extracting a remote sensing image town based on scene and pixel information. The method comprises the following steps: carrying out grid division on a remote sensing image to be processed to generate a plurality of grid images; inputting the grid image into a target scene classification network model to obtain a grid classification image; performing edge extraction operation on the classified images of the grids, and marking the classified images of the grids which are edges; inputting the marked grid classified image into a target pixel classified network model, and acquiring a pixel classification result image as a town extraction result image of a corresponding region in a town extraction result; assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image of the town category; assigning a value of 0 to a corresponding area in the town extraction result image aiming at the non-town category grid image; and carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image. The invention achieves the completeness of town range extraction and the accuracy of town boundary extraction.
Description
Technical Field
The invention relates to the technical field of satellite remote sensing image processing, in particular to a remote sensing image town extraction method and device based on scene and pixel information.
Background
The town crowd is a main occurrence place of spatial redistribution and transfer of production elements under the new situation of economic globalization, is a product of industrial space integration, and occupies a core position in national and regional economic development. In the new environment, competition in economic development between regions and countries requires close cooperation and interaction between cities and industries within a metropolitan area according to the role they play. Scientific and reasonable definition and understanding of the town pool are beneficial to monitoring, managing and promoting coordinated development among various cities in the town pool. In traditional urban geographic research, the definition and definition of urban groups mainly depend on socio-economic statistics. The methods have higher requirements on the completeness and accuracy of each index of statistical data, and cannot visually reflect the spatial relationship among cities in the town group, the space form and the space mode of the town group. The introduction of the remote sensing technology provides a new way for acquiring spatial information of towns and town groups. The existing remote sensing-based city and urban community research proves that the remote sensing technology has great advantages in the aspect of acquiring and analyzing urban spatial information.
The remote sensing image town identification is generally carried out by adopting the existing land covering/land utilization classification method, the classification also belongs to the category of pattern identification, and the research of the classification method mainly focuses on the feature extraction of town ground object targets and the design and selection of classifiers. The town extraction based on the remote sensing image needs to obtain a boundary range of town distribution, the town usually contains a plurality of different ground surface coverage types, the ground surface coverage types are represented as a multi-land complex, and impermeable layer types such as artificial building facilities and the like can sporadically appear outside the town distribution range. Under the condition, the traditional classification method based on the pixel can accurately extract the boundary range of town distribution, but usually only can extract impervious layers such as roads, residential areas and the like in towns, so that lakes, rivers, vegetation and other places in the town range cannot be all extracted at one time, and meanwhile, scattered artificial building facilities outside the town range can be wrongly extracted as the town area, so that the town extraction result is fragmented and lacks integrity. On the other hand, the classification method based on the scene can extract the overall range of the town area from the whole in a grid mode according to the spatial distribution characteristics of the town, but the method is not ideal for processing the town boundary and mostly presents a sawtooth phenomenon. In order to improve the integrity of town range extraction and the accuracy of town boundary extraction, the application requirement of the town extraction of the current remote sensing image cannot be well met by singly considering any one method.
Disclosure of Invention
The technical problem solved by the invention is as follows: the method and the device for extracting the town of the remote sensing image based on the scene and the pixel information are provided.
In order to solve the technical problem, an embodiment of the present invention provides a remote sensing image town extraction method based on pixel information, including:
carrying out grid division on the obtained remote sensing image to be processed to generate a plurality of grid images corresponding to the remote sensing image to be processed;
inputting each grid image into a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image;
performing edge extraction operation on the classified grid image, and marking the classified grid image which is the edge according to an extraction result;
inputting the marked grid classified image into a pre-trained target pixel classified network model to obtain a pixel classification result image corresponding to the marked grid classified image, and taking the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result;
assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image judged as the town category by the target scene classification network model;
assigning a value of 0 to a corresponding area in the town extraction result image aiming at the grid image judged as a non-town category by the target scene classification network model;
and carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image.
Optionally, before the inputting each grid image into a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image, the method further includes:
obtaining homologous town satellite remote sensing images and non-town satellite remote sensing images;
cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cut image corresponding to the town satellite remote sensing image and a second cut image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
labeling the first cut image and the second cut image respectively according to the town scene category and the non-town scene category, and respectively generating a first labeled image corresponding to the first cut image and a second labeled image corresponding to the second cut image;
training an initial scene classification network model based on the first annotation image and the second annotation image to obtain the target scene classification network model.
Optionally, before the step of inputting the labeled classified image of the mesh into a pre-trained target pixel classification network model to obtain a pixel classification result image corresponding to the labeled classified image of the mesh, and taking the pixel classification result image as an extraction result of a corresponding region in a town extraction result, the method further includes:
obtaining homologous town satellite remote sensing images and non-town satellite remote sensing images;
cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cut image corresponding to the town satellite remote sensing image and a second cut image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
labeling the town type and the non-town type of each pixel in the first cut image and the second cut image to respectively generate a first pixel labeling image corresponding to the first cut image and a second pixel labeling image corresponding to the second cut image;
and training an initial pixel classification network model based on the pixel annotation images of the town categories in the first pixel annotation image and the second pixel annotation image to obtain the target pixel classification network model.
Optionally, the performing an edge extraction operation on the classified mesh image, and labeling the classified mesh image as an edge according to an extraction result includes:
and performing edge extraction on the edges of the classified grid images based on an image edge detection algorithm, marking the grid images identified as the edges as E, and not marking the non-edge grid images.
Optionally, the inputting the labeled grid classified image into a pre-trained target pixel classification network model to obtain a pixel classification result image corresponding to the labeled grid classified image, and using the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result includes:
establishing a blank image with the same size as the remote sensing image to be processed so as to store the town extraction result image;
carrying out condition judgment on each grid classified image in the grid classified images;
if the classified grid image is of a town type and is marked as E, inputting the classified grid image into the target pixel classification network model to predict a pixel classification result, and assigning the pixel classification result image to the same area in the blank image;
if the classified image of the grid is an image of a town category and is not marked, all pixels in the same area in the blank image are assigned to be 1;
and if the classified grid image is of a non-town type, all pixels in the same area in the blank image are assigned to be 0, and finally the town extraction result image is a 0-1 binary image.
Optionally, the performing hole filling post-processing on the town extraction result image to obtain a town extraction binary image includes:
and performing post-processing on the town mosaic image by using a hole filling algorithm in morphological operation, and filling pixels which are located inside the town and have pixel values of 0 to obtain a town extracted binary image.
In order to solve the above technical problem, an embodiment of the present invention further provides a remote sensing image town extraction device based on scene and pixel information, including:
the grid image generation module is used for carrying out grid division on the acquired remote sensing image to be processed and generating a plurality of grid images corresponding to the remote sensing image to be processed;
a classified image obtaining module, configured to input each grid image into a pre-trained target scene classification network model, so as to obtain a grid classified image corresponding to each grid image;
the classified image marking module is used for executing edge extraction operation on the grid classified image and marking the grid classified image which is the edge according to an extraction result;
an extraction result image acquisition module, configured to input the labeled grid classified image into a pre-trained target pixel classification network model to acquire a pixel classification result image corresponding to the labeled grid classified image, and use the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result;
the first image assignment module is used for assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image judged as the town category by the target scene classification network model;
the second image assignment module is used for assigning 0 to a corresponding area in the town extraction result image aiming at the grid image which is judged to be in a non-town category by the target scene classification network model;
and the town binary image acquisition module is used for carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image.
Optionally, the method further comprises:
the first remote sensing image acquisition module is used for acquiring a homologous town satellite remote sensing image and a non-town satellite remote sensing image;
the first cutting image acquisition module is used for cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cutting image corresponding to the town satellite remote sensing image and a second cutting image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
the first labeling image generating module is used for labeling the first clipping image and the second clipping image respectively according to the town scene category and the non-town scene category, and respectively generating a first labeling image corresponding to the first clipping image and a second labeling image corresponding to the second clipping image;
and the target classification model acquisition module is used for training an initial scene classification network model based on the first annotation image and the second annotation image to obtain the target scene classification network model.
Optionally, the method further comprises:
the second remote sensing image acquisition module is used for acquiring a homologous town satellite remote sensing image and a non-town satellite remote sensing image;
the second cutting image acquisition module is used for cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cutting image corresponding to the town satellite remote sensing image and a second cutting image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer greater than or equal to 1;
the second labeling image generating module is used for labeling each pixel in the first clipping image and the second clipping image in a town category and a non-town category, and respectively generating a first pixel labeling image corresponding to the first clipping image and a second pixel labeling image corresponding to the second clipping image;
and the target pixel classification model acquisition module is used for training an initial pixel classification network model based on the pixel annotation images of the town categories in the first pixel annotation image and the second pixel annotation image to obtain the target pixel classification network model.
Optionally, the classified image labeling module includes:
and the classified image marking unit is used for carrying out edge extraction on the classified images of the grids based on an image edge detection algorithm, marking the grid images identified as edges as E, and not marking the grid images without edges.
Optionally, the extraction result image obtaining module includes:
the blank image establishing unit is used for establishing a blank image with the same size as the remote sensing image to be processed so as to store the town extraction result image;
a grid classified image judging unit, configured to perform condition judgment on each grid classified image in the grid classified images;
a first image assignment unit, configured to, if the classified grid image is of a town category and is marked as E, input the classified grid image into the target pixel classification network model to predict a pixel classification result thereof, and assign the pixel classification result image to the same region in the blank image;
a second image assignment unit, configured to assign all pixels in the same area in the blank image to 1 if the classified grid image is an image of a town category and is not marked;
and the third image assignment unit is used for assigning all pixels in the same area in the blank image as 0 if the classified grid image is of a non-town type, and finally, the town extraction result image is a 0-1 binary image.
Optionally, the town binary image obtaining module includes:
and the town binary image acquisition unit is used for performing post-processing on the town mosaic image by utilizing a hole filling algorithm in morphological operation, filling pixels which are located inside the town and have pixel values of 0, and obtaining the town extracted binary image.
Compared with the prior art, the invention has the advantages that:
the embodiment of the invention provides a method and a device for extracting town images of remote sensing images based on scene and pixel information. Carrying out grid division on the obtained remote sensing image to be processed to generate a plurality of grid images corresponding to the remote sensing image to be processed; inputting each grid image into a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image; performing edge extraction operation on the classified grid image, and marking the classified grid image which is the edge according to an extraction result; inputting the marked grid classified image into a pre-trained target pixel classified network model to obtain a pixel classification result image corresponding to the marked grid classified image, and taking the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result; assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image judged as the town category by the target scene classification network model; assigning a value of 0 to a corresponding area in the town extraction result image aiming at the grid image judged as a non-town category by the target scene classification network model; and carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image. The embodiment of the invention firstly utilizes a scene classification method to determine the range of the town distribution target area, then utilizes the pixel classification to subdivide the town boundary, and finally utilizes a hole filling algorithm to obtain the accurate and complete town distribution area.
Drawings
FIG. 1 is a flow chart of steps of a method for extracting a town from a remote sensing image based on scene and pixel information, provided by an embodiment of the invention;
fig. 2 is a schematic structural diagram of a remote sensing image town extraction device based on scene and pixel information provided by an embodiment of the invention.
Detailed Description
Example one
Referring to fig. 1, a flowchart illustrating steps of a town extraction method for a remote sensing image based on scene and pixel information according to an embodiment of the present invention is shown, and as shown in fig. 1, the town extraction method for a remote sensing image based on scene and pixel information may specifically include the following steps:
step 101: and carrying out grid division on the obtained remote sensing image to be processed to generate a plurality of grid images corresponding to the remote sensing image to be processed.
The embodiment of the invention can be applied to extracting the town extraction scene of the town distribution area in the remote sensing image.
The remote sensing image to be processed is a satellite remote sensing image needing town extraction.
After the remote sensing image to be processed is obtained, grid division can be performed on the remote sensing image to be processed to generate a plurality of grid images corresponding to the remote sensing image to be processed, it can be understood that for the remote sensing image to be processed, the remote sensing image to be processed must be a satellite remote sensing image with a training target scene classification network model and a target pixel classification network model which are homologous, the size of the remote sensing image to be processed can be P × Q, wherein P and Q are positive integers, grid division is performed on the image according to a sliding window with the size of N × N, wherein N is a positive integer and is an integer multiple of 2, each grid image is input into the trained CNN network model, the scene type of the grid image is output, the town type is 1, the non-town type is 0, a grid classification result image of the original remote sensing image is obtained, and the image is a binary image of 0-1. The method comprises the steps that a training image is divided according to a training target scene classification network model and a target pixel classification network model, wherein NxN is the size of the training image divided according to the training target scene classification network model and the target pixel classification network model, and in the process, the size of a grid image divided by a remote sensing image to be processed is required to be kept consistent with that of a grid image divided by a training sample.
After grid division is performed on the remote sensing image to be processed, and a plurality of grid images corresponding to the remote sensing image to be processed are generated, step 102 is executed.
Step 102: and inputting each grid image into a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image.
The target scene classification network model may be a CNN network model, the CNN network model may perform classification of a mesh image into a town category and a non-town category, and a training process of the CNN network model may be described in detail in combination with the following specific implementation manner.
In a specific implementation manner of the present invention, before the step 102, the method may further include:
step A1: obtaining homologous town satellite remote sensing images and non-town satellite remote sensing images;
step A2: cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cut image corresponding to the town satellite remote sensing image and a second cut image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
step A3: labeling the first cut image and the second cut image respectively according to the town scene category and the non-town scene category, and respectively generating a first labeled image corresponding to the first cut image and a second labeled image corresponding to the second cut image;
step A4: training an initial scene classification network model based on the first annotation image and the second annotation image to obtain the target scene classification network model.
In the embodiment of the invention, a training sample set of a town scene image and a non-town scene image can be established first, a CNN (convolutional neural network) model of the convolutional neural network is trained, specifically, a certain satellite remote sensing image is selected, images of different spatial distribution areas are downloaded to ensure the representativeness of the established sample set, the images are cut according to the size of NxN (namely a first cut image and a second cut image), wherein N is a positive integer, the types of the cut image blocks are labeled, the image block of the town scene is labeled as 1 (namely the first labeled image), the image block of the non-town scene is labeled as 0 (namely the second labeled image), and the training sample set of the town scene image and the non-town scene image is obtained.
After the training sample sets of the town scene images and the non-town scene images are obtained, a certain pre-trained CNN network model can be selected, the network output structure is modified to adapt to the classification requirements of 2 classes of towns and non-towns, and the pre-trained CNN network model is subjected to fine tuning training of network parameters by using the constructed town scene images and the non-town scene image training sample sets to obtain the trained CNN network model, namely the target scene classification network model.
At this time, after a plurality of grid images corresponding to the remote sensing image to be processed are generated, each grid image may be input to the target scene classification network model, and the scene category of each grid image is output by the target scene classification network model, where the town category is 1 and the non-town category is 0, so as to obtain a grid classification result image of the remote sensing image to be processed, where the image is a 0-1 binary image.
After each grid image is input to a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image, step 103 is performed.
Step 103: and performing edge extraction operation on the classified images of the grids, and marking the classified images of the grids which are edges according to the extraction result.
After the classified grid image corresponding to each grid image input by the target scene classification network model is obtained, the classified images of the town grids belonging to the town category in the classified grid images can be obtained, specifically, the image processing edge detection algorithm can be used for carrying out edge extraction on the classified grid result image, the grid identified as the edge is marked as E, and other grids which are not the edge are not marked.
After the town mesh classification image is acquired, step 104 is performed.
Step 104: and inputting the marked grid classified image into a pre-trained target pixel classified network model to obtain a pixel classification result image corresponding to the marked grid classified image, and taking the pixel classification result image as a town extraction result image of a corresponding region in the town extraction result.
Step 105: assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image judged as the town category by the target scene classification network model;
step 106: assigning a value of 0 to a corresponding area in the town extraction result image aiming at the grid image judged as a non-town category by the target scene classification network model;
the target pixel classification network model may be a model for classifying each pixel in the town mesh classification image, and in this example, the target pixel classification network model may be an FCN network model. The training process for the target pixel classification network model can be described in detail in conjunction with the following specific implementation manner.
In another specific implementation manner of the present invention, before the step 104, the method may further include:
step B1: obtaining homologous town satellite remote sensing images and non-town satellite remote sensing images;
step B2: cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cut image corresponding to the town satellite remote sensing image and a second cut image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
step B3: labeling the town type and the non-town type of each pixel in the first cut image and the second cut image to respectively generate a first pixel labeling image corresponding to the first cut image and a second pixel labeling image corresponding to the second cut image;
step B4: and training an initial pixel classification network model based on the pixel annotation images of the town categories in the first pixel annotation image and the second pixel annotation image to obtain the target pixel classification network model.
In the embodiment of the invention, a town pixel classification truth value image set can be established, a full connection network FCN (fuzzy C-means network) model is trained, satellite remote sensing images which are homologous with town and non-town scene images are selected, images in different space distribution areas are downloaded to ensure the representativeness of the established sample set, the images are cut according to the size of NxN, wherein N is a positive integer and is an integral multiple of 2, each pixel in the cut image block is labeled with a category, the town pixel is labeled as 1, the non-town pixel is labeled as 0, a pixel category labeled image with the same size as the image block is obtained, and the pixel category labeled image and the original image block form the town pixel classification truth value image set.
And designing an FCN network structure, requiring the size of an input image to be NxN, wherein N is a positive integer, the size of an output image is the same as that of the input image, and performing parameter training on the FCN network by using the constructed town pixel classification true value image set to obtain a trained FCN network model.
After the town grid classified image is obtained, the town grid classified image can be input into a target pixel classified network model to predict the pixel classification result, the pixel classification result image is used as the extraction result of the corresponding region in the town extraction result, the grid image judged as the town category by the CNN network model and without the mark of the grid is directly assigned with 1 for the corresponding region in the town extraction result, and the grid image judged as the non-town category by the CNN network model and with 0 for the corresponding region in the town extraction result. The specific implementation process is as follows: establishing a blank image I with the same size as the remote sensing image to be subjected to town extraction and the size of P multiplied by Q, wherein the blank image I is used for storing a town extraction result image, and P and Q are positive integers; and (3) carrying out condition judgment on each grid image in the grid classification result image: if the grid is of a town category and marked as E, inputting the grid into an FCN (fuzzy C-means network) model to predict a pixel classification result, and assigning the pixel classification result image to the same area in the image I; if the grid is a grid image of a town category but is not marked, all pixels of the same area in the image I are directly assigned to be 1; if the grid is in a non-town category, all pixels in the same area in the image I are directly assigned to be 0; and finally, obtaining a town extraction preliminary result which is a 0-1 binary image.
After assigning values to the corresponding regions of the resulting town extraction result image, step 107 is performed.
Step 107: and carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image.
Further, hole filling post-processing is carried out on the town extraction result image to obtain a final town extraction binary image, specifically, a hole filling algorithm in morphological operation can be used for carrying out post-processing on a preliminary town extraction result (a processed image), pixels which are located inside the town and have pixel values of 0 are filled, and then the final town extraction binary image is obtained.
The remote sensing image town extraction method based on scene and pixel information provided by the embodiment of the invention generates a plurality of grid images corresponding to the remote sensing image to be processed by grid division of the obtained remote sensing image to be processed; inputting each grid image into a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image; performing edge extraction operation on the classified grid image, and marking the classified grid image which is the edge according to an extraction result; inputting the marked grid classified image into a pre-trained target pixel classified network model to obtain a pixel classification result image corresponding to the marked grid classified image, and taking the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result; assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image judged as the town category by the target scene classification network model; assigning a value of 0 to a corresponding area in the town extraction result image aiming at the grid image judged as a non-town category by the target scene classification network model; and carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image. The embodiment of the invention firstly utilizes a scene classification method to determine the range of the town distribution target area, then utilizes the pixel classification to subdivide the town boundary, and finally utilizes a hole filling algorithm to obtain the accurate and complete town distribution area.
Example two
Referring to fig. 2, a schematic structural diagram of a remote sensing image town extraction device based on scene and pixel information according to an embodiment of the present invention is shown, and as shown in fig. 2, the remote sensing image town extraction device based on scene and pixel information may specifically include the following modules:
a grid image generation module 210, configured to perform grid division on the obtained remote sensing image to be processed, and generate a plurality of grid images corresponding to the remote sensing image to be processed;
a classified image obtaining module 220, configured to input each grid image into a pre-trained target scene classification network model, so as to obtain a grid classified image corresponding to each grid image;
a classified image labeling module 230, configured to perform an edge extraction operation on the classified grid image, and label the classified grid image as an edge according to an extraction result;
an extraction result image obtaining module 240, configured to input the labeled grid classified image into a pre-trained target pixel classification network model, so as to obtain a pixel classification result image corresponding to the labeled grid classified image, and use the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result;
a first image assignment module 250, configured to assign a value of 1 to a corresponding region in the town extraction result image, for an unlabeled grid image that is determined as a town category by the target scene classification network model;
a second image assignment module 260, configured to assign a value of 0 to a corresponding area in the town extraction result image, for the mesh image determined as a non-town category by the target scene classification network model;
and a town binary image obtaining module 270, configured to perform hole filling post-processing on the town extraction result image to obtain a town extraction binary image.
Optionally, the method further comprises:
the first remote sensing image acquisition module is used for acquiring a homologous town satellite remote sensing image and a non-town satellite remote sensing image;
the first cutting image acquisition module is used for cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cutting image corresponding to the town satellite remote sensing image and a second cutting image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
the first labeling image generating module is used for labeling the first clipping image and the second clipping image respectively according to the town scene category and the non-town scene category, and respectively generating a first labeling image corresponding to the first clipping image and a second labeling image corresponding to the second clipping image;
and the target classification model acquisition module is used for training an initial scene classification network model based on the first annotation image and the second annotation image to obtain the target scene classification network model.
Optionally, the method further comprises:
the second remote sensing image acquisition module is used for acquiring a homologous town satellite remote sensing image and a non-town satellite remote sensing image;
the second cutting image acquisition module is used for cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cutting image corresponding to the town satellite remote sensing image and a second cutting image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer greater than or equal to 1;
the second labeling image generating module is used for labeling each pixel in the first clipping image and the second clipping image in a town category and a non-town category, and respectively generating a first pixel labeling image corresponding to the first clipping image and a second pixel labeling image corresponding to the second clipping image;
and the target pixel classification model acquisition module is used for training an initial pixel classification network model based on the pixel annotation images of the town categories in the first pixel annotation image and the second pixel annotation image to obtain the target pixel classification network model.
Optionally, the classified image labeling module includes:
and the classified image marking unit is used for carrying out edge extraction on the classified images of the grids based on an image edge detection algorithm, marking the grid images identified as edges as E, and not marking the grid images without edges.
Optionally, the extraction result image obtaining module includes:
the blank image establishing unit is used for establishing a blank image with the same size as the remote sensing image to be processed so as to store the town extraction result image;
a grid classified image judging unit, configured to perform condition judgment on each grid classified image in the grid classified images;
a first image assignment unit, configured to, if the classified grid image is of a town category and is marked as E, input the classified grid image into the target pixel classification network model to predict a pixel classification result thereof, and assign the pixel classification result image to the same region in the blank image;
a second image assignment unit, configured to assign all pixels in the same area in the blank image to 1 if the classified grid image is an image of a town category and is not marked;
and the third image assignment unit is used for assigning all pixels in the same area in the blank image as 0 if the classified grid image is of a non-town type, and finally, the town extraction result image is a 0-1 binary image.
Optionally, the town binary image obtaining module includes:
and the town binary image acquisition unit is used for performing post-processing on the town mosaic image by utilizing a hole filling algorithm in morphological operation, filling pixels which are located inside the town and have pixel values of 0, and obtaining the town extracted binary image.
The remote sensing image town extraction device based on scene and pixel information provided by the embodiment of the invention generates a plurality of grid images corresponding to the remote sensing image to be processed by grid division of the obtained remote sensing image to be processed; inputting each grid image into a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image; performing edge extraction operation on the classified grid image, and marking the classified grid image which is the edge according to an extraction result; inputting the marked grid classified image into a pre-trained target pixel classified network model to obtain a pixel classification result image corresponding to the marked grid classified image, and taking the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result; for the grid image judged as the town category by the target scene classification network model and not marked, assigning a value of 1 to a corresponding area in the town extraction result image; assigning a value of 0 to a corresponding area in the town extraction result image aiming at the grid image judged as a non-town category by the target scene classification network model; and carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image. The embodiment of the invention firstly utilizes a scene classification method to determine the range of the town distribution target area, then utilizes the pixel classification to subdivide the town boundary, and finally utilizes a hole filling algorithm to obtain the accurate and complete town distribution area.
It should be noted that the above-described embodiments may enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Thus, it will be appreciated by those skilled in the art that the invention may be modified and equivalents may be substituted; all technical solutions and modifications thereof which do not depart from the spirit and technical essence of the present invention should be covered by the scope of the present patent.
Those skilled in the art will appreciate that those matters not described in detail in the present specification are well known in the art.
Claims (12)
1. A remote sensing image town extraction method based on scene and pixel information is characterized by comprising the following steps:
carrying out grid division on the obtained remote sensing image to be processed to generate a plurality of grid images corresponding to the remote sensing image to be processed;
inputting each grid image into a pre-trained target scene classification network model to obtain a grid classification image corresponding to each grid image;
performing edge extraction operation on the classified grid image, and marking the classified grid image which is the edge according to an extraction result;
inputting the marked grid classified image into a pre-trained target pixel classified network model to obtain a pixel classification result image corresponding to the marked grid classified image, and taking the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result;
assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image judged as the town category by the target scene classification network model;
assigning a value of 0 to a corresponding area in the town extraction result image aiming at the grid image judged as a non-town category by the target scene classification network model;
and carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image.
2. The method according to claim 1, wherein before said inputting each of said mesh images into a pre-trained target scene classification network model to obtain a mesh classification image corresponding to each of said mesh images, further comprising:
obtaining homologous town satellite remote sensing images and non-town satellite remote sensing images;
cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cut image corresponding to the town satellite remote sensing image and a second cut image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
labeling the first cut image and the second cut image respectively according to the town scene category and the non-town scene category, and respectively generating a first labeled image corresponding to the first cut image and a second labeled image corresponding to the second cut image;
training an initial scene classification network model based on the first annotation image and the second annotation image to obtain the target scene classification network model.
3. The method according to claim 1, wherein before inputting the labeled classified image of the mesh into a pre-trained target image element classification network model to obtain an image element classification result image corresponding to the labeled classified image of the mesh, and using the image element classification result image as an extraction result of a corresponding region in the town extraction result, the method further comprises:
obtaining homologous town satellite remote sensing images and non-town satellite remote sensing images;
cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cut image corresponding to the town satellite remote sensing image and a second cut image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
labeling the town type and the non-town type of each pixel in the first cut image and the second cut image to respectively generate a first pixel labeling image corresponding to the first cut image and a second pixel labeling image corresponding to the second cut image;
and training an initial pixel classification network model based on the pixel annotation images of the town categories in the first pixel annotation image and the second pixel annotation image to obtain the target pixel classification network model.
4. The method of claim 1, wherein performing an edge extraction operation on the mesh classification image and labeling mesh classification images that are edges according to the extraction result comprises:
and carrying out edge extraction on the classified grid image based on an image edge detection algorithm, marking the grid image identified as the edge as E, and not marking the non-edge grid image.
5. The method according to claim 4, wherein said inputting the labeled classified image of the mesh into a pre-trained target pixel classification network model to obtain a pixel classification result image corresponding to the labeled classified image of the mesh, and using the pixel classification result image as a town extraction result image of a corresponding region in the town extraction result comprises:
establishing a blank image with the same size as the remote sensing image to be processed so as to store the town extraction result image;
carrying out condition judgment on each grid classified image in the grid classified images;
if the classified grid image is of a town type and is marked as E, inputting the classified grid image into the target pixel classification network model to predict a pixel classification result, and assigning the pixel classification result image to the same area in the blank image;
if the classified image of the grid is an image of a town category and is not marked, all pixels in the same area in the blank image are assigned to be 1;
and if the classified grid image is of a non-town type, all pixels in the same area in the blank image are assigned to be 0, and finally the town extraction result image is a 0-1 binary image.
6. The method of claim 1, wherein the performing hole filling post-processing on the town extraction result image to obtain a town extraction binary image comprises:
and performing post-processing on the town mosaic image by using a hole filling algorithm in morphological operation, and filling pixels which are located inside the town and have pixel values of 0 to obtain a town extracted binary image.
7. A remote sensing image town extraction device based on scene and pixel information is characterized by comprising:
the grid image generation module is used for carrying out grid division on the acquired remote sensing image to be processed and generating a plurality of grid images corresponding to the remote sensing image to be processed;
a classified image obtaining module, configured to input each grid image into a pre-trained target scene classification network model, so as to obtain a grid classified image corresponding to each grid image;
the classified image marking module is used for executing edge extraction operation on the grid classified image and marking the grid classified image which is the edge according to an extraction result;
an extraction result image acquisition module, configured to input the labeled grid classified image into a pre-trained target pixel classification network model to acquire a pixel classification result image corresponding to the labeled grid classified image, and use the pixel classification result image as a town extraction result image of a corresponding region in a town extraction result;
the first image assignment module is used for assigning a value of 1 to a corresponding area in the town extraction result image aiming at the unmarked grid image judged as the town category by the target scene classification network model;
the second image assignment module is used for assigning 0 to a corresponding area in the town extraction result image aiming at the grid image which is judged to be in a non-town category by the target scene classification network model;
and the town binary image acquisition module is used for carrying out hole filling post-processing on the town extraction result image to obtain a town extraction binary image.
8. The apparatus of claim 7, further comprising:
the first remote sensing image acquisition module is used for acquiring a homologous town satellite remote sensing image and a non-town satellite remote sensing image;
the first cutting image acquisition module is used for cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cutting image corresponding to the town satellite remote sensing image and a second cutting image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
the first labeling image generating module is used for labeling the first clipping image and the second clipping image respectively according to the town scene category and the non-town scene category, and respectively generating a first labeling image corresponding to the first clipping image and a second labeling image corresponding to the second clipping image;
and the target classification model acquisition module is used for training an initial scene classification network model based on the first annotation image and the second annotation image to obtain the target scene classification network model.
9. The apparatus of claim 7, further comprising:
the second remote sensing image acquisition module is used for acquiring a homologous town satellite remote sensing image and a non-town satellite remote sensing image;
the second cutting image acquisition module is used for cutting the town satellite remote sensing image and the non-town satellite remote sensing image according to the size of NxN to respectively obtain a first cutting image corresponding to the town satellite remote sensing image and a second cutting image corresponding to the non-town satellite remote sensing image; wherein N is a positive integer and is an integral multiple of 2;
the second labeling image generating module is used for labeling each pixel in the first clipping image and the second clipping image in a town category and a non-town category, and respectively generating a first pixel labeling image corresponding to the first clipping image and a second pixel labeling image corresponding to the second clipping image;
and the target pixel classification model acquisition module is used for training an initial pixel classification network model based on the pixel annotation images of the town categories in the first pixel annotation image and the second pixel annotation image to obtain the target pixel classification network model.
10. The apparatus of claim 7, wherein the classified image labeling module comprises:
and the classified image marking unit is used for carrying out edge extraction on the classified images of the grids based on an image edge detection algorithm, marking the grid images identified as edges as E, and not marking the grid images without edges.
11. The apparatus of claim 10, wherein the extraction-result image obtaining module comprises:
the blank image establishing unit is used for establishing a blank image with the same size as the remote sensing image to be processed so as to store the town extraction result image;
a grid classified image judging unit, configured to perform condition judgment on each grid classified image in the grid classified images;
a first image assignment unit, configured to, if the classified grid image is of a town category and is marked as E, input the classified grid image into the target pixel classification network model to predict a pixel classification result thereof, and assign the pixel classification result image to the same region in the blank image;
a second image assignment unit, configured to assign all pixels in the same area in the blank image to 1 if the classified grid image is an image of a town category and is not marked;
and the third image assignment unit is used for assigning all pixels in the same area in the blank image as 0 if the classified grid image is of a non-town type, and finally, the town extraction result image is a 0-1 binary image.
12. The apparatus of claim 7, wherein the town binary map obtaining module comprises:
and the town binary image acquisition unit is used for performing post-processing on the town mosaic image by utilizing a hole filling algorithm in morphological operation, filling pixels which are located inside the town and have pixel values of 0, and obtaining the town extracted binary image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011024302.4A CN112215096B (en) | 2020-09-25 | 2020-09-25 | Remote sensing image town extraction method and device based on scene and pixel information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011024302.4A CN112215096B (en) | 2020-09-25 | 2020-09-25 | Remote sensing image town extraction method and device based on scene and pixel information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112215096A true CN112215096A (en) | 2021-01-12 |
| CN112215096B CN112215096B (en) | 2023-04-07 |
Family
ID=74051150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011024302.4A Active CN112215096B (en) | 2020-09-25 | 2020-09-25 | Remote sensing image town extraction method and device based on scene and pixel information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112215096B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766811A (en) * | 2018-12-31 | 2019-05-17 | 复旦大学 | The end-to-end detection and recognition methods of sea ship in a kind of satellite-borne SAR image |
| CN109858450A (en) * | 2019-02-12 | 2019-06-07 | 中国科学院遥感与数字地球研究所 | Ten meter level spatial resolution remote sensing image cities and towns extracting methods of one kind and system |
| CN110443143A (en) * | 2019-07-09 | 2019-11-12 | 武汉科技大学 | The remote sensing images scene classification method of multiple-limb convolutional neural networks fusion |
| CN110866494A (en) * | 2019-11-14 | 2020-03-06 | 三亚中科遥感研究所 | Optical remote sensing image-based town group extraction method and system |
-
2020
- 2020-09-25 CN CN202011024302.4A patent/CN112215096B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766811A (en) * | 2018-12-31 | 2019-05-17 | 复旦大学 | The end-to-end detection and recognition methods of sea ship in a kind of satellite-borne SAR image |
| CN109858450A (en) * | 2019-02-12 | 2019-06-07 | 中国科学院遥感与数字地球研究所 | Ten meter level spatial resolution remote sensing image cities and towns extracting methods of one kind and system |
| CN110443143A (en) * | 2019-07-09 | 2019-11-12 | 武汉科技大学 | The remote sensing images scene classification method of multiple-limb convolutional neural networks fusion |
| CN110866494A (en) * | 2019-11-14 | 2020-03-06 | 三亚中科遥感研究所 | Optical remote sensing image-based town group extraction method and system |
Non-Patent Citations (3)
| Title |
|---|
| DONGAO MA 等: "SiftingGAN: Generating and Sifting Labeled Samples to Improve the Remote Sensing Image Scene Classification Baseline In Vitro", 《IEEE GEOSCIENCE AND REMOTE SENSING LETTERS》 * |
| WEI ZHANG 等: "Remote Sensing Image Scene Classification Using CNN-CapsNet", 《REMOTE SENSING》 * |
| 史路璐 等: "一种面向土地覆盖分类的卷积神经网络模型", 《遥感信息》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112215096B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yang et al. | Building extraction at scale using convolutional neural network: Mapping of the united states | |
| CN111626947B (en) | Map vectorization sample enhancement method and system based on generation of countermeasure network | |
| CN110263717B (en) | Method for determining land utilization category of street view image | |
| CN109858450B (en) | Ten-meter-level spatial resolution remote sensing image town extraction method and system | |
| CN111028255B (en) | Farmland area pre-screening method and device based on priori information and deep learning | |
| CN111598101B (en) | Urban area intelligent extraction method, system and equipment based on remote sensing image scene segmentation | |
| CN108363951B (en) | Automatic acquisition method of deep learning sample library corresponding to remote sensing image land type identification | |
| CN113449594A (en) | Multilayer network combined remote sensing image ground semantic segmentation and area calculation method | |
| CN110334719B (en) | Method and system for extracting building image in remote sensing image | |
| CN112232328A (en) | Remote sensing image building area extraction method and device based on convolutional neural network | |
| CN112836614B (en) | High-resolution remote sensing image classification method based on residual error network and transfer learning | |
| CN112347970A (en) | Remote sensing image ground object identification method based on graph convolution neural network | |
| CN110781882A (en) | License plate positioning and identifying method based on YOLO model | |
| CN111666909A (en) | Suspected contaminated site space identification method based on object-oriented and deep learning | |
| CN113780175B (en) | Remote sensing identification method for typhoon and storm landslide in high vegetation coverage area | |
| CN113610024B (en) | Multi-strategy deep learning remote sensing image small target detection method | |
| CN113628180A (en) | Semantic segmentation network-based remote sensing building detection method and system | |
| CN114898089B (en) | Functional area extraction and classification method fusing high-resolution images and POI data | |
| CN111414855B (en) | Telegraph pole sign target detection and identification method based on end-to-end regression model | |
| CN114943902A (en) | Urban vegetation unmanned aerial vehicle remote sensing classification method based on multi-scale feature perception network | |
| Yan et al. | Estimation of urban-scale photovoltaic potential: A deep learning-based approach for constructing three-dimensional building models from optical remote sensing imagery | |
| CN116310628A (en) | Token mask mechanism-based large-scale village-in-city extraction method | |
| CN117727046A (en) | Novel mountain torrent front-end instrument and meter reading automatic identification method and system | |
| CN117315541B (en) | Ground garbage identification method and system | |
| CN112785610B (en) | Lane line semantic segmentation method integrating low-level features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |