CN110032963B - Dynamic monitoring method for spartina alterniflora new-born plaques - Google Patents

Dynamic monitoring method for spartina alterniflora new-born plaques Download PDF

Info

Publication number
CN110032963B
CN110032963B CN201910270990.3A CN201910270990A CN110032963B CN 110032963 B CN110032963 B CN 110032963B CN 201910270990 A CN201910270990 A CN 201910270990A CN 110032963 B CN110032963 B CN 110032963B
Authority
CN
China
Prior art keywords
image
spartina alterniflora
resolution
plaques
new
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201910270990.3A
Other languages
Chinese (zh)
Other versions
CN110032963A (en
Inventor
柯樱海
陈盟盟
李鹏
张双月
王明丽
吕明苑
周德民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Capital Normal University
Original Assignee
Capital Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Capital Normal University filed Critical Capital Normal University
Priority to CN201910270990.3A priority Critical patent/CN110032963B/en
Publication of CN110032963A publication Critical patent/CN110032963A/en
Application granted granted Critical
Publication of CN110032963B publication Critical patent/CN110032963B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/80Geometric correction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/62Analysis of geometric attributes of area, perimeter, diameter or volume
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/188Vegetation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10032Satellite or aerial image; Remote sensing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30181Earth observation
    • G06T2207/30188Vegetation; Agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a dynamic monitoring method of spartina alterniflora new plaques, belonging to the technical field of environmental monitoring; the method mainly aims to solve the problems that the river mouth wetland environment is complex, the field monitoring and measurement work of the spartina alterniflora regenerated patches is difficult to carry out, and the existing application is more that a middle-space resolution satellite can only meet the monitoring requirement of landscape scale due to the limitation of the space resolution, and the regenerated tiny patches cannot be identified; the dynamic change monitoring of the new patch is not facilitated by the single-platform high-resolution satellite due to the revisitation period and the weather limit of a river mouth, which is cloudy and rainy, the multiple-platform high-resolution satellite has different spatial resolutions and cannot be used for accurately identifying and measuring the new patch, and therefore a large-range non-contact dynamic monitoring method for the new patch of the spartina alterniflora based on the multiple-platform high-resolution satellite data fusion technology is provided.

Description

Dynamic monitoring method for spartina alterniflora new-born plaques
Technical Field
The invention relates to the technical field of environmental monitoring, in particular to a dynamic monitoring method for spartina alterniflora new plaques.
Background
The coastal wetland is positioned in an ecological staggered zone between a land ecosystem and a marine ecosystem, has rich biodiversity and provides an important ecological service function; meanwhile, due to the dual influence of nature and human, the coastal wetland is also a sensitive ecological fragile area, the spartina alterniflora is an important foreign species invasive species of the coastal wetland in China, and is listed in the first invasive species list published by the original national environmental protection bureau in 2003.
The diffusion of spartina alterniflora is divided into long-distance diffusion and short-distance diffusion, and generally, into three stages: firstly, the spartina alterniflora seeds successfully colonize to form new tiny plaques, and the plaques are not adjacent to each other; then, the tiny plaques are continuously expanded outwards in a large cake mode to form large plaques; finally, the plaque attaches in sheets that continue to expand. Researches show that necessary weed control measures are taken at the initial stage of planting and diffusing the spartina alterniflora, so that the prevention and control efficiency of spartina alterniflora invasion can be greatly improved. Therefore, the method has important effects on the efficient prevention and control of the spartina alterniflora.
For monitoring and measuring the new plaque, the dimension, area and perimeter of the small plaque are mostly measured by traditional field investigation. The method for field investigation is time-consuming and labor-consuming, and can only realize measurement of local plaques in a research area. In addition, the estuary wetland environment is complex, and is difficult to enter in many areas, thereby increasing the difficulty of field measurement.
The satellite remote sensing technology is gradually applied to monitoring landscape scale vegetation change by researchers due to the characteristics of multiple time phases and large range. At present, satellites with medium spatial resolution such as terrestrial satellites are widely applied, and due to the limitation of the spatial resolution, new small plaques cannot be identified, and the new small plaques can only be identified after the spartina alterniflora is spread into pieces. The high spatial resolution remote sensing data, especially the sub-meter resolution remote sensing data, provides possibility for the identification of new-born small plaques. Due to the revisiting period of a single satellite and the weather limit of a cloudy and rainy river mouth, the single satellite is not beneficial to monitoring the dynamic change of the newborn plaque. The joint use of multi-source, multi-platform high resolution satellite data may help solve this problem. The invention provides a large-range non-contact spartina alterniflora newborn plaque dynamic monitoring method based on a multi-platform high-resolution satellite data fusion technology, which aims to solve the defects that high-resolution satellite data mostly have different spatial resolutions, and the errors and uncertainty of plaque identification and monitoring can be increased by directly using the data.
Disclosure of Invention
The invention aims to provide a dynamic monitoring method of spartina alterniflora new plaques, which aims to solve the problems proposed in the background technology:
(1) the estuary wetland environment is complex and is difficult to access in many places. The field monitoring and measurement of the spartina alterniflora new plaque are time-consuming and labor-consuming, and the distribution condition of the new plaque is difficult to know integrally;
(2) the method is mainly characterized in that a medium-spatial-resolution satellite can only meet the monitoring requirement of landscape scale due to the limitation of spatial resolution, and can not identify new small plaques;
(3) the single-platform high-resolution satellite is not beneficial to monitoring the dynamic change of the newborn plaque due to the revisiting period and the weather limit of a cloudy and rainy river mouth;
(4) the multi-platform high-resolution satellite has different spatial resolutions, and can not obtain accurate identification and measurement of new plaques by direct respective use.
In order to achieve the purpose, the invention adopts the following technical scheme:
the dynamic monitoring method for spartina alterniflora new-born plaques is characterized by comprising the following steps:
s1, identifying potential planting areas of new plaques: utilizing the medium-resolution remote sensing satellite image, according to a visual interpretation method, fully considering time phase difference of the spartina alterniflora and other vegetation, identifying the range boundary of the spartina alterniflora from sea to land, and combining tide level data to obtain a tidal zone range, so that a tidal zone area without identifying the range boundary of the spartina alterniflora is used as a potential planting area of a new patch;
s2, multi-source multi-temporal high spatial resolution satellite image acquisition: collecting GF-1, GF-2, SPOT-6 or WorldView-2 images of the same growing season month covering the potential planting area for many years, and ensuring that one image exists in the growing season every year;
s3, preprocessing the multi-source multi-temporal high-spatial-resolution satellite image: carrying out radiometric calibration, atmospheric correction, image fusion, registration, embedding and cutting pretreatment on the multi-source multi-temporal high-spatial-resolution satellite image;
s4, super-resolution reconstruction of the processed satellite image: performing super-resolution reconstruction on high resolution images with different resolutions by using an FSRCNN model deep learning method;
s5, identification of spartina alterniflora new plaques: performing primary image pre-segmentation by adopting a watershed segmentation algorithm of pixel gray values to obtain a primary segmentation unit; then, the spectrum and the shape characteristics are coupled, and the pattern and the spot are combined to complete the overall segmentation so as to achieve the aim of identifying the spartina alterniflora patch;
s6, extracting and measuring the spartina alterniflora new-growing plaque: superposing the segmented result image on a 0.5m image, and extracting the spartina alterniflora new patch in the first step through visual interpretation; secondly, four adjacent coordinates of the minimum outer-wrapping rectangle of each image spot are calculated in the ARCGIS; summarizing the north-south and east-west distances of each image spot through the attribute table; finally, the area and the perimeter of each image spot are obtained through the calculation geometry of the attribute table;
s7, field measurement and result correction: selecting identified patches from the extraction result graph, dividing the identified patches into 5 categories according to the patch area from large to small, selecting 10 patches which are easy to reach in the field from each category, using a method of GPS positioning and visual interpretation of high-definition images to go to the field, positioning each patch, and measuring the area, the perimeter, the east-west-south-north distance, the shape, the height, the number of plants, the coverage, the phenological period, the vitality and the attributes of the surrounding environment of each patch; and (3) performing linear fitting by taking the field measured value as X and the plaque measured value in the satellite image extraction result graph as Y to obtain a fitting equation:
Y=a*X+b
correcting the satellite measurement values of all plaques by using a fitting equation;
s8, Merremia crocea newborn plaque expansion dynamic measurement: and (3) executing the steps S1-S7 by using the multi-temporal multi-platform high-spatial-resolution image to obtain dynamic measurement of the dilation of the spartina alterniflora neopatch, and performing timely dynamic monitoring on the diffusion mode of the early-identified spartina alterniflora neopatch by using the multi-temporal accurate measurement value of the spartina alterniflora neopatch obtained in the steps S1-S7 through a statistical analysis method.
Preferably, the image preprocessing procedure mentioned in S3 is as follows:
1) radiometric calibration: performing radiometric calibration on the multispectral data and the panchromatic data according to an Apply Gain and Offset tool provided by ENVI;
2) and atmospheric correction: carrying out atmospheric correction on the high-resolution multispectral image by adopting a FLAASH atmospheric correction module in ENVI software;
3) and orthorectification: correcting image space distortion caused by elevation fluctuation of an image by using DEM data to generate a multi-center projection plane orthorectified image;
4) and image fusion: respectively fusing the multi-source multi-temporal high spatial resolution satellite images by adopting a Gram-Schmidt method to obtain fused multi-source multi-temporal high spatial resolution satellite images;
5) and image registration: selecting 20-25 ground control points by taking the corrected remote sensing image in the area as a reference image, and correcting by adopting a quadratic polynomial resampling technology, wherein the total error is controlled within 0.5 pixel;
6) and image cutting: and cutting the corrected image by means of vector data of the potential planting area of the existing new plaque to obtain a preprocessed image.
Preferably, the FSRCNN model mentioned in S4 is divided into five layers, which are a feature extraction layer, a contraction layer, a mapping layer, an expansion layer and a reconstruction layer, respectively, and the first four layers are convolution processing for extracting low-dimensional data from high-dimensional data; and the last part of the reconstruction layer is deconvolution processing for mapping low-dimensional data to high-dimensional output, and Conv (filter size, filter number, channel number) represents convolution processing and Deconv (filter size, filter number, channel number) represents deconvolution processing respectively.
Preferably, the FSRCNN model uses the function of the pri LU as the activation function after each convolutional layer, and the function of the pri LU is shown as the following formula:
Figure GDA0002683070830000051
reducing a loss value between the reconstructed image and the target image by using a reverse error propagation algorithm, obtaining a weight parameter of the network after training is finished, and adopting a mean square error as a loss function; the quality of an image after reconstruction compared with an original image is evaluated quantitatively by calculating the peak signal-to-noise ratio, and the higher the peak signal-to-noise ratio is, the smaller the distortion after compression is; the peak signal-to-noise ratio is defined by the mean square error, and the two mxn monochromatic images I and K, if one is a noise approximation of the other, have their mean square error defined as:
Figure GDA0002683070830000061
the peak signal-to-noise mathematical formula is as follows:
Figure GDA0002683070830000062
wherein, MAXIIs the maximum value representing the color of the image point.
Preferably, the identification of the spartina alterniflora new patch mentioned in S5 is further combined with the color and shape features represented by the high-resolution image to obtain a final segmentation result, and the step S5 is implemented by a watershed algorithm and a fast patch combination technique, and the specific technical steps are as follows:
i, obtaining a sub-element: firstly, reading a color image, converting the color image into a gray image, and then carrying out primary segmentation on the image through a watershed segmentation algorithm to obtain a sub-primitive;
II, cost criterion of pattern spot combination: adopting a merging cost criterion function, wherein the function consists of a spectral heterogeneity parameter and a shape heterogeneity parameter of a merged image spot:
f=w×hcolor+(1-w)×hshape
wherein w is the weight assigned by the heterogeneity of spectrum and shape, and the interval is [0, 1 ]],hcolorHue weight sum hshapeShape weight, interval is (0, 1);
the spectral heterogeneity is the difference between the standard deviation of the parent pattern spot after combination and the sum of the standard deviations of the two sub-pattern spots before combination, and is weighted according to the area:
Figure GDA0002683070830000071
where c is the total number of bands, wcThe weights assigned by the user to each band are used to calculate the spectral difference of the combined patch in the multi-band image.
Shape heterogeneity is in turn composed of two part weighting of compactness heterogeneity and smoothness heterogeneity:
hshape=wcmpct×hcmpct+(1-wcmpct)×hsmooth
the compactness difference is then calculated by the following formula:
Figure GDA0002683070830000072
the smoothness difference is calculated by the following formula:
Figure GDA0002683070830000073
in the above formula, l is the actual perimeter of the object, n is the number of pixels of the object, and b is the perimeter of the circumscribed rectangle of the object; the weight occupied by the compactness and the smoothness is set and adjusted by a user;
it is obviously inefficient to repeatedly calculate the standard deviation of the parent pattern spot in the merging process, and therefore the following fast calculation method of the standard deviation of the merged sample is adopted:
Figure GDA0002683070830000074
m in the above formula1、m2Respectively the mean values of the two plaques;
and III, rapidly combining the image spots, and controlling the whole combining process by adopting a scale parameter on the basis of determining conditions of initial small image spots and combining cost functions, wherein the essence of the parameter is a combining cost threshold value.
Compared with the prior art, the invention provides a dynamic monitoring method for spartina alterniflora new plaques, which has the following beneficial effects:
(1) compared with the traditional field sampling method, the method is time-saving and labor-saving, and can be used for summarizing the large-scale anagenesis plaques of the spartina alterniflora in the coastal wetland; the method has the advantages of non-contact, and avoids the situation that people cannot enter the method and cannot know the initial diffusion of the spartina alterniflora due to complex environment; the method has the advantages that the accurate measurement of the spartina alterniflora new plaque through the multisource multi-temporal satellite data can accurately capture the situation of the spartina alterniflora diffusion initial stage in a short time, and the prevention and control efficiency of the spartina alterniflora is improved;
(2) compared with the medium-resolution satellite remote sensing technology, the high-resolution satellite remote sensing technology is adopted, the spatial resolution and the revisit period are greatly improved, and the possibility of extracting the tiny patches is provided;
(3) the method adopts the deep learning super-resolution reconstruction to fuse the data with multiple platforms and different resolutions, greatly reduces the error and uncertainty of the result caused by different resolutions of the data, has higher precision, can accurately reflect the fine change of the fine patch, and is more suitable for the early monitoring of the rapidly diffused spartina alterniflora.
Drawings
FIG. 1 is a general flow chart of a dynamic monitoring method for anagenesis plaques of Spartina alterniflora;
FIG. 2 is a multi-temporal multi-source high-resolution satellite image preprocessing flow chart of the dynamic monitoring method for the spartina alterniflora new patch provided by the invention;
FIG. 3 is a diagram of the FSRCNN model structure of the dynamic monitoring method for the anagenesis plaques of spartina alterniflora;
FIG. 4 is a flow chart of a watershed segmentation-based multi-scale segmentation method for dynamically monitoring anagen plaques of spartina alterniflora;
FIG. 5 is a flow chart of field measurement and result correction of the dynamic monitoring method for spartina alterniflora new plaque proposed by the present invention;
fig. 6 is a schematic structural diagram of an expanding mode of a spartina alterniflora neogenesis plaque of the dynamic monitoring method of spartina alterniflora neogenesis plaque provided by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
In the description of the present invention, it is to be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention.
Example 1:
please refer to fig. 1;
the dynamic monitoring method of spartina alterniflora new-born plaques comprises the following steps:
s1, identifying potential planting areas of new plaques: utilizing the medium-resolution remote sensing satellite image, according to a visual interpretation method, fully considering time phase difference of the spartina alterniflora and other vegetation, identifying the range boundary of the spartina alterniflora from sea to land, and combining tide level data to obtain a tidal zone range, so that a tidal zone area without identifying the range boundary of the spartina alterniflora is used as a potential planting area of a new patch;
s2, multi-source multi-temporal high spatial resolution satellite image acquisition: collecting GF-1, GF-2, SPOT-6 or WorldView-2 images of the same growing season month covering the potential planting area for many years, and ensuring that one image exists in the growing season for each year;
s3, preprocessing the multi-source multi-temporal high-spatial-resolution satellite image: carrying out radiometric calibration, atmospheric correction, image fusion, registration, embedding and cutting pretreatment on the multi-temporal multi-source high-resolution satellite image;
s4, super-resolution reconstruction of the processed satellite image: performing super-resolution reconstruction on high resolution images with different resolutions by using an FSRCNN model deep learning method;
s5, identification of spartina alterniflora new plaques: performing image pre-segmentation once by using a watershed segmentation algorithm of pixel gray values to obtain a primary segmentation unit (which can be called a sub-primitive because the primary segmentation unit is not the final segmentation result); then, the spectrum and the shape characteristics are coupled, and the pattern and the spot are combined to complete the overall segmentation so as to achieve the aim of identifying the spartina alterniflora patch;
s6, extracting and measuring the spartina alterniflora new-growing plaque: superposing the segmented result image on a 0.5m image, and extracting the spartina alterniflora new patch in the first step through visual interpretation; secondly, calculating the four-adjacent coordinates of the minimum outer-wrapping rectangle of each image spot in an ARCGIS (geographic information System series software); summarizing the north-south and east-west distances of each image spot through the attribute table; finally, the area and the perimeter of each image spot are obtained through the calculation geometry of the attribute table;
s7, field measurement and result correction: selecting identified plaques from the extraction result graph, dividing the plaques into 5 categories according to the area of the plaques, selecting 10 plaques which are easy to reach in the field from large to small, using a GPS positioning and visual interpretation method to locate the plaques in the field, positioning each plaque, and measuring the area, the circumference, the distance in the south and north directions, the shape, the height, the number of plants, the coverage, the phenological period (the vegetative period, the bud period, the flowering period, the fructification period and the dormancy period), the vitality (strong, medium and weak) and the attributes of the surrounding environment of each plaque. And (3) performing linear fitting by taking the field measured value as X and the plaque measured value in the satellite image extraction result graph as Y to obtain a fitting equation:
Y=a*X+b
correcting the satellite measurement values of all plaques by using a fitting formula;
s8, Merremia crocea newborn plaque expansion dynamic measurement: and (3) executing the steps S1-S7 by using the multi-temporal multi-platform high-spatial-resolution image to obtain dynamic measurement of the dilation of the spartina alterniflora neopatch, and performing timely dynamic monitoring on the diffusion mode of the early-identified spartina alterniflora neopatch by using the multi-temporal accurate measurement value of the spartina alterniflora neopatch obtained in the steps S1-S7 through a statistical analysis method.
Compared with the traditional field sampling method, the method is time-saving and labor-saving, and can be used for summarizing the large-scale anagenesis plaques of the spartina alterniflora in the coastal wetland; the method has the advantages of non-contact, and avoids the situation that people cannot enter the method and cannot know the initial diffusion of the spartina alterniflora due to complex environment; the method has the advantages that the accurate measurement of the spartina alterniflora new plaque through the multisource multi-temporal satellite data can accurately capture the situation of the spartina alterniflora diffusion initial stage in a short time, and the prevention and control efficiency of the spartina alterniflora is improved; compared with the medium-resolution satellite remote sensing technology, the high-resolution satellite remote sensing technology is adopted, the spatial resolution and the revisit period are greatly improved, and the possibility of extracting the tiny patches is provided; in addition, the invention adopts the deep learning super-resolution reconstruction to fuse the data with multiple platforms and different resolutions, thereby greatly reducing the error and uncertainty of the result caused by different resolutions of the data, having higher precision, being capable of accurately reflecting the fine change of the fine patch and being more suitable for the early monitoring of the rapidly diffused spartina alterniflora.
Example 2:
referring to fig. 2, the embodiment 1 is different from the above embodiments;
the image data preprocessing flow mentioned in S3 is:
1) radiometric calibration: carrying out radiometric calibration on multispectral data and panchromatic data according to an Apply Gain and offset tool provided by an ENVI (integrated remote sensing image processing platform);
2) and atmospheric correction: carrying out atmospheric correction on the high-resolution multispectral image by adopting a FLAASH atmospheric correction module in ENVI (integrated remote sensing image processing platform) software;
3) and orthorectification: correcting image space distortion caused by elevation fluctuation of an image by using DEM data to generate a multi-center projection plane orthorectified image;
4) and image fusion: respectively fusing multi-source multi-temporal high-spatial-resolution satellite images by adopting a Gram-Schmidt method to obtain fused multi-temporal multi-source high-resolution satellite images;
5) and image registration: selecting 20-25 ground control points by taking the corrected remote sensing image in the area as a reference image, and correcting by adopting a quadratic polynomial resampling technology, wherein the total error is controlled within 0.5 pixel;
6) and image cutting: and cutting the corrected image by means of vector data of the potential planting area of the existing new plaque to obtain a preprocessed image.
Utilizing medium-resolution remote sensing satellite images (such as a land satellite and a sentry-2 satellite), according to a visual interpretation method, fully considering time phase difference of the spartina alterniflora and other local vegetation and combining multi-time phase images (images in 10 months later are mainly selected, the reed of the local vegetation enters a decay period and the spartina alterniflora is in a mature period), identifying the range boundary of the spartina alterniflora from sea to land in 2013, and combining tide level data to obtain a tidal zone range, so that the tidal zone region without identifying the range boundary of the spartina alterniflora is used as a potential planting region of a new patch; selecting a typical spartina alterniflora newborn patch potential planting area of the coastal wetland of the yellow river delta, and collecting GF-1(2m resolution), GF-2(1m resolution) and SPOT-6(1.5m resolution) images of the same growing season month (such as 7-9 months) for many years (such as nearly five years) covering the potential planting area, thereby ensuring that one image exists in the growing season every year.
The data are shown in table 1.
Figure GDA0002683070830000131
Then, the multi-temporal multi-source high-resolution satellite image is subjected to radiometric calibration, atmospheric correction, image fusion, registration, mosaic and cutting preprocessing, and the preprocessed image is obtained by referring to the image data preprocessing flow, as shown in fig. 2
Example 3:
referring to fig. 3, the embodiment 1 or 2 is different from the above embodiments;
the FSRCNN model mentioned in S4 is divided into five layers, namely a feature extraction layer, a contraction layer, a mapping layer, an expansion layer and a reconstruction layer, wherein the former four layers are convolution processing and used for extracting low-dimensional data from high-dimensional data; and the last part of reconstruction layer is deconvolution processing, which is used for mapping low-dimensional data into high-dimensional output, and Conv (fi, ni, ci) is used for representing convolution processing, and Deconv (fi, ni, ci) is used for representing deconvolution processing, wherein variables fi, ni, ci respectively represent the size of the filter, the number of the filters and the number of channels.
In the step, high resolution images (table 1) with different resolutions are unified to 0.5m resolution by a super-resolution reconstruction method. The method uses the FSRCNN deep learning method to carry out super-resolution reconstruction. First, 2-fold, 3-fold and 4-fold FSRCNN models were trained. Next, performing FSRCNN test: a Set 5 image data Set is selected as test data, after model parameters are selected, reconstruction is carried out on a Set 5 image by respectively adopting three algorithms of bicubic interpolation, SRCNN and FSRCNN under the conditions of 2-time amplification, 3-time amplification and 4-time amplification, objective evaluation indexes are given, meanwhile, local amplification is carried out so as to compare reconstruction quality more intuitively, and finally, an image after image enhancement conforming to human vision is given. The FSRCNN model has significant advantages in reconstruction quality both from objective evaluation criteria and from subjective vision. Finally realizing 4 times of super-resolution reconstruction on GF-1 with the length of 2m, and amplifying the image resolution by 4 times; the resolution of the GF-2 image with the same 1m is amplified by 2 times; the SPOT-6 image resolution of 1.5m is magnified by a factor of 3. All collected high resolution images with different resolutions are unified to 0.5m resolution.
Example 4:
referring to fig. 3, the difference between any of embodiments 1-3 is that;
the FSRCNN network model uses the PRe LU function as the activation function after each convolutional layer, which is shown as the following formula:
Figure GDA0002683070830000141
reducing a loss value between the reconstructed image and the target image by using a reverse error propagation algorithm, obtaining a weight parameter of the network after training is finished, and adopting a Mean Square Error (MSE) as a loss function; and quantitatively evaluating the quality of the reconstructed image of one image compared with the original image by calculating the peak signal-to-noise ratio (PSNR), wherein the higher the peak signal-to-noise ratio is, the smaller the distortion after compression is. The peak signal-to-noise ratio is often defined simply by the Mean Square Error (MSE), the mean square error of two mxn monochromatic images I and K, if one is a noisy approximation of the other, being defined as:
Figure GDA0002683070830000151
the peak signal-to-noise mathematical formula is as follows:
Figure GDA0002683070830000152
wherein, MAXIIs the maximum value representing the color of the image point.
Firstly, carrying out FRCNN model training: the present invention uses the 91-image and General-100 image datasets as training images and the Set 5 image dataset was selected as the test image. These images are subjected to data enhancement as high-resolution images, and the high-resolution grayscale images are subjected to scaling processing of 2, 3 times or 4 times in size, respectively, and these scaled images are referred to herein as low-resolution images. The method for training the convolutional neural network trains the convolutional neural network through Caffe, and low-resolution/high-resolution images need to be respectively subjected to amplification conditions of 2 times, 3 times and 4 times according to 102/192,72/192And 62/212And (5) partitioning. Deconvolution layer initialization settings were of gaussian type and 0.001 standard deviation value. At the beginningWhen training the convolutional neural network, only the 91-image data set is used for training, and the initial learning rate of the convolutional layer is set to be 0.001 and the learning rate of the deconvolution layer is 0.0001. After the network is close to saturation, 91-image and General-100 enhanced images are used as training data, at which time the learning rate of the convolutional layer needs to be changed to 0.0005 and the deconvolution layer to 0.00005. In order to test the property of the FSRCNN structure, a group of control experiments are designed, under the condition that a fixed amplification factor is 3, different values are respectively taken for three sensitive variables, namely a low-resolution image feature dimension d, the number s of shrinkage filters and a mapping depth m, and d is specifically selected to be 48 and 56; s-12, 16 and m-2, 3, 4, so we performed 12 different combinations of experiments in total. Finally, it was found that the performance and parameters in FSRCNN (56,12,4) were optimally balanced, with PSNR achieving one of the highest results. We select FSRCNN (56,12,4) as the default network. The training process is carried out under the condition that the amplification factor is 3, namely the training result of the 3-time model is obtained, and now only the deconvolution network is finely adjusted to train the x-time amplification factor to obtain the training result of the x-time model.
Secondly, performing FSRCNN model test: a Set 5 image data Set is selected as test data, after model parameters are selected, the Set 5 image is respectively reconstructed by adopting three algorithms of bicubic interpolation, SRCNN and FSRCNN under the condition of x-time amplification, objective evaluation indexes are given, reconstruction quality is compared visually from vision, and finally an image after image enhancement conforming to human vision is given. The FSRCNN model has significant advantages in reconstruction quality both from objective evaluation criteria and from subjective vision.
Example 5:
referring to FIGS. 4-6, the embodiment 1-4 is different from the embodiment shown in FIGS;
the identification of the spartina alterniflora new patch mentioned in S5, the present invention also combines the characteristics of color, shape, etc. reflected by the high resolution image to obtain the final segmentation result, the above step S5 is realized by watershed algorithm and fast pattern patch combination technology, the specific technical steps are:
i, obtaining a sub-element: firstly, reading a color image, converting the color image into a gray image, and then carrying out primary segmentation on the image through a watershed segmentation algorithm to obtain a sub-primitive;
II, cost criterion of pattern spot combination: adopting a merging cost criterion function, wherein the function consists of a spectral heterogeneity parameter and a shape heterogeneity parameter of a merged image spot:
f=w×hcolor+(1-w)×hshape
where w is the weight assigned to spectral and shape heterogeneity, and the interval is [0, 1 ]],hcolorHue weight sum hshapeShape weight, interval is (0, 1);
the spectral heterogeneity is the difference between the standard deviation of the parent pattern spot after combination and the sum of the standard deviations of the two sub-pattern spots before combination, and is weighted according to the area:
Figure GDA0002683070830000171
where c is the total number of bands, wcThe weights assigned by the user to each band (default to 1.0) are used to calculate the spectral difference of the combined patch in the multi-band image.
Shape heterogeneity is in turn composed of two part weighting of compactness heterogeneity and smoothness heterogeneity:
hshape=wcmpct×hcmpct+(1-wcmpct)×hsmooth
the compactness difference is then calculated by the following formula:
Figure GDA0002683070830000172
the smoothness difference is calculated by the following formula:
Figure GDA0002683070830000173
in the above formula, l is the actual perimeter of the object, n is the number of pixels of the object, and b is the perimeter of the circumscribed rectangle of the object. The weight occupied by the compactness and the smoothness is set and adjusted by a user.
It is obviously inefficient to repeatedly calculate the standard deviation of the parent pattern spot in the merging process, and therefore the following fast calculation method of the standard deviation of the merged sample is adopted:
Figure GDA0002683070830000181
m in the above formula1、m2Respectively the mean values of the two plaques;
and III, rapidly combining the image spots, and controlling the whole combining process by adopting a scale parameter on the basis of determining conditions such as initial small image spots, a combining cost function and the like, wherein the essence of the parameter is a combining cost threshold value.
If the merging cost of all the image spots and the neighbors to be merged in a certain merging process is greater than the square of the scale parameter, the whole merging process is ended, and the image segmentation is completed; otherwise, after all the image spots are combined, various features and topological relations of the new image spots need to be recalculated, and the new image spots are combined again (see fig. 3), until the condition is met, the algorithm can be terminated, and the segmentation is completed.
The process of extraction and measurement is done in ArcMap 10.2. Firstly, superposing a segmented result image on a 0.5m image, and extracting the spartina alterniflora new patch in the first step through visual interpretation; secondly, four adjacent coordinates (namely xmax, xmin, ymin, ymax) of the minimum outer rectangle of each image spot are calculated in the ARCGIS 10.2; summarizing the north-south distance and the east-west distance of each image spot through an attribute table (the north-south distance is ymax-ymin, and the east-west distance is xmax-xmin); and finally, obtaining the area and the perimeter of each image spot through the calculation geometry of the attribute table.
Example 6:
the difference between the embodiments 1 to 5 is that:
the invention uses multi-temporal multi-platform high spatial resolution images to execute the steps S1-S7, and obtains the dynamic measurement of the dilation of the new plaques of the spartina alterniflora. Firstly, extracting a spartina alterniflora distribution map in a research area by using a land satellite, and obtaining a potential planting area of a new patch by combining tide data. And secondly, collecting multi-platform high-spatial-resolution images (at least one image exists every year) of the spartina alterniflora growing season in the research area for many years (such as nearly five years), and carrying out preprocessing such as fusion, registration and the like to obtain images used for the third step after preprocessing. And thirdly, obtaining an x-time model through the training of the FSRCNN model network, and unifying the multi-period images preprocessed in the second step to 0.5m resolution by adopting a proper x-time model through a super-resolution reconstruction method. And fourthly, reflecting basic characteristics such as color, shape and the like by combining a watershed segmentation algorithm with a high-resolution image, segmenting the multi-period image of 0.5m obtained in the third step to obtain the spartina alterniflora patch, wherein the hue weight is 0.7, the shape is 0.3, the segmentation result and the visual segmentation effect are greatly different due to overlarge shape weight, the weights of compactness and smoothness are generally 0.5 and are suitable, and the scale parameter is generally set to be 50. The rest of the work is finished in ArcGis, and the identified spartina alterniflora patch result is extracted and related attribute values including area, perimeter, south-north direction and east-west direction distances are obtained. And fifthly, performing linear fitting on the field measured value X and the plaque measured value Y extracted in the first four steps to obtain a fitting formula Y which is a X X + b, and correcting the satellite measured values of all the years by using the fitting formula. Finally, accurate measurement values of the anagen plaques of the spartina alterniflora in multiple periods are obtained through the first five steps, the measurement values are analyzed through statistics and the like, the diffusion mode of the early-identified anagen plaques of the spartina alterniflora is dynamically monitored in time, and the method plays an important role in efficient prevention and control of the spartina alterniflora
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be able to cover the technical scope of the present invention and the equivalent alternatives or modifications according to the technical solution and the inventive concept of the present invention within the technical scope of the present invention.

Claims (5)

1. The dynamic monitoring method for spartina alterniflora new-born plaques is characterized by comprising the following steps:
s1, identifying potential planting areas of new plaques: utilizing the medium-resolution remote sensing satellite image, according to a visual interpretation method, fully considering time phase difference of the spartina alterniflora and other vegetation, identifying the range boundary of the spartina alterniflora from sea to land, and combining tide level data to obtain a tidal zone range, so that a tidal zone area without identifying the range boundary of the spartina alterniflora is used as a potential planting area of a new patch;
s2, multi-source multi-temporal high spatial resolution satellite image acquisition: collecting GF-1, GF-2, SPOT-6 or WorldView-2 images of the same growing season month covering the potential planting area for many years, and ensuring that one image exists in the growing season every year;
s3, preprocessing the multi-source multi-temporal high-spatial-resolution satellite image: carrying out radiometric calibration, atmospheric correction, image fusion, registration, embedding and cutting pretreatment on the multi-source multi-temporal high-spatial-resolution satellite image;
s4, super-resolution reconstruction of the processed satellite image: performing super-resolution reconstruction on high resolution images with different resolutions by using an FSRCNN model deep learning method;
s5, identification of spartina alterniflora new plaques: performing primary image pre-segmentation by adopting a watershed segmentation algorithm of pixel gray values to obtain a primary segmentation unit; then, the spectrum and the shape characteristics are coupled, and the pattern and the spot are combined to complete the overall segmentation so as to achieve the aim of identifying the spartina alterniflora patch;
s6, extracting and measuring the spartina alterniflora new-growing plaque: superposing the segmented result image on a 0.5m image, and extracting the spartina alterniflora new patch in the first step through visual interpretation; secondly, four adjacent coordinates of the minimum outer-wrapping rectangle of each image spot are calculated in the ARCGIS; summarizing the north-south and east-west distances of each image spot through the attribute table; finally, the area and the perimeter of each image spot are obtained through the calculation geometry of the attribute table;
s7, field measurement and result correction: selecting identified patches from the extraction result graph, dividing the identified patches into 5 categories according to the patch area from large to small, selecting 10 patches which are easy to reach in the field from each category, using a method of GPS positioning and visual interpretation of high-definition images to go to the field, positioning each patch, and measuring the area, the perimeter, the east-west-south-north distance, the shape, the height, the number of plants, the coverage, the phenological period, the vitality and the attributes of the surrounding environment of each patch; and (3) performing linear fitting by taking the field measured value as X and the plaque measured value in the satellite image extraction result graph as Y to obtain a fitting equation:
Y=a*X+b
correcting the satellite measurement values of all plaques by using a fitting equation;
s8, Merremia crocea newborn plaque expansion dynamic measurement: and (3) executing the steps S1-S7 by using the multi-temporal multi-platform high-spatial-resolution image to obtain dynamic measurement of the dilation of the spartina alterniflora neopatch, and performing timely dynamic monitoring on the diffusion mode of the early-identified spartina alterniflora neopatch by using the multi-temporal accurate measurement value of the spartina alterniflora neopatch obtained in the steps S1-S7 through a statistical analysis method.
2. The method for dynamically monitoring anabolic plaques of spartina alterniflora according to claim 1, wherein: the image preprocessing procedure mentioned in S3 is as follows:
1) radiometric calibration: performing radiometric calibration on the multispectral data and the panchromatic data according to an Apply Gain and Offset tool provided by ENVI;
2) and atmospheric correction: carrying out atmospheric correction on the high-resolution multispectral image by adopting a FLAASH atmospheric correction module in ENVI software;
3) and orthorectification: correcting image space distortion caused by elevation fluctuation of an image by using DEM data to generate a multi-center projection plane orthorectified image;
4) and image fusion: respectively fusing the multi-source multi-temporal high spatial resolution satellite images by adopting a Gram-Schmidt method to obtain fused multi-source multi-temporal high spatial resolution satellite images;
5) and image registration: selecting 20-25 ground control points by taking the corrected remote sensing image in the area as a reference image, and correcting by adopting a quadratic polynomial resampling technology, wherein the total error is controlled within 0.5 pixel;
6) and image cutting: and cutting the corrected image by means of vector data of the potential planting area of the existing new plaque to obtain a preprocessed image.
3. The method for dynamically monitoring anabolic plaques of spartina alterniflora according to claim 1, wherein: the FSRCNN model mentioned in the S4 is divided into five layers which are respectively a feature extraction layer, a contraction layer, a mapping layer, an expansion layer and a reconstruction layer, wherein the former four layers are convolution processing and are used for extracting low-dimensional data from high-dimensional data; and the last part of the reconstruction layer is deconvolution processing for mapping low-dimensional data to high-dimensional output, and Conv (filter size, filter number, channel number) represents convolution processing and Deconv (filter size, filter number, channel number) represents deconvolution processing respectively.
4. The method for dynamically monitoring anabolic plaques of spartina alterniflora according to claim 1 or 3, wherein: the FSRCNN model uses the PRe LU function as the activation function after each convolutional layer, which is shown as the following equation:
Figure FDA0002683070820000031
reducing a loss value between the reconstructed image and the target image by using a reverse error propagation algorithm, obtaining a weight parameter of the network after training is finished, and adopting a mean square error as a loss function; the quality of an image after reconstruction compared with an original image is evaluated quantitatively by calculating the peak signal-to-noise ratio, and the higher the peak signal-to-noise ratio is, the smaller the distortion after compression is; the peak signal-to-noise ratio is defined by the mean square error, and the two mxn monochromatic images I and K, if one is a noise approximation of the other, have their mean square error defined as:
Figure FDA0002683070820000041
the peak signal-to-noise mathematical formula is as follows:
Figure FDA0002683070820000042
wherein, MAXIIs the maximum value representing the color of the image point.
5. The method for dynamically monitoring anabolic plaques of spartina alterniflora according to claim 1, wherein: the identification of the spartina alterniflora new patch mentioned in S5 is combined with the color and shape characteristics represented by the high-resolution image to obtain a final segmentation result, and the step S5 is implemented by a watershed algorithm and a fast pattern patch combination technology, and specifically includes the following steps:
i, obtaining a sub-element: firstly, reading a color image, converting the color image into a gray image, and then carrying out primary segmentation on the image through a watershed segmentation algorithm to obtain a sub-primitive;
II, cost criterion of pattern spot combination: adopting a merging cost criterion function, wherein the function consists of a spectral heterogeneity parameter and a shape heterogeneity parameter of a merged image spot:
f=w×hcolor+(1-w)×hshape
wherein w is the weight assigned by the heterogeneity of spectrum and shape, and the interval is [0, 1 ]],hcolorHue weight sum hshapeShape weight, interval is (0, 1);
the spectral heterogeneity is the difference between the standard deviation of the parent pattern spot after combination and the sum of the standard deviations of the two sub-pattern spots before combination, and is weighted according to the area:
Figure FDA0002683070820000051
where c is the total number of bands, wcThe weights of all the wave bands are distributed by the user, so that the spectral difference of the plaque combination in the multi-band image is calculated;
shape heterogeneity is in turn composed of two part weighting of compactness heterogeneity and smoothness heterogeneity:
hshape=wcmpct×hcmpct+(1-wcmpct)×hsmooth
the compactness difference is then calculated by the following formula:
Figure FDA0002683070820000052
the smoothness difference is calculated by the following formula:
Figure FDA0002683070820000053
in the above formula, l is the actual perimeter of the object, n is the number of pixels of the object, and b is the perimeter of the circumscribed rectangle of the object; the weight occupied by the compactness and the smoothness is set and adjusted by a user;
it is obviously inefficient to repeatedly calculate the standard deviation of the parent pattern spot in the merging process, and therefore the following fast calculation method of the standard deviation of the merged sample is adopted:
Figure FDA0002683070820000054
m in the above formula1、m2Respectively the mean values of the two plaques;
and III, rapidly combining the image spots, and controlling the whole combining process by adopting a scale parameter on the basis of determining conditions of initial small image spots and combining cost functions, wherein the essence of the parameter is a combining cost threshold value.
CN201910270990.3A 2019-04-04 2019-04-04 Dynamic monitoring method for spartina alterniflora new-born plaques Expired - Fee Related CN110032963B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910270990.3A CN110032963B (en) 2019-04-04 2019-04-04 Dynamic monitoring method for spartina alterniflora new-born plaques

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910270990.3A CN110032963B (en) 2019-04-04 2019-04-04 Dynamic monitoring method for spartina alterniflora new-born plaques

Publications (2)

Publication Number Publication Date
CN110032963A CN110032963A (en) 2019-07-19
CN110032963B true CN110032963B (en) 2020-10-30

Family

ID=67237546

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910270990.3A Expired - Fee Related CN110032963B (en) 2019-04-04 2019-04-04 Dynamic monitoring method for spartina alterniflora new-born plaques

Country Status (1)

Country Link
CN (1) CN110032963B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113128525A (en) * 2021-05-19 2021-07-16 内蒙古农业大学 Control device and method for desert grassland population patch identification
CN113487493B (en) * 2021-06-02 2023-08-18 厦门大学 GANilla-based SAR image automatic colorization method
CN113436071B (en) * 2021-06-21 2022-04-29 武汉光谷信息技术股份有限公司 Multi-source satellite remote sensing image mosaic method, system and storage medium
CN113408460B (en) * 2021-06-30 2022-03-11 中国科学院东北地理与农业生态研究所 Method for detecting spartina alterniflora distribution based on remote sensing big data and cloud platform
CN114220002B (en) * 2021-11-26 2022-11-15 通辽市气象台(通辽市气候生态环境监测中心) Method and system for monitoring invasion of foreign plants based on convolutional neural network
CN114782842A (en) * 2022-04-21 2022-07-22 中国农业科学院农业基因组研究所 Spartina alterniflora recognition and early warning method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100067799A1 (en) * 2008-09-17 2010-03-18 Microsoft Corporation Globally invariant radon feature transforms for texture classification
CN103528570A (en) * 2013-10-23 2014-01-22 环境保护部卫星环境应用中心 Spatial distribution acquiring method and system for spartina alterniflora in northern sea area of Guangxi
US10664702B2 (en) * 2016-12-30 2020-05-26 International Business Machines Corporation Method and system for crop recognition and boundary delineation

Also Published As

Publication number Publication date
CN110032963A (en) 2019-07-19

Similar Documents

Publication Publication Date Title
CN110032963B (en) Dynamic monitoring method for spartina alterniflora new-born plaques
CN108596103B (en) High-resolution remote sensing image building extraction method based on optimal spectral index selection
CN108764255B (en) Method for extracting winter wheat planting information
CN109919875B (en) High-time-frequency remote sensing image feature-assisted residential area extraction and classification method
CN111832518B (en) Space-time fusion-based TSA remote sensing image land utilization method
CN102800074B (en) Synthetic aperture radar (SAR) image change detection difference chart generation method based on contourlet transform
Caprioli et al. Accuracy assessment of per-field classification integrating very fine spatial resolution satellite imagery with topographic data
CN111008664B (en) Hyperspectral sea ice detection method based on space-spectrum combined characteristics
CN105787457A (en) Evaluation method for improving vegetation classified remote sensing precision through integration of MODIS satellite and DEM
CN107688777B (en) Urban green land extraction method for collaborative multi-source remote sensing image
CN112884672B (en) Multi-frame unmanned aerial vehicle image relative radiation correction method based on contemporaneous satellite images
CN104217426A (en) Object-oriented water-body extracting method based on ENVISAT ASAR and Landsat TM remote sensing data
CN112285710A (en) Multi-source remote sensing reservoir water storage capacity estimation method and device
CN113887493B (en) Black and odorous water body remote sensing image identification method based on ID3 algorithm
CN115862010B (en) High-resolution remote sensing image water body extraction method based on semantic segmentation model
CN115294183A (en) Disc-shaped sub-lake water body time sequence extraction method based on multi-source remote sensing data
CN115510377B (en) Soil erosion intensity estimation method and system and storage medium
CN107688776A (en) A kind of urban water-body extracting method
CN112364289A (en) Method for extracting water body information through data fusion
CN110909821B (en) Method for carrying out high-space-time resolution vegetation index data fusion based on crop reference curve
CN115457325A (en) Object and spectrum rule oriented multi-spectral remote sensing image unsupervised classification method
CN115018859A (en) Urban built-up area remote sensing extraction method and system based on multi-scale space nesting
CN108563674B (en) Sea area geographic element measurement method, system and device based on RS and GIS
Nelson et al. Combining moderate-resolution time-series RS data from SAR and optical sources for rice crop characterisation: examples from Bangladesh
CN116842343B (en) Method for quantifying influence of urban forest on temperature based on satellite observation and space conversion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20201030