CN107301631B - SAR image speckle reduction method based on non-convex weighted sparse constraint - Google Patents
SAR image speckle reduction method based on non-convex weighted sparse constraint Download PDFInfo
- Publication number
- CN107301631B CN107301631B CN201710506574.XA CN201710506574A CN107301631B CN 107301631 B CN107301631 B CN 107301631B CN 201710506574 A CN201710506574 A CN 201710506574A CN 107301631 B CN107301631 B CN 107301631B
- Authority
- CN
- China
- Prior art keywords
- image
- estimated
- image block
- coefficient
- coefficient matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 239000011159 matrix material Substances 0.000 claims abstract description 32
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 7
- 230000009466 transformation Effects 0.000 claims description 10
- 239000000654 additive Substances 0.000 claims description 4
- 230000000996 additive effect Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 8
- 230000014759 maintenance of location Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 238000009499 grossing Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention discloses an SAR image speckle reduction method based on non-convex weighted sparse constraint, and belongs to the technical field of digital image processing. Firstly, searching a similar image block set for each target image block through similarity comparison and performing singular value decomposition to obtain a coefficient matrix by utilizing the sparsity of the similar image block set in a transform domain, then performing non-convex weighting constraint on the coefficient matrix, estimating the coefficient matrix through threshold shrinkage to enable the estimated coefficient matrix to be closer to a real coefficient, and finally reconstructing a speckle reduction result by utilizing the estimated coefficient matrix; according to the speckle reduction method, the non-convex weighting constraint is carried out on the coefficient matrix, so that the speckle reduced image effectively inhibits speckle noise while details are kept, a more accurate speckle reduction image is obtained, and the target identification is easier, so that the method can be used for reducing the speckle of the SAR image.
Description
Technical Field
The invention belongs to the technical field of digital image processing, and particularly relates to a non-convex weighted sparse constraint image speckle reduction method based on an image block set, which is used for SAR image speckle reduction processing.
Background
Synthetic Aperture Radar (SAR) imaging is widely applied to civil and military aspects such as topographic mapping, disaster prediction, battlefield reconnaissance and the like due to the characteristics of strong weather condition interference resistance and high distance direction and azimuth direction resolution of the SAR imaging all day long, but the SAR image has serious speckle noise due to the specific imaging process of the SAR, so that the difficulty of small target identification is easily caused, and therefore, the SAR image needs to be subjected to speckle suppression before subsequent segmentation identification and other processing.
The method for inhibiting the speckle before SAR image imaging is mainly a multi-vision technology, namely, multiple sub-images of the same scene are subjected to average processing, the method can preliminarily inhibit the speckle of the SAR image, and most SAR image speckle inhibition methods are mainly concentrated after imaging and generally divided into two categories of airspace and transform domain. The method mainly analyzes the distribution of an SAR image environment model and the distribution of a noise model, and carries out filtering processing on the image in the airspace by combining a signal estimation theory, wherein more classical methods comprise Lee filtering, Frost filtering, Kuan filtering and the like, but the speckle suppression capability is limited and the retention capability of the image edge details is insufficient. The transform domain filtering method starts from the development of wavelet technology and is introduced into SAR image despeckle, and on the basis of the development, a series of multi-scale transform methods are proposed. In recent years, with the development of sparse theory, a method for reconstructing based on sparsity and non-local similarity of images is becoming a hot point of research. Due to the fact that the image has sparsity in a transform domain, similar structures exist in different regions in the image, and speckle noise can be further suppressed by combining the two characteristics, in the method, the more classical SAR-BM3D method is adopted, the speckle reduction result is still at a higher level at present, but the method is easy to generate an artifact phenomenon in a smooth region and bring interference in target identification.
Disclosure of Invention
The invention aims to provide an SAR image speckle reduction method based on non-convex weighting sparse constraint aiming at the defect of image detail retention in the existing SAR image speckle reduction. According to the method, the non-local similarity and the low-rank structurality of the SAR image are fully considered, and the non-convex weighting constraint is carried out on the coefficient matrix of the similar image block set, so that the estimated SAR image can restrain speckle noise while a large amount of details in the image are reserved. The method comprises the following steps:
step one, establishing a non-convex weighted sparse constraint model
Firstly, carrying out logarithmic transformation on an SAR image, converting a multiplicative noise model into an additive noise model, and then carrying out the ith target image block x in an input imageiComparing the similarity with all image blocks in the search range, and selecting the similarityThe highest image blocks and the target image block jointly form a similar image block set Rix, wherein RiExtracting a matrix for the image block, and finally establishing a non-convex weighted sparse constraint model as follows:
where X and y represent the real image and the initial image, respectively, to be estimated, XiFor a set of similar image blocks of the real image to be estimated,represents XiThe weighted p-norm of the corresponding coefficient matrix (0 < p < 1), ω is the weight vector, and λ and η are the parameters for balancing the terms.
Step two, decomposition and transformation of the model
Decomposing the constraint model in the step one, and converting the constraint model into a sub-problem about solving a similar image block set:
and sub-problems with image reconstruction:
for the sub-problem of the similar image block set solved by the formula (2), firstly, the input similar image block set R is subjected toix is subjected to singular value decomposition to obtain a corresponding coefficient matrixiThen, converting the non-convex weighted sparse constraint model aiming at the similar image block set in the real image into a non-convex weighted constraint model aiming at a coefficient matrix of the similar image block set in the real image:
wherein ΔiFor sets X of similar image blocks in a real image to be estimatediA corresponding matrix of coefficients is then formed,jis ΔiCoefficient of the j-th coefficient, ωjIs the corresponding weight parameter.
Step three, estimating a coefficient matrix and reconstructing an image
Coefficient matrix Delta of formula (4) in Pair step twoiWhen making an estimation, due to ΔiIs relatively independent, so the estimation model for each coefficient is:
wherein gamma isjRepresenting a matrix of coefficientsiThen, each coefficient is estimated by utilizing threshold shrinkage, and after the estimated value of each coefficient is obtained, the estimated similar image block set X in the real image can be obtainediThen, the sub-problem of the image reconstruction of equation (3) is solved using equation (6):
and loop iteration solving about similar image block set XiAnd estimating the subproblem of the image x until convergence or iteration times are reached, and then performing exponential transformation on the estimated image x to obtain a finally estimated speckle reduction SAR image.
The method has the innovation points that the coefficient matrix of the SAR image is subjected to non-convex weighting constraint by utilizing the low-rank characteristic of a similar image block set in the speckle reduction process of the SAR image; and the coefficient matrix is estimated by utilizing threshold shrinkage, so that the estimation result is closer to the true value, and the method is used for reducing the speckle of the SAR image.
The invention has the beneficial effects that: similar image block matching and singular value decomposition are carried out by combining the local sparsity and non-local similarity of the image blocks, so that the sparse representation performance is improved; utilizing a non-convex weighting constraint coefficient matrix to enable the coefficient to be closer to a true value; and each dimension coefficient is estimated by utilizing threshold shrinkage, so that the estimation result is more accurate, and therefore, the finally estimated image not only retains a large amount of details, but also effectively inhibits the generation of artifacts, and the overall effect is closer to a real image.
The invention mainly adopts a simulation experiment method for verification, and all steps and conclusions are verified to be correct on MATLAB 9.0.
Drawings
FIG. 1 is a workflow block diagram of the present invention;
FIG. 2 is a SAR image to be despecked used in the simulation of the present invention;
FIG. 3 is a plot of the PPB process versus the plaque reduction results of FIG. 2;
FIG. 4 is a graph of SAR-BM3D method versus speckle reduction results of FIG. 2;
FIG. 5 is a graph of the plaque reduction results of FIG. 2 for the method of the present invention.
Detailed Description
Referring to fig. 1, the invention relates to a SAR image speckle reduction method based on non-convex weighted sparse constraint, which comprises the following specific steps:
step one, establishing a non-convex weighted sparse constraint model
Carrying out logarithmic transformation on the SAR image to convert a multiplicative noise model into an additive noise model:
on the basis of an additive model, for each target image block x within the imageiAnd comparing the similarity with all image blocks in the search range, wherein in order to meet the multiplicative model characteristic of the SAR image, the similarity between two image blocks is compared by adopting an equation (8):
wherein xi(k) Representing image blocks xiSelecting S-1 image blocks with highest similarity to the kth pixel value to form a similar image block set R with the target image blockix, and establishing a non-convex weighted sparse constraint model according to the formula (1).
Step two, decomposition and transformation of the model
After the non-convex weighted constraint model is established, the model is decomposed into two subproblems according to the formula (2) and the formula (3), wherein a similar image block set of the real image in the formula (2) is obtainedA set R of similar image blocks to the input imageix is decomposed according to singular values:
SVD(Rix)=Ui·i·Viformula (9)
WhereiniIs Rix corresponding coefficient matrix, UiAnd ViRespectively, left and right orthogonal transformation matrixes, and then converting a non-convex weighted sparse constraint model for the similar image block set in the real image in formula (2) into a non-convex weighted constraint model for the similar image block set coefficient matrix in the real image in formula (4), wherein a weight parameter omegajCan be calculated from equation (7):
wherein gamma isjIs composed ofiThe j coefficient in (c) is a constant which changes according to different views of the SAR image, and in order to avoid a tiny positive number of the numerical overflow problem, the formula (4) is further converted into a scalar form:
i.e. an optimization problem that translates to the sum of the functions corresponding to each coefficient.
Step three, estimating a coefficient matrix and reconstructing an image
Since each coefficient in the optimization problem of formula (11) in step two is relatively independent, the optimization problem of solving each coefficient can be converted into formula (5), and formula (5) is solved to obtain:
wherein τ is a threshold, and is an iterative solution thereof, and the threshold τ can be obtained by solving the derivative characteristic when an extremum is taken according to equation (5):
iterative solutions can be obtained after multiple iterative convergence by equation (14):
(l+1)=|γj|-ωjp((l))p-1formula (14)
Wherein l is iteration times, after each coefficient is estimated through the threshold shrinkage, the estimated value of the coefficient matrix of the similar image block set of the real image can be obtained, then a reconstructed image is solved by using a formula (6), and the similar image block set X is solved through circular iterationiAnd estimating the subproblem of the image x until convergence or iteration times are reached, and then performing exponential transformation on the estimated image x to obtain a finally estimated speckle reduction SAR image.
The effect of the invention can be further illustrated by the following simulation experiment:
experimental conditions and contents
The experimental conditions are as follows: the input image used for the experiment is fig. 2, with a pixel size of 256 × 256. In the experiment, each spot reduction method is realized by using MATLAB language programming.
The experimental contents are as follows: under the experimental conditions described above, the PPB method and the SAR-BM3D method were used for comparison with the method of the present invention. The objective evaluation index of the speckle reduction capability is comprehensively measured by using the homogeneous region variance and the equivalent vision ENL as well as the edge preservation coefficient EPI of the whole image.
Experiment 1: the speckle reduction treatment is carried out on the image 2 by the method of the invention and the existing PPB method and SAR-BM3D method respectively. The PPB method is one of the more classical methods for SAR noise reduction at present, and especially in a homogeneous region, the speckle reduction result is shown in fig. 3; the SAR-BM3D method uses linear minimum mean square error in the transform domain to estimate the coefficients and is known for detail retention capability, and its speckle reduction result is shown in fig. 4. The method of the invention sets the size of the image block in the experimentThe number S of the image blocks contained in the similar image block set is set as follows:s80, the final reconstruction result is fig. 5.
Comparing the PPB method with the method of the invention, the PPB method is similar to the method of the invention in the performance of the smooth area, the smooth degree is slightly better than the method in some areas, but part of the details are transited smoothly in the speckle reduction result of the area with rich details, the processing result is not as good as the method of the invention; the result of the SAR-BM3D method is similar to the detail retention capacity of the method of the invention, but a large amount of artifacts exist in a smooth area, and the smoothing effect is inferior to that of the method of the invention and the PPB method; the method of the invention utilizes a non-convex weighting sparse constraint method to constrain the coefficient matrix, and adopts threshold shrinkage to realize the estimation of the coefficient matrix, so that the speckle reduction result not only can keep most details in the original image, but also has better smoothing effect in a smoothing area, the whole image has good visual effect, and the subsequent processing of SAR images such as target recognition is convenient.
TABLE 1 comparison of the indices of different speckle reduction methods
Table 1 shows the corresponding variance, the ENL value, and the EPI value of the entire image when speckle reduction is performed on the two regions in fig. 2 by using different methods, where a smaller variance or a higher ENL value indicates better speckle reduction effect in the smooth region, and a higher EPI value indicates better retention of edge details, so the speckle reduction result of the SAR image should be combined with the results of the two indexes. It can be seen that the method of the present invention is more prominent in both smoothness and detail retention, and inhibits speckle while retaining details, compared to other methods, while the PPB method performs better only on the ENL value and variance, and does not perform as well on the EPI value as the SAR-BM3D method and the method of the present invention, and the SAR-BM3D method performs better on the EPI value, as opposed to the PPB method, and does not perform as well on the variance and ENL value as the PPB method and the method, consistent with intuitive visual results.
The experiments show that the speckle reduction method effectively inhibits speckle noise while retaining a large amount of detail information, and has good visual effect and objective evaluation index, so that the speckle reduction method is effective for reducing the speckle of the SAR image.
Claims (1)
1. A SAR image speckle reduction method based on non-convex weighting sparse constraint is characterized by comprising the following specific steps:
step one, establishing a non-convex weighted sparse constraint model
Firstly, carrying out logarithmic transformation on an SAR image to be processed, converting a multiplicative noise model into an additive noise model, then carrying out similarity comparison on the ith target image block in an input initial image and all image blocks in the search range of the ith target image block, and using R to carry out similarity comparisoniAnd as an image block extraction matrix, selecting a plurality of image blocks with highest similarity to form a similar image block set together with the target image block, and finally establishing a non-convex weighted sparse constraint model as follows:
where X and y represent the real image and the initial image, respectively, to be estimated, XiFor a set of similar image blocks of the real image to be estimated,represents XiThe weighted p norm of the corresponding coefficient matrix, wherein p is more than 0 and less than 1, omega is a weight vector, and lambda and η are parameters of balance terms;
step two, decomposition and transformation of the model
Decomposing the constraint model in the step one, and converting the constraint model into a sub-problem about solving a similar image block set:
and sub-problems with image reconstruction:
for solving the subproblems of the similar image block set, firstly, the input similar image block set is subjected to singular value decomposition to obtain a corresponding coefficient matrixiThen, converting the non-convex weighted sparse constraint model aiming at the similar image block set in the real image into a non-convex weighted constraint model aiming at a coefficient matrix of the similar image block set in the real image:
wherein ΔiFor sets X of similar image blocks in a real image to be estimatediA corresponding matrix of coefficients is then formed,jis ΔiCoefficient of the j-th coefficient, ωjIs the corresponding weight parameter;
step three, estimating a coefficient matrix and reconstructing an image
Coefficient matrix delta in pair step twoiWhen making an estimation, due to ΔiIs relatively independent, so the estimation model for each coefficient is:
wherein gamma isjRepresenting a matrix of coefficientsiThen, each coefficient is estimated by utilizing threshold shrinkage, and after the estimated value of each coefficient is obtained, the estimated similar image block set X in the real image can be obtainediThen solving the subproblem of image reconstruction:
and loop iteration solving about similar image blocksSet XiAnd solving the subproblems of the real image x to be estimated until convergence or iteration times are reached, and then performing exponential transformation on the estimated real image x to obtain a final estimated speckle reduction SAR image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710506574.XA CN107301631B (en) | 2017-06-28 | 2017-06-28 | SAR image speckle reduction method based on non-convex weighted sparse constraint |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710506574.XA CN107301631B (en) | 2017-06-28 | 2017-06-28 | SAR image speckle reduction method based on non-convex weighted sparse constraint |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107301631A CN107301631A (en) | 2017-10-27 |
CN107301631B true CN107301631B (en) | 2020-09-18 |
Family
ID=60135069
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710506574.XA Expired - Fee Related CN107301631B (en) | 2017-06-28 | 2017-06-28 | SAR image speckle reduction method based on non-convex weighted sparse constraint |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107301631B (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109658340B (en) * | 2018-10-17 | 2023-04-25 | 南京航空航天大学 | SAR image rapid denoising method based on RSVD and histogram preservation |
CN112099010B (en) * | 2020-09-16 | 2021-05-25 | 中国人民解放军国防科技大学 | ISAR (inverse synthetic aperture radar) imaging method for target with micro-motion component based on structured non-convex low-rank representation |
CN112488960A (en) * | 2020-12-16 | 2021-03-12 | 中国人民解放军国防科技大学 | SAR image speckle suppression method based on boundary and context constraint |
CN112927165A (en) * | 2021-03-22 | 2021-06-08 | 重庆邮电大学 | SAR image speckle suppression method based on NSST domain three-dimensional block matching |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104933683A (en) * | 2015-06-09 | 2015-09-23 | 南昌大学 | Non-convex low-rank reconstruction method for rapid magnetic resonance (MR) imaging |
CN104952051A (en) * | 2015-07-07 | 2015-09-30 | 西安电子科技大学 | Low-rank image restoration method based on Gaussian mixture model |
CN105427264A (en) * | 2015-12-23 | 2016-03-23 | 重庆大学 | Image reconstruction method based on group sparsity coefficient estimation |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8935308B2 (en) * | 2012-01-20 | 2015-01-13 | Mitsubishi Electric Research Laboratories, Inc. | Method for recovering low-rank matrices and subspaces from data in high-dimensional matrices |
US20160156652A1 (en) * | 2012-04-20 | 2016-06-02 | Numerica Corporaition | Pattern detection in sensor networks |
-
2017
- 2017-06-28 CN CN201710506574.XA patent/CN107301631B/en not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104933683A (en) * | 2015-06-09 | 2015-09-23 | 南昌大学 | Non-convex low-rank reconstruction method for rapid magnetic resonance (MR) imaging |
CN104952051A (en) * | 2015-07-07 | 2015-09-30 | 西安电子科技大学 | Low-rank image restoration method based on Gaussian mixture model |
CN105427264A (en) * | 2015-12-23 | 2016-03-23 | 重庆大学 | Image reconstruction method based on group sparsity coefficient estimation |
Non-Patent Citations (3)
Title |
---|
Nonlocal Low-Rank-Based Compressed Sensing for Remote Sensing Image Reconstruction;Jingbo Wei et al.;《 IEEE Geoscience and Remote Sensing Letters 》;20161031;第13卷(第10期);第1557 - 1561页 * |
基于过完备字典的非凸压缩感知理论与方法研究基于过完备字典的非凸压缩感知理论与方法研究;林乐平;《中国博士学位论文全文数据库信息科技辑》;20170215;第I138-99页 * |
基于非局部相似和低秩矩阵逼近的SAR图像去噪;赵杰等;《计算机科学》;20170615;第44卷(第6A期);第183-187页 * |
Also Published As
Publication number | Publication date |
---|---|
CN107301631A (en) | 2017-10-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112200750B (en) | Ultrasonic image denoising model establishing method and ultrasonic image denoising method | |
CN107301631B (en) | SAR image speckle reduction method based on non-convex weighted sparse constraint | |
CN112233026A (en) | SAR image denoising method based on multi-scale residual attention network | |
CN108399611B (en) | Multi-focus image fusion method based on gradient regularization | |
CN104123705B (en) | A kind of super-resolution rebuilding picture quality Contourlet territory evaluation methodology | |
CN109447073B (en) | Infrared weak and small target detection method based on tensor robust principal component analysis | |
CN107730482B (en) | Sparse fusion method based on regional energy and variance | |
CN107292852B (en) | Image denoising algorithm based on low-rank theory | |
CN104217406B (en) | SAR image noise reduction method based on shear wave coefficient processing | |
CN110517195B (en) | Unsupervised SAR image denoising method | |
CN105427264A (en) | Image reconstruction method based on group sparsity coefficient estimation | |
CN105894476A (en) | Fused SAR image noise reduction processing method based on dictionary learning | |
CN102567973A (en) | Image denoising method based on improved shape self-adaptive window | |
CN104657951A (en) | Multiplicative noise removal method for image | |
CN109636722B (en) | Method for reconstructing super-resolution of online dictionary learning based on sparse representation | |
CN112785636B (en) | Multi-scale enhanced monocular depth estimation method | |
CN107274360B (en) | A kind of high spectrum image denoising method based on Fisher dictionary learning, low-rank representation | |
CN106934398B (en) | Image de-noising method based on super-pixel cluster and rarefaction representation | |
CN106296583B (en) | Based on image block group sparse coding and the noisy high spectrum image ultra-resolution ratio reconstructing method that in pairs maps | |
CN109741258B (en) | Image super-resolution method based on reconstruction | |
CN115082336A (en) | SAR image speckle suppression method based on machine learning | |
CN104978716A (en) | SAR image noise reduction method based on linear minimum mean square error estimation | |
CN107392861B (en) | Sparse representation SAR image speckle reduction method based on Gaussian proportion mixed model | |
CN112686830A (en) | Super-resolution method of single depth map based on image decomposition | |
CN116051444A (en) | Effective infrared and visible light image self-adaptive fusion method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200918 |