CN112837233B - Polarization image defogging method for acquiring transmissivity based on differential polarization - Google Patents

Polarization image defogging method for acquiring transmissivity based on differential polarization Download PDF

Info

Publication number
CN112837233B
CN112837233B CN202110050501.0A CN202110050501A CN112837233B CN 112837233 B CN112837233 B CN 112837233B CN 202110050501 A CN202110050501 A CN 202110050501A CN 112837233 B CN112837233 B CN 112837233B
Authority
CN
China
Prior art keywords
image
polarization
transmittance
light
atmospheric
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN202110050501.0A
Other languages
Chinese (zh)
Other versions
CN112837233A (en
Inventor
宋雪艳
宫爱玲
李尚位
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kunming University of Science and Technology
Original Assignee
Kunming University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kunming University of Science and Technology filed Critical Kunming University of Science and Technology
Priority to CN202110050501.0A priority Critical patent/CN112837233B/en
Publication of CN112837233A publication Critical patent/CN112837233A/en
Application granted granted Critical
Publication of CN112837233B publication Critical patent/CN112837233B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G06T5/73
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention discloses a polarization image defogging method for acquiring transmissivity based on differential polarization, and belongs to the technical field of image defogging processing. The method of the invention is to use a polarization imaging method to obtain two orthogonal polarization images of the same scene. And establishing an atmospheric scattering model based on the physical model of the foggy image. And combining the bright channel prior and the dark channel prior to respectively calculate the bright channel image and the dark channel image of the two sub-images to obtain a transmittance image t1 of the dark channel and an atmospheric light value at infinity. According to the differential image defogging method, the optical thickness is replaced by the approximate fog density, and a differential transmittance map t2 is obtained. And combining the dark channel transmittance graph and the differential transmittance graph to obtain a total transmittance image, performing threshold processing on the total transmittance image, and performing refinement processing by using guide filtering. And substituting the obtained transmissivity and atmospheric light into an atmospheric scattering model to obtain a fog-free image. The invention obviously improves the image quality, obtains more reflected light information of objects and improves the definition of the restored image.

Description

Polarization image defogging method for acquiring transmissivity based on differential polarization
Technical Field
The invention relates to a polarization image defogging method for acquiring transmissivity based on differential polarization, and belongs to the technical field of image defogging processing.
Background
In recent years, haze weather is increasingly aggravated, which causes the quality of images acquired by an imaging system to be reduced, and further seriously influences the analysis and judgment of a visual system; therefore, improving the definition of the foggy image to obtain better visual effect and richer detail information processing algorithms has become a key field of research. Currently, numerous researchers have proposed many methods for processing foggy day images; atmospheric light formed by scattering sunlight by haze and other scattering particles has partial polarization characteristics, so that the polarized optical imaging defogging method is suitable for various haze weathers of various scenes, has a wide application range and becomes one of research hotspots; under the strong scattering environment, the target detection and identification based on the polarization imaging technology has the advantages and the application which are not available in other imaging modes; the dark channel prior algorithm is not suitable for sky and large-area white areas, and the atmospheric light estimated value is easily overlarge, so that the color distortion problem is caused.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a polarization image defogging method based on differential polarization and combined with a bright-dark channel prior to acquiring the transmittance, so as to overcome the defects of the prior art.
The technical scheme adopted by the invention is as follows: a polarized image defogging method based on differential polarization and combined with light and dark channel prior transmittance acquisition comprises the following steps:
the method specifically comprises the following steps:
(1) imaging a target scene by using a polarization imaging device, rotating a polaroid to obtain two orthogonal polarization images of the same scene, and recording a light intensity image with the minimum influence of fog on the images as I || Intensity map with the greatest effect on the image, denoted I
The atmospheric scattering model is established based on the physical model of the foggy image, and comprises two parts:
the scene target is directly transparent: d (x, y) ═ L (x, y) t (x, y)
Atmospheric light: b (x, y) ═ a [1-t(x,y)]
Wherein L (x, y) is the reflected light intensity of the target, i.e. fog-free image, t is e -βd Beta d is the optical thickness, A Atmospheric light value at infinity;
the total light intensity, i.e. the foggy image, is an incoherent superposition of the two, i.e.:
I(x,y)=D(x,y)+B(x,y)=L(x,y)t(x,y)+A [1-t(x,y)]
the target reflected light intensity can be obtained:
Figure GDA0003032271960000021
(2) inputting two orthogonal polarization images, combining bright and dark channel prior, respectively calculating bright and dark channel images of two sub-images, and obtaining a transmittance image t1 of a dark channel and an atmospheric light value at infinity;
(3) according to the difference image defogging method, the fog concentration is approximately used for replacing the optical thickness, and then a difference transmittance graph t2 is obtained;
(4) combining the dark channel transmittance graph with the differential transmittance graph to obtain a total transmittance image, performing threshold processing on the total transmittance image, and performing refinement processing by using guide filtering to obtain an optimized transmittance image;
(5) substituting the obtained total transmittance image and the atmospheric scattering image into the large imageIn the air scattering model, a defogged restored image is obtained, namely:
Figure GDA0003032271960000022
further, the dark channel prior theory in step (2) of the present invention is as follows: for most images without sky areas, the pixel value of at least one of the three color channels of all the pixel points of the image block is the lowest, and a dark channel image is defined as follows:
Figure GDA0003032271960000023
wherein I c Representing the color channel of the image I, wherein omega (x) represents an image block taking a pixel point x as the center, counting finds that the image is out of the sky area and a dark channel I dark The value is extremely low and approaches to zero, so that a dark channel transmittance image can be obtained
Figure GDA0003032271960000024
Omega is an adjusting parameter;
similar to dark channel prior, the pixel value for the presence of one channel is very large, approaching 1, and the bright channel image is defined as:
Figure GDA0003032271960000025
the image block is maximized, the channel is maximized, and the bright channel of any pixel point is close to the atmospheric light value A of the fog-free image light :I light →A light (ii) a The atmospheric light value is estimated by combining the two sub-images as:
Figure GDA0003032271960000026
where m, n are the size of the image.
Further, step (4) of the present invention includes the following steps: the two subgraphs are respectively processed by sum and difference, and the sum and the difference can be obtainedObtaining a total light intensity graph I ═ I || +I And polarization difference diagram Δ I ═ I || -I (ii) a Degree of image polarization
Figure GDA0003032271960000027
Atmospheric light polarization degree:
Figure GDA0003032271960000031
based on the polarization characteristics of light, it can be approximately considered that the polarization state of light waves reaching the polarization imaging system is mainly caused by atmospheric light, so that there are:
Figure GDA0003032271960000032
therefore, when Δ B is equal to Δ I, it is possible to obtain
Figure GDA0003032271960000033
When the transmittance is determined, the atmospheric light is deformed: b (x, y) ═ a [1-t(x,y)]→B(x,y)= A H(x,y),
Figure GDA0003032271960000034
Wherein P is scatt P, H (x, y) is the haze concentration, H (x, y) 1-t (x, y) 1-e -βd Since H ∈ β d, the differential image transmittance can be determined by roughly replacing the optical thickness of β d with H:
t2(x,y)=e -η*H(x,y)
further, the total transmission in step (5) of the present invention is: t is alpha t1+ beta t2, and alpha and beta are correction coefficients; obtaining an atmospheric scattering image B (x, y) ═ A by combining an atmospheric scattering model [1-t(x,y)](ii) a In order to avoid the pixel value of the calculated transmissivity from being too low, threshold processing is carried out on the total transmissivity, and pixel points with the pixel values smaller than 0.1 are set to be 0.1; and taking the total image obtained by performing the processing on the two sub-images as a guide filtering image, and performing guide filtering processing on the transmissivity image to obtain an optimized transmissivity image.
Compared with the prior art, the invention has the beneficial effects that:
according to the method, the polarization differential imaging and the bright-dark channel prior algorithm are combined, and compared with the traditional dark channel algorithm, the quality of the restored image is obviously improved, namely the problems of color distortion and blocking effect after dark channel prior processing do not exist, the detail information is richer, more image contents are obtained, and the definition of the restored image is improved.
Drawings
FIG. 1 is a flow chart of the algorithm of the present invention;
fig. 2 (a) is a fog image to be processed of orthogonal image synthesis;
fig. 2 (b) is a restored image of the difference algorithm defogging;
fig. 2 (c) is a restored image of the dark channel algorithm defogging;
fig. 2 (d) shows a restored image of the defogging by the algorithm herein.
Detailed Description
The invention will be further described with reference to the drawings and the embodiments, but the scope of the invention is not limited thereto.
Example 1
A polarization image defogging method based on differential polarization and combined with light and dark channel prior transmittance acquisition specifically comprises the following steps:
(1) imaging a target scene by using a polarization imaging method and equipment, and rotating a polaroid to obtain two orthogonal polarization images of the same scene; respectively, the intensity map (denoted as I) in which the fog has the least influence on the image || ) And the intensity map (denoted as I) that has the greatest effect on the image ) (ii) a Two orthogonal images were read in MATLAB (both steps below were implemented on the MATLAB platform).
According to the imaging characteristics of a scattering environment, an atmospheric scattering model comprises two parts:
the scene target is directly transparent: d (x, y) ═ L (x, y) t (x, y)
Atmospheric light: b (x, y) ═ a [1-t(x,y)]
Wherein (x, y) represents the coordinates of the pixel in the diagram,l (x, y) is the reflected light intensity of the target, i.e. the fog-free image, t ═ e -βd Beta d is the optical thickness, A Is the atmospheric light value at infinity.
The total light intensity (i.e. the foggy image) is the incoherent superposition of the two, i.e.:
I(x,y)=D(x,y)+B(x,y)=L(x,y)t(x,y)+A [1-t(x,y)]
the target reflected light intensity can be obtained:
Figure GDA0003032271960000041
(2) obtaining dark channel images of two subgraphs according to a dark channel prior theory: for most images without sky areas, the pixel value of at least one of the three color channels of all the pixel points of the image block is the lowest; define dark channel images as:
Figure GDA0003032271960000042
wherein I c Representing the color channel of the image I, wherein omega (x) represents an image block taking a pixel point x as the center, counting finds that the image is out of the sky area and a dark channel I dark Very low value (here I) dark The sum of two sub-dark channel maps) approaches zero, so that the coarse image transmittance can be obtained
Figure GDA0003032271960000043
ω 0.95 is the adjustment parameter.
Similar to dark channel prior, there is a maximum pixel value for one channel, approaching 1, and the bright channel image is defined as:
Figure GDA0003032271960000044
the image block is maximized, the channel is maximized, and the bright channel of any pixel point is close to the atmospheric light value A of the fog-free image light :I light →A light . The atmospheric light value is estimated by combining the two sub-images as follows:
Figure GDA0003032271960000051
where m, n are the size of the image.
And respectively carrying out sum and difference processing on the two subgraphs to obtain a total light intensity graph (to-be-processed haze graph) I ═ I || +I And polarization difference diagram Δ I ═ I || -I (ii) a Degree of image polarization
Figure GDA0003032271960000052
Atmospheric light polarization degree:
Figure GDA0003032271960000053
based on the polarization characteristics of light, it can be approximated that the polarization state of light waves arriving at a polarization imaging system is mainly caused by atmospheric light, so that there are:
Figure GDA0003032271960000054
therefore, when Δ B is equal to Δ I, it is possible to obtain
Figure GDA0003032271960000055
When the transmittance is determined, the atmospheric light is deformed: b (x, y) ═ a [1-t(x,y)]→B(x,y)= A H(x,y),
Figure GDA0003032271960000056
Wherein P is scatt P, H (x, y) is the haze concentration, H (x, y) 1-t (x, y) 1-e -βd Since H ℃βd, the differential image transmittance can be determined by roughly replacing the optical thickness of β d with H:
t2(x,y)=e -η*H(x,y) η is set to 0.9.
The total transmission was: t is α t1+ β t2, α and β are correction coefficients, α is 0.4, and β is 0.9. Combination is bigObtaining an atmospheric scattering image B (x, y) ═ A by using a gas scattering model [1-t(x,y)]Then, in order to avoid the pixel value of the calculated transmittance from being too low, threshold processing is performed on the total transmittance, and the pixel point with the pixel value less than 0.1 is set to be 0.1.
In order to avoid the block effect of the transmissivity t and the loss of image information, the total image obtained by performing the processing on the two sub-images is used as a guide filtering image, and the guide filtering processing is performed on the transmissivity image; specifically, the coarse transmittance t is used as the input image p of the guide filtering, and the guide image I needs the same edge information as the input image, so that the images are processed by using the original images to be processed respectively to obtain optimized transmittance images.
And finally, substituting the optimized transmittance image and the optimized atmosphere scattering image into an atmosphere scattering model to obtain a defogged restored image, namely:
Figure GDA0003032271960000057
compared with the original image in fig. 2 and the image processed by other methods, it can be seen from A, B, C that the image information restored by the processing result of the method of the present invention is more and there is no color distortion problem.

Claims (2)

1. A polarization image defogging method for acquiring transmittance based on differential polarization is characterized by comprising the following steps:
(1) imaging a target scene by using a polarization imaging device, rotating a polaroid to obtain two orthogonal polarization images of the same scene, and recording a light intensity image with the minimum influence of fog on the images as I Intensity map with the greatest effect on the image, denoted I
The atmospheric scattering model is established based on the physical model of the foggy image, and comprises two parts:
the scene target is directly transparent: d (x, y) ═ L (x, y) t (x, y)
Atmospheric light: b (x, y) ═ a [1-t(x,y)]
Where L (x, y) is the reflected light intensity of the target, i.e. the fog-free image, and t ═ e -βd Beta d is the optical thickness, A Is at infinityA gas-light value;
the total light intensity, i.e. the foggy image, is the incoherent superposition of the two, i.e.:
I(x,y)=D(x,y)+B(x,y)=L(x,y)t(x,y)+A [1-t(x,y)]
the target reflected light intensity can be obtained:
Figure FDA0003706602830000011
(2) inputting two orthogonal polarization images, combining bright channel prior and dark channel prior, respectively calculating the bright channel image and the dark channel image of the two sub-images, and obtaining a transmissivity image t1 of a dark channel and an atmospheric light value at infinity;
(3) according to the difference image defogging method, the fog concentration is approximately used for replacing the optical thickness, and then a difference transmittance graph t2 is obtained;
(4) combining the dark channel transmittance graph with the differential transmittance graph to obtain a total transmittance image, performing threshold processing on the total transmittance image, and performing refinement processing by using guide filtering to obtain an optimized transmittance image;
(5) substituting the obtained total transmittance image and the obtained atmospheric scattering image into an atmospheric scattering model to obtain a defogging restoration image, namely:
Figure FDA0003706602830000012
the dark channel prior theory in the step (2): for most images without sky areas, the pixel value of at least one of the three color channels of all the pixel points of the image block is the lowest, and a dark channel image is defined as follows:
Figure FDA0003706602830000013
wherein I c Representing the color channel of the image I, and omega (x) representing the image block with the pixel point x as the center, counting to find out that the dark channel I is outside the sky area in the image dark The value is extremely low and approaches to zero, so that the transmissivity image of a dark channel can be obtained
Figure FDA0003706602830000014
Omega is an adjusting parameter;
similar to dark channel prior, the pixel value for the presence of one channel is very large, approaching 1, and the bright channel image is defined as:
Figure FDA0003706602830000021
the image block is maximized, the channel is maximized, and the bright channel of any pixel point is close to the atmospheric light value A of the fog-free image light :I light →A light (ii) a The atmospheric light value is estimated by combining the two sub-images as:
Figure FDA0003706602830000022
wherein m, n is the size of the image;
the step (3) comprises the following steps: and respectively carrying out sum and difference processing on the two sub-graphs to obtain a total light intensity graph I-I +I And the polarization difference diagram Δ I ═ I -I (ii) a Degree of image polarization
Figure FDA0003706602830000023
Atmospheric light polarization degree:
Figure FDA0003706602830000024
based on the polarization characteristics of light, it can be approximately considered that the polarization state of light waves reaching the polarization imaging system is mainly caused by atmospheric light, so that there are:
Figure FDA0003706602830000025
therefore, when Δ B is equal to Δ I, it is available
Figure FDA0003706602830000026
When the transmittance is determined, the atmospheric light is deformed: b (x, y) ═ a [1-t(x,y)]→B(x,y)=A H(x,y),
Figure FDA0003706602830000027
Wherein P is scatt P, H (x, y) is the haze concentration, H (x, y) 1-t (x, y) 1-e -βd Since H ∈ β d, the differential image transmittance can be determined by roughly replacing the optical thickness of β d with H:
t2(x,y)=e -η*H(x,y) where η is set to 0.9.
2. The polarization image defogging method according to claim 1, wherein said transmittance is obtained based on differential polarization, and said method comprises: the total transmission in the step (4) is as follows: t is alpha t1+ beta t2, and alpha and beta are correction coefficients; obtaining an atmospheric scattering image B (x, y) ═ A by combining an atmospheric scattering model [1-t(x,y)](ii) a In order to avoid the pixel value of the calculated transmissivity from being too low, threshold processing is carried out on the total transmissivity, and pixel points with the pixel values smaller than 0.1 are placed at 0.1; and taking the total image obtained by performing the summation processing on the two sub-images as a guide filtering image, and performing guide filtering processing on the transmittance image to obtain an optimized transmittance image.
CN202110050501.0A 2021-01-14 2021-01-14 Polarization image defogging method for acquiring transmissivity based on differential polarization Expired - Fee Related CN112837233B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110050501.0A CN112837233B (en) 2021-01-14 2021-01-14 Polarization image defogging method for acquiring transmissivity based on differential polarization

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110050501.0A CN112837233B (en) 2021-01-14 2021-01-14 Polarization image defogging method for acquiring transmissivity based on differential polarization

Publications (2)

Publication Number Publication Date
CN112837233A CN112837233A (en) 2021-05-25
CN112837233B true CN112837233B (en) 2022-09-09

Family

ID=75928199

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110050501.0A Expired - Fee Related CN112837233B (en) 2021-01-14 2021-01-14 Polarization image defogging method for acquiring transmissivity based on differential polarization

Country Status (1)

Country Link
CN (1) CN112837233B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610728A (en) * 2021-08-05 2021-11-05 北京信息科技大学 Polarization double-image defogging method based on four-dark-channel mean comparison
CN114324185A (en) * 2022-01-04 2022-04-12 浙江大学 Underwater polarization detection device based on Stokes vector
CN114841885B (en) * 2022-05-10 2023-03-10 中国矿业大学(北京) Defogging fusion processing method based on polarization image data
CN116337088B (en) * 2023-05-30 2023-08-11 中国人民解放军国防科技大学 Foggy scene relative motion estimation method and device based on bionic polarization vision
CN116596805B (en) * 2023-07-14 2023-09-29 山东大学 Polarization defogging method based on polarization state difference of scene object and atmosphere light
CN117649439B (en) * 2024-01-30 2024-04-09 山东省海洋资源与环境研究院(山东省海洋环境监测中心、山东省水产品质量检验中心) Sea weed bed area acquisition method, system, equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919879A (en) * 2019-03-13 2019-06-21 重庆邮电大学 A kind of image defogging method based on dark channel prior Yu bright channel prior

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5995140B2 (en) * 2012-01-19 2016-09-21 株式会社リコー Imaging apparatus, vehicle system including the same, and image processing method
CN105139347B (en) * 2015-07-10 2018-12-14 中国科学院西安光学精密机械研究所 A kind of polarization imaging defogging method of combination dark channel prior principle
CN107240080A (en) * 2017-06-12 2017-10-10 合肥工业大学 A kind of Misty Image reconstructing method and system
CN107966412B (en) * 2017-12-05 2020-04-10 天津大学 Image defogging method based on orthogonal polarization image stretching and polarization degree correlation
CN111292258B (en) * 2020-01-15 2023-03-10 长安大学 Image defogging method based on dark channel prior and bright channel prior

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919879A (en) * 2019-03-13 2019-06-21 重庆邮电大学 A kind of image defogging method based on dark channel prior Yu bright channel prior

Also Published As

Publication number Publication date
CN112837233A (en) 2021-05-25

Similar Documents

Publication Publication Date Title
CN112837233B (en) Polarization image defogging method for acquiring transmissivity based on differential polarization
CN110232666B (en) Underground pipeline image rapid defogging method based on dark channel prior
CN106910175B (en) Single image defogging algorithm based on deep learning
CN110148095B (en) Underwater image enhancement method and enhancement device
CN108596849B (en) Single image defogging method based on sky region segmentation
CN106157267B (en) Image defogging transmissivity optimization method based on dark channel prior
CN102750674B (en) Video image defogging method based on self-adapting allowance
Tripathi et al. Removal of fog from images: A review
CN102170574A (en) Real-time video defogging system
TWI489416B (en) Image recovery method
Wang et al. Variational single nighttime image haze removal with a gray haze-line prior
CN105989583B (en) A kind of image defogging method
CN112053298B (en) Image defogging method
CN111210396A (en) Multispectral polarization image defogging method combined with sky light polarization model
CN104899836A (en) Foggy image enhancing device and method based on near infrared multispectral imaging
CN113610728A (en) Polarization double-image defogging method based on four-dark-channel mean comparison
CN113487509B (en) Remote sensing image fog removal method based on pixel clustering and transmissivity fusion
CN106709876B (en) Optical remote sensing image defogging method based on dark image element principle
CN109934779B (en) Defogging method based on guided filtering optimization
CN112465720A (en) Image defogging method and device based on image sky segmentation and storage medium
CN109191405B (en) Aerial image defogging algorithm based on transmittance global estimation
CN110246102B (en) Method for clearly processing video in rainy days
CN116385293A (en) Foggy-day self-adaptive target detection method based on convolutional neural network
CN115439349A (en) Underwater SLAM optimization method based on image enhancement
CN108416739A (en) Traffic image defogging method based on non-down sampling contourlet and Markov random field

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20220909

CF01 Termination of patent right due to non-payment of annual fee