CN112435171B - Reconstruction method of image resolution - Google Patents

Reconstruction method of image resolution Download PDF

Info

Publication number
CN112435171B
CN112435171B CN202110116306.3A CN202110116306A CN112435171B CN 112435171 B CN112435171 B CN 112435171B CN 202110116306 A CN202110116306 A CN 202110116306A CN 112435171 B CN112435171 B CN 112435171B
Authority
CN
China
Prior art keywords
pixel
image
blank
pixels
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN202110116306.3A
Other languages
Chinese (zh)
Other versions
CN112435171A (en
Inventor
朱金伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Xitong Intelligent Technology Co ltd
Original Assignee
Hangzhou Xitong Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Xitong Intelligent Technology Co ltd filed Critical Hangzhou Xitong Intelligent Technology Co ltd
Priority to CN202110116306.3A priority Critical patent/CN112435171B/en
Publication of CN112435171A publication Critical patent/CN112435171A/en
Application granted granted Critical
Publication of CN112435171B publication Critical patent/CN112435171B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the technical field of image processing, in particular to a method for reconstructing image resolution, which comprises the steps of projecting pixels of an original image into an enlarged image in an equal proportion, assigning blank pixels inserted into reference pixels according to the projected reference pixels, and enabling the assignment of the blank pixels to be closer to the values of the reference pixels closer to the blank pixels during assignment so as to enable the overall sharpness of the image to be higher and avoid the excessive blurring of the image after resolution reconstruction; meanwhile, the edge position is assigned in an exponential mode, so that the RGB value of the blank pixel at the edge position can be rapidly converged, the value of the blank pixel at the edge position can be closer to the value of the reference pixel at the edge, and edge blurring and sawtooth are avoided.

Description

Reconstruction method of image resolution
Technical Field
The invention relates to the technical field of image processing, in particular to a reconstruction method of image resolution.
Background
The resolution of an image refers to the resolution of an imaging system to the details of the image, and is one of the important indicators for measuring the quality of the image. The high-resolution image can provide abundant detailed information, and with the continuous progress of economy, science and technology and civilization, the demand of the high-resolution image is increasing in various fields such as medicine, safety, entertainment and the like. For example, a physician may wish to identify a lesion by high resolution CT or B-ultrasound imaging; the public security department hopes to identify the identity of a suspect or vehicle information through a high-resolution monitoring image; entertainment businesses desire to achieve more realistic and fine visual effects for viewers through high-resolution video.
The most direct method for improving the resolution is to increase the hardware resolution of the digital image acquisition system, and the method is mainly considered from the aspects of improving the resolution of an image sensor and the resolution of a lens. However, the hardware method has technical bottleneck, and is expensive and difficult to popularize and apply. The method of improving the resolution of an image by a software method is called a super-resolution reconstruction technique, which reconstructs one or more high-resolution images using complementary information that may exist between low-resolution images. The super-resolution reconstruction technology is slowly developed at the beginning, research is mainly focused on the trial and exploration of using some classical methods (such as interpolation, regularization, least square method and the like) for the technology, and the reconstruction effect is often not ideal. Recently, with the proposition and development of theories and methods such as graph cut algorithm, sparse representation, deep learning and the like, the super-resolution reconstruction technology makes great progress under the influence of the new theories and new methods, and the combination of the new theories and the super-resolution reconstruction technology remarkably improves the reconstruction effect and the reconstruction speed.
Analyzing the existing reconstruction methods, some interpolation methods have poor effect when improving the resolution, and blur and sawtooth often occur.
Disclosure of Invention
In view of the above, the present invention provides a method for reconstructing image resolution, which can solve the problems in the background art.
The invention relates to a reconstruction method of image resolution, which comprises the following steps:
acquiring an RGB value of each pixel of an image through a computer, establishing an x-y coordinate system by taking the lower left corner of the image as a zero point, and corresponding the coordinate of each pixel point to the RGB value to obtain a pixel reference value A (x, y) of the original image, wherein the A (x, y) is the RGB value of the pixel of the original image;
establishing an x ' -y ' coordinate system of a reconstructed image by taking the lower left corner of the blank image as a zero point, projecting a pixel reference value of an original image into the coordinate system of the reconstructed image according to a resolution reconstruction proportion n, and obtaining a pixel value A ' (nx, ny) of the reconstructed image, wherein A ' (x, y) ═ A ' (nx, ny) is an RGB value of a pixel of the reconstructed image;
the blank pixels B (x ', y') between the projected pixels of the coordinate system of the reconstructed image are assigned according to the following assignment rule:
(1) when the blank pixel B (x ', y') is located on a straight line connecting the pixels a1'(x1', y1') and a2' (x2', y2') of any two reconstructed images, the blank pixel B (x ', y') is located on a straight line connecting the pixels a1'(x1', y1') and a2' (x2', y2') of any two reconstructed images
Figure GDA0002968260140000021
Figure GDA0002968260140000022
K is an offset;
(2) when the blank pixel B (x ', y') is not on the pen-direct line of any two pixels of the reconstructed image, the blank pixel B (x ', y') is not on the pen-direct line of any two pixels of the reconstructed image
Figure GDA0002968260140000023
Here, the pixel A3(x3', y3') is a pixel of a reconstructed image closest to the blank pixel B (x ', y'), and m is a pixel distance between the pixel A3(x3', y3') and the blank pixel B (x ', y').
Further, the offset K is used to adjust the pixel bias value of the reconstructed image.
Further, the pixel RGB values of the reconstructed image are converted into gray values and then assigned, and the formula of gray value conversion is gray ═ R × 0.3+ G × 0.59+ B × 0.11.
Further, any two adjacent blank pixels B1(x1', y1') and B2(x2', y2') assigned are satisfying: when B1(x1', y1') -B2(x2', y2') ≧ α, the assigned blank pixel B1(x1', y1') is decreased or the assigned blank pixel B2(x2', y2') is increased in value so that B1(x1', y1') -B2(x2', y2') < α, α is a preset threshold.
Further, n2The resolution reconstruction multiplying power is n less than or equal to 4, wherein n is a natural number.
The invention has the beneficial effects that: according to the image resolution reconstruction method, the pixels of the original image are projected into the amplified image in an equal proportion, the blank pixels inserted into the reference pixels are assigned according to the projected reference pixels, and when the blank pixels are assigned, the assignment of the blank pixels can be closer to the values of the reference pixels closer to the blank pixels, so that the overall sharpness of the image is higher, and the image after resolution reconstruction is prevented from being excessively blurred; meanwhile, the edge position is assigned in an exponential mode, so that the RGB value of the blank pixel at the edge position can be rapidly converged, the value of the blank pixel at the edge position can be closer to the value of the reference pixel at the edge, and edge blurring and sawtooth are avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and it will be apparent to those skilled in the art that other related drawings can be obtained from the drawings without inventive effort.
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a coordinate system reconstruction for pixel processing of the present invention;
FIG. 3 is a schematic representation of the reconstruction assignment of the pixel process of the present invention;
FIG. 4 is a schematic diagram of the present invention before and after resolution reconstruction.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As shown in fig. 1: the method for reconstructing the image resolution of the embodiment includes the steps:
acquiring an RGB value of each pixel of an image through a computer, establishing an x-y coordinate system by taking the lower left corner of the image as a zero point, and corresponding the coordinate of each pixel point to the RGB value to obtain a pixel reference value A (x, y) of the original image, wherein the A (x, y) is the RGB value of the pixel of the original image;
establishing an x ' -y ' coordinate system of a reconstructed image by taking the lower left corner of the blank image as a zero point, projecting a pixel reference value of an original image into the coordinate system of the reconstructed image according to a resolution reconstruction proportion n to obtain a pixel value A ' (nx, ny) of the reconstructed image, wherein A ' (nx, ny) is an RGB value of a pixel of the reconstructed image, and A (x, y) ═ A ' (nx, ny), n2For the resolution reconstruction multiplying factor, n is less than or equal to 4, that is, the reconstructed image pixels can only be 16 times of the original image at most, if the multiplying factor is too large, the number of blank pixels inserted between the reference pixels is too large, and the image can be seriously distorted after being assigned through operation.
Assigning blank pixels B (x ', y ') between projected pixels of the coordinate system of the reconstructed image, before assigning, for the convenience of calculation, converting A (x, y) and A ' (nx, ny) into gray values, before the gray values are 0-255, and converting the gray values into gray values of R0.3 + G0.59 + B0.11 according to the following assignment rule:
(1) when the blank pixel B (x ', y') is located on a straight line connecting the pixels a1'(x1', y1') and a2' (x2', y2') of any two reconstructed images, the blank pixel B (x ', y') is located on a straight line connecting the pixels a1'(x1', y1') and a2' (x2', y2') of any two reconstructed images
Figure GDA0002968260140000041
Figure GDA0002968260140000042
K is an offset amount used to adjust the pixel polarization value of the reconstructed image, the assignment of the blank pixel is determined according to the distance from the blank pixel B (x ', y') closer to the reference pixel a1'(x1', y1'), the closer to the reference pixel a1' (x1', y1'), the closer to the reference pixel a2'(x2', y2'), the closer to the reference pixel a2' (x2', y2') the blank pixel B (x ', y') is assigned, to the reference pixels a1'(x1', y1') and a2' (x2', y 2').
(2) When the blank pixel B (x ', y') is not on the pen-direct line of any two pixels of the reconstructed image, the blank pixel B (x ', y') is not on the pen-direct line of any two pixels of the reconstructed image
Figure GDA0002968260140000051
Wherein the pixel A3(x3', y3') is the pixel of the reconstructed image closest to the blank pixel B (x ', y'), and m is the pixel distance between the pixel A3(x3', y3') and the blank pixel B (x ', y'), and the assignment of the blank pixel B (x ', y') is closer to the reference pixel A3(x3', y3') and exponentially changed as the blank pixel B (x ', y') is closer to the reference pixel A3(x3', y3'), so that the edge of the image has a narrow gradient color band.
In addition, in order to avoid the phenomenon of stripes, lines and the like caused by the large color difference between blank pixels after the blank pixels are assigned after the resolution reconstruction occurs, in this embodiment, if any two adjacent assigned blank pixels B1(x1', y1') and B2(x2', y2') satisfy: when B1(x1', y1') -B2(x2', y2') ≧ α, the assigned blank pixel B1(x1', y1') is decreased or the assigned blank pixel B2(x2', y2') is increased in value so that B1(x1', y1') -B2(x2', y2') < α, specifically, an average value algorithm or a threshold value algorithm may be used, for example, an interpolated value β is defined upon occurrence
B1(x1', y1') -B2(x2', y 2'). gtoreq.alpha,
B1(x1'，y1')'＝B1(x1'，y1')-β；
B2'(x2'，y2')＝B2(x2'，y2')+β。
so that the difference is less than the threshold a.
The threshold β needs to be set according to the pixel RGB value distribution of a specific image, and is generally 5 to 15.
And finally, restoring the gray value into an RGB value.
Specifically, as shown in fig. 2-3, in the present embodiment, n is 3, according to the reconstruction method in the present invention, a coordinate system x-y is first established using the lower left corner of the original image as a zero point, for convenience, the present embodiment takes 4 pixels at the middle position for explanation, the coordinates of the 4 pixels are a1 (10000), a2(10001,10000), A3(10001,10000), and a4 (10001), and the gray values of the four pixels after gray value conversion are: a1 (10000) ═ 120; a2(10001,10000) ═ 210; a3(10001,10000) ═ 60; a4 (10001) ═ 150, for ease of illustration, the bias values for the four adjacent pixels are larger;
(1) taking a blank image with pixel initial values of 255 (namely white), establishing a coordinate system x '-y' of a reconstructed image by taking a pixel point at the lower left corner as a zero point, wherein coordinates of the four projected pixels are respectively A1 '(30000), A2' (30003,30000), A3'(30003,30000) and A4' (30003), and at the moment, 12 white pixel points B1-B12 are generated among the four reference pixels, wherein the 12 white pixel points are all located on straight connecting lines of the four reference pixels, and in addition, some white pixels are located at the edges of the four reference pixels, and are located on connecting lines of the four reference pixels and other reference pixels, which is not to be said at first;
where B1 and B2 are between A1 'and A2', with coordinates B1(30001,30000), B2(30002,30000), respectively, brought into the assignment formula, the offset value K is first set to 0,
Figure GDA0002968260140000061
Figure GDA0002968260140000062
the gray values of B1 and B2 are 150 and 180;
similarly, the gray-scale values corresponding to B3, B6, B7, B10, B11 and B12 are determined as 100, 190, 80, 170, 70 and 120, and the gray-scale values are substituted into B3, B6, B7, B10, B11 and B12;
wherein, B5 and B8 are located between A2 'and A3', B4 and B9 are located between A1 'and A4', and are substituted into the assignment formula, so that the gray values of B4, B5, B8 and B9 are respectively: 130. 160, 110 and 140, and bringing the gray values into B4, B5, B8 and B9;
(2) when the white pixels B13-B32 of the edge portions of the four reference pixels are the most marginal pixels, the edge assignment formula is used
Figure GDA0002968260140000063
Assigning values to B13-B32, and substituting gray values of A1', B1, B2, A2', B3, B7 and A3' into a formula, wherein B15 and B21, B16 and B22, B17 and B23, B18 and B24 are respectively nearest to A1', B1, B2 and A2', and the gray values of B21-B24 are respectively: 44.1, 55.2, 66.2 and 77.3, the grey values of B15-B18 are 16.2, 20.3, 24.4 and 28.4, respectively;
b13, B14, B19 and B20 are nearest to A1' and the gray values are 16.2, 16.2 and 44.1 respectively;
the gray values of B25-B32 are 16.2, 44.1, 13.5, 36.8, 10.8, 29.4, 8.1 and 22.1 respectively;
at the moment, an obvious gradual change gray black edge appears at the edge of the image where the four reference pixels are located, the whole image is surrounded, the gradual change area is narrow, edge blurring is avoided, the gradual change black edge can be changed into a gradual change white edge if needed, and only the edge needs to be changedThe assignment formula becomes B (x ', y') -a 3(x3', y3') × emWhen B (x ', y') is greater than 255, 255 is required.
It should be noted that, in the actual image, it rarely occurs that the gray values of the adjacent pixels are greatly different, so that the gray values of the filling pixels are not always greatly different as shown in the embodiment, and the excessive effect of the actual filling pixels is more natural.
And finally, if the reconstructed picture has a color cast phenomenon, assigning a value to the offset value K to enable the reconstructed picture to be more natural, and removing the gray-black gradient edge at the edge of the reconstructed picture by cutting, wherein the final effect is as shown in FIG. 4 (the left side is before reconstruction, and the right side is after reconstruction).
According to the image resolution reconstruction method, the pixels of the original image are projected into the amplified image in an equal proportion, the blank pixels inserted into the reference pixels are assigned according to the projected reference pixels, and when the blank pixels are assigned, the assignment of the blank pixels can be closer to the values of the reference pixels closer to the blank pixels, so that the overall sharpness of the image is higher, and the image after resolution reconstruction is prevented from being excessively blurred; meanwhile, the edge position is assigned in an exponential mode, so that the RGB value of the blank pixel at the edge position can be rapidly converged, the value of the blank pixel at the edge position can be closer to the value of the reference pixel at the edge, and edge blurring and sawtooth are avoided.
Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all of them should be covered in the claims of the present invention.

Claims (5)

1. A method of reconstructing image resolution, characterized by: the method comprises the following steps:
acquiring an RGB value of each pixel of an image through a computer, establishing an x-y coordinate system by taking the lower left corner of the image as a zero point, and corresponding the coordinate of each pixel point to the RGB value to obtain a pixel reference value A (x, y) of the original image, wherein the A (x, y) is the RGB value of the pixel of the original image;
establishing an x ' -y ' coordinate system of a reconstructed image by taking the lower left corner of the blank image as a zero point, projecting a pixel reference value of an original image into the coordinate system of the reconstructed image according to a resolution reconstruction proportion n, and obtaining a pixel value A ' (nx, ny) of the reconstructed image, wherein A ' (x, y) ═ A ' (nx, ny) is an RGB value of a pixel of the reconstructed image;
the blank pixels B (x ', y') between the projected pixels of the coordinate system of the reconstructed image are assigned according to the following assignment rule:
when the blank pixel B (x ', y') is located on a straight line connecting the pixels a1'(x1', y1') and a2' (x2', y2') of any two reconstructed images, the blank pixel B (x ', y') is located on a straight line connecting the pixels a1'(x1', y1') and a2' (x2', y2') of any two reconstructed images
Figure FDA0002968260130000011
Figure FDA0002968260130000012
K is an offset;
when the blank pixel B (x ', y') is not on the pen-direct line of any two pixels of the reconstructed image, the blank pixel B (x ', y') is not on the pen-direct line of any two pixels of the reconstructed image
Figure FDA0002968260130000013
Here, the pixel A3(x3', y3') is a pixel of a reconstructed image closest to the blank pixel B (x ', y'), and m is a pixel distance between the pixel A3(x3', y3') and the blank pixel B (x ', y').
2. A method of image resolution reconstruction as claimed in claim 1, wherein: the offset K is used to adjust the pixel bias value of the reconstructed image.
3. A method of image resolution reconstruction as claimed in claim 1, wherein: and converting the pixel RGB value of the reconstructed image into a gray value, and then assigning the gray value, wherein the gray value is converted into a formula of R0.3 + G0.59 + B0.11.
4. A method of image resolution reconstruction as claimed in claim 1, wherein: any two adjacent blank pixels B1(x1', y1') and B2(x2', y2') assigned are satisfied: b1(x1', y1') -B2(x2', y2') ≧ α, the assigned blank pixel B1(x1', y1') is decreased or the assigned blank pixel B2(x2', y2') is increased in value so that B1(x1', y1') -B2(x2', y2') < α, α preset threshold.
5. A method of image resolution reconstruction as claimed in claim 1, wherein: n is2The resolution reconstruction multiplying power is n less than or equal to 4, and n is a natural number.
CN202110116306.3A 2021-01-28 2021-01-28 Reconstruction method of image resolution Expired - Fee Related CN112435171B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110116306.3A CN112435171B (en) 2021-01-28 2021-01-28 Reconstruction method of image resolution

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110116306.3A CN112435171B (en) 2021-01-28 2021-01-28 Reconstruction method of image resolution

Publications (2)

Publication Number Publication Date
CN112435171A CN112435171A (en) 2021-03-02
CN112435171B true CN112435171B (en) 2021-04-20

Family

ID=74697381

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110116306.3A Expired - Fee Related CN112435171B (en) 2021-01-28 2021-01-28 Reconstruction method of image resolution

Country Status (1)

Country Link
CN (1) CN112435171B (en)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4053021B2 (en) * 2004-05-12 2008-02-27 三洋電機株式会社 Image enlargement apparatus and program
CN102622723A (en) * 2011-05-25 2012-08-01 上海大学 Image interpolation based on CUDA (compute unified device architecture) and edge detection
CN103020657A (en) * 2012-12-28 2013-04-03 沈阳聚德视频技术有限公司 License plate Chinese character recognition method
CN107194877A (en) * 2017-06-16 2017-09-22 南京大学金陵学院 A kind of guarantor side interpolation super-resolution computational methods based on single image
CN109903224B (en) * 2019-01-25 2023-03-31 珠海市杰理科技股份有限公司 Image scaling method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN112435171A (en) 2021-03-02

Similar Documents

Publication Publication Date Title
CN110163237B (en) Model training and image processing method, device, medium and electronic equipment
CN103034973B (en) Based on the adaptive image scaling method of bicubic interpolation
Hwang et al. Adaptive image interpolation based on local gradient features
US8520009B1 (en) Method and apparatus for filtering video data using a programmable graphics processor
TWI406571B (en) Image Processing Method for Enhancing Image Edge Resolution
JP4498361B2 (en) How to speed up Retinex-type algorithms
DE102015113240A1 (en) SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT FOR SHADING USING A DYNAMIC OBJECT ROOM GATE
JP4443064B2 (en) Method and apparatus for performing local color correction
CN107993208A (en) It is a kind of based on sparse overlapping group prior-constrained non local full Variational Image Restoration method
WO2020098721A1 (en) Image presentation method and apparatus for virtual reality device, device and computer storage medium
CN113793272B (en) Image noise reduction method and device, storage medium and terminal
CN107292819A (en) A kind of infrared image super resolution ratio reconstruction method protected based on edge details
CN111429371A (en) Image processing method and device and terminal equipment
KR20100077307A (en) Image texture filtering method, storage medium of storing program for executing the same and apparatus performing the same
DE102020133244A1 (en) Edge-conscious upscaling for improved screen content quality
CN104063875B (en) Strengthen video image smoothness and the ultra-resolution ratio reconstructing method of definition
US20030001862A1 (en) Method for the minimization of artifacts in full frame animations transferred to NTSC interlaced video
CN113837945A (en) Display image quality optimization method and system based on super-resolution reconstruction
JP5316385B2 (en) Image processing apparatus and image processing program
CN112435171B (en) Reconstruction method of image resolution
CN115311145B (en) Image processing method and device, electronic equipment and storage medium
CN108492264B (en) Single-frame image fast super-resolution method based on sigmoid transformation
CN114240785A (en) Denoising method and system for ray tracing rendering continuous frames
WO2020241337A1 (en) Image processing device
CN113689333A (en) Image enhancement method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20210420

Termination date: 20220128

CF01 Termination of patent right due to non-payment of annual fee