CN112862700B - Hyperspectral remote sensing image blind denoising method based on noise attention - Google Patents
Hyperspectral remote sensing image blind denoising method based on noise attention Download PDFInfo
- Publication number
- CN112862700B CN112862700B CN202110058699.7A CN202110058699A CN112862700B CN 112862700 B CN112862700 B CN 112862700B CN 202110058699 A CN202110058699 A CN 202110058699A CN 112862700 B CN112862700 B CN 112862700B
- Authority
- CN
- China
- Prior art keywords
- noise
- network
- denoising
- noise intensity
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000000605 extraction Methods 0.000 claims description 13
- 230000001629 suppression Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 12
- 230000002708 enhancing effect Effects 0.000 abstract 1
- 230000002401 inhibitory effect Effects 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000011084 recovery Methods 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000004088 simulation Methods 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013256 Gubra-Amylin NASH model Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012733 comparative method Methods 0.000 description 1
- 238000011109 contamination Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10036—Multispectral image; Hyperspectral image
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a hyperspectral remote sensing image blind denoising method based on noise attention, which comprises the steps of obtaining noise intensities of different wave bands by constructing a noise estimation network, inhibiting a high-noise wave band by using an attention mechanism, and enhancing a low-noise wave band; and a denoising network is built again, useless information in the data is removed as much as possible, useful information is obtained, the denoising effect is improved, and the accuracy and the applicability of blind denoising are further improved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a hyperspectral remote sensing image blind denoising method.
Background
The hyperspectral remote sensing image originates from the early 60 th century, is a fusion product of various high-end scientific technologies such as information processing technology, space measurement technology and computer science technology, and is widely applied to the fields of military exploration, planetary exploration, ground object classification and the like. Because hyperspectral images contain a large amount of ground object information, in order to be able to extract more hyperspectral application values, people usually process and analyze the obtained hyperspectral data, thereby obtaining interesting and valuable information. In order to obtain the above information, one would need a high quality hyperspectral image, so that the image can be more realistic as much as possible, and the result of further analysis is maximized. Unfortunately, however, almost all hyperspectral images are inevitably contaminated by noise during transmission, and the reasons for such contamination are also all aspects, and the factors of the devices are also interference of the external environment, and it is these useless interference noise due to doping that creates a great challenge for the subsequent research work of the images, and even greatly reduces the reliability of the processing technology. Therefore, developing popular and feasible hyperspectral image denoising methods is an academic goal of many scientific researchers, and the hyperspectral denoising field is also an important research direction in the academic world.
Over the past decade, many image restoration algorithms have been proposed, and the most common hyperspectral image denoising algorithm is to perform denoising processing on a band-by-band basis, and the algorithms can be divided into a conventional algorithm and a deep learning-based algorithm. In the literature M.Maggioni, V.Katkovnik, K.Egiazarian, nonlocal transform-domain filter for volumetric data denoising and reconstruction, in IEEE Transactions on Image Processing and 2013, the maggioni et al propose a non-local self-similar denoising method, which adopts wavelet change to perform threshold segmentation on the basis of three-dimensional block self-similarity, and then performs secondary denoising, thereby obtaining a satisfactory denoising effect. Zhang et al in documents H.Zhang, W.He, L.Zhang, hyperspectral image restoration using low-rank matrix recovery, in IEEE Transactions on Geoscience and Remote Sensing,2014 propose a denoising method based on low rank recovery, which directly learns images based on rank constraint, so that the effect of image recovery can be improved according to the constraint on the rank of an image matrix.
However, both the self-similar method and the low rank matrix recovery method have limitations. First, there are a large number of manually set parameters to control how good the denoising effect is. Second, the algorithm has a large computational complexity, and the processing of high-dimensional hyperspectral data is time-consuming.
With the continuous development of computer science and technology in recent years, deep learning has made a significant breakthrough in the field of machine learning, and has achieved a great improvement in the fields of image segmentation, image detection and recognition, image classification and the like, so that the deep learning is also applied to the field of hyperspectral image denoising to improve the image denoising effect. In the documents Y.Chang, L.Yan, H.Fang, HSI-DeNet Hyperspectral image restoration via convolutional neural network, in IEEE Transactions on Geoscience and Remote Sensing and 2019, chang et al propose a simple 18-layer stacked neural network for denoising and recovering hyperspectral images, and adopts a GAN model for global image correction, so that a good recovery effect is obtained. On the basis of the Q.Yuan et al, in the literature of Q.Yuan, Q.Zhang, J.Li, et al, hyperspectral image denoising employing a spatial-spectral deep residual convolutional neural network, in IEEE Transactions on Geoscience and Remote Sensing and 2019, a multi-scale feature extraction denoising method combining multi-spectrum and single spectrum is proposed, and the surrounding multi-spectrum channel information is added to improve the prior information supplement of an image when a single channel is denoised, so that the overall denoising effect is improved.
However, the above methods all have respective limitations. First, there are a large number of super parameters in the conventional algorithm that need to be corrected manually, and a large number of parameter adjustments are often needed for different data sets to obtain a better effect. Second, existing deep learning methods cannot well suppress non-uniform noise, and when the noise intensities of different wavebands are not uniform, the denoising robustness of these methods is poor.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a hyperspectral remote sensing image blind denoising method based on noise attention, which acquires noise intensities of different wave bands by constructing a noise prediction network, suppresses a high-noise wave band by using an attention mechanism and enhances a low-noise wave band; and a denoising network is built again, useless information in the data is removed as much as possible, useful information is obtained, the denoising effect is improved, and the accuracy and the applicability of blind denoising are further improved.
The technical scheme adopted by the invention for solving the technical problems comprises the following steps:
step 1: constructing a noise intensity estimation network based on multi-layer convolution;
the noise intensity estimation network is formed by directly connecting 15 3×3 convolutions in series; the input of the network is H multiplied by W multiplied by C noise images, the length and width of the noise images are H and W respectively, and the number of channels is C; the output of the network is a noise intensity predictive image of H multiplied by W multiplied by C, the length and width of the noise intensity predictive image are H and W respectively, and the number of channels is C; the noise intensity predictive image is used for predicting the noise intensity of each pixel in the noise image;
step 2: constructing a denoising network;
the Denoising network is formed by sequentially connecting four denoise blocks and a Concat layer in series, wherein each denoise Block comprises four multi-scale feature extraction units and a Concat layer;
the output of the first denoise Block is sent to each denoise Block and the Concat layer of the Denoising network; the output of the second and third de-noising blocks is sent to a Concat layer of the de-noising network in addition to the output of the subsequent de-noising blocks; the output of the fourth de-noising Block is sent to a Concat layer of the de-noising network;
the multi-scale feature extraction unit is formed by connecting a plurality of expansion convolutions; the features extracted by each multi-scale feature extraction unit are sent into a Concat layer inside a denoise Block and a subsequent multi-scale feature extraction unit;
step 3: carrying out noise intensity estimation on hyperspectral single wave bands and surrounding K wave bands of the noise image by using a noise intensity estimation network, and obtaining a noise intensity estimation graph of the estimated wave bands;
step 4: using an attention mechanism to perform high-noise suppression and low-noise enhancement on the noise intensity estimated graph of the estimated wave band obtained in the step 3;
step 5: inputting the noise intensity predictive image of the estimated wave band processed in the step 4 into the denoising network constructed in the step 2, and removing noise;
step 6: and (3) converting the wave bands of noise intensity estimation in the step (3), and repeatedly executing the steps (3) to (6) until the denoising of all the wave bands is completed, and finally, blind denoising of hyperspectral remote sensing data is achieved.
The beneficial effects of the invention are as follows:
1. the denoising effect is better. According to the method, through the noise pre-estimated attention network, the noise intensity of different wave bands can be accurately obtained, and the attention module is utilized to accurately inhibit and enhance the different wave bands, so that useless information in data is removed as much as possible, useful information is obtained, and the denoising effect is improved.
2. Blind denoising is better in robustness. According to the method, the noise prediction network is introduced, so that the respective band noise level can be obtained for any data set, the network self-adaption has good blind denoising effect on different data sets, and the blind denoising robustness is stronger.
3. The denoising process is more efficient. The denoising network of the method has better noise suppression capability, and has fewer model parameters and algorithm complexity, thereby improving the denoising quality and efficiency.
Drawings
FIG. 1 is a block flow diagram of the method of the present invention.
Fig. 2 is a block diagram of a denoising network according to the method of the present invention.
Detailed Description
The invention will be further described with reference to the drawings and examples.
As shown in fig. 1, a hyperspectral remote sensing image blind denoising method based on noise attention comprises the following steps:
step 1: constructing a noise intensity estimation network based on multi-layer convolution;
the noise intensity estimation network is formed by directly connecting 15 3×3 convolutions in series; the input of the network is H multiplied by W multiplied by C noise images, the length and width of the noise images are H and W respectively, and the number of channels is C; the output of the network is a noise intensity predictive image of H multiplied by W multiplied by C, the length and width of the noise intensity predictive image are H and W respectively, and the number of channels is C; the noise intensity predictive image is used for predicting the noise intensity of each pixel in the noise image;
step 2: constructing a denoising network;
the Denoising network is formed by sequentially connecting four denoise blocks and a Concat layer in series, wherein each denoise Block comprises four multi-scale feature extraction units and a Concat layer;
the output of the first denoise Block is sent to each denoise Block and the Concat layer of the Denoising network; the output of the second and third de-noising blocks is sent to a Concat layer of the de-noising network in addition to the output of the subsequent de-noising blocks; the output of the fourth de-noising Block is sent to a Concat layer of the de-noising network;
the multi-scale feature extraction unit is formed by connecting a plurality of expansion convolutions; the features extracted by each multi-scale feature extraction unit are sent into a Concat layer inside a denoise Block and a subsequent multi-scale feature extraction unit;
step 3: carrying out noise intensity estimation on hyperspectral single wave bands and surrounding K wave bands of the noise image by using a noise intensity estimation network, and obtaining a noise intensity estimation graph of the estimated wave bands;
step 4: using an attention mechanism to perform high-noise suppression and low-noise enhancement on the noise intensity estimated graph of the estimated wave band obtained in the step 3;
step 5: inputting the noise intensity predictive image of the estimated wave band processed in the step 4 into the denoising network constructed in the step 2, and removing noise;
step 6: and (3) converting the wave bands of noise intensity estimation in the step (3), and repeatedly executing the steps (3) to (6) until the denoising of all the wave bands is completed, and finally, blind denoising of hyperspectral remote sensing data is achieved.
Specific examples:
1. simulation conditions
The embodiment is that the CPU isSimulation is carried out on the i 5-3470.2 GHz CPU and the memory 16G, WINDOWS operating system by using MATLAB software.
The data set used in the experiments was fromThe set of Washington DC Mall remote sensing data of the institution, whose website is https:// engineering. Purdue. Edu/-biehl/MultiSpec/hyperspectral. Html, contains a piece of hyperspectral remote sensing data of 1280 in length, 303 in width, and 191 bands, on which the present embodiment is based on experiments.
2. Emulation content
Clustering experiments were performed on the dataset. In order to compare the effectiveness of the present invention, the non-local self-similar denoising method (BM 4D) mentioned in the "m. maggioni et al, documents M.Maggioni, V.Katkovnik, K.Egiazarian, nonlocal transform-domain filter for volumetric data denoising and reconstruction, in IEEE Transactions on Image Processing,2013," the low rank representation denoising method (LRMR) mentioned in the "h.zhang et al, documents H.Zhang, W.He, L.Zhang, hyperspectral image restoration using low-rank matrix recovery, in IEEE Transactions on Geoscience and Remote Sensing,2014," and the depth denoising method (HSID-CNN) mentioned in the "q.yuan et al, documents Q.Yuan, Q.Zhang, J.Li, et al, hyperspectral image denoising employing a spatial-spectral deep residual convolutional neural network, in IEEE Transactions on Geoscience and Remote Sensing,2019," and other methods were selected for comparison with the inventive method, and the indexes such as average peak signal to noise ratio (MPSNR), average structural similarity (MSSIM) were used in the present experiment to evaluate the quality of denoising. The comparison results are shown in Table 1 (highest values are indicated by bold).
Table 1 comparison of experimental results (σ=10-70)
| Method | LRMR | LRTV | BM4D | HSID-CNN | HSI-SDeCNN | The method |
| MPSNR | 30.88 | 30.64 | 27.59 | 30.63 | 30.45 | 32.86 |
| MSSIM | 0.970 | 0.971 | 0.933 | 0.966 | 0.966 | 32.98 |
As can be seen from table 1, the blind denoising performance of the present method is superior to other comparative methods. The effectiveness of the invention can be verified through the simulation experiment.
Claims (1)
1. A hyperspectral remote sensing image blind denoising method based on noise attention is characterized by comprising the following steps:
step 1: constructing a noise intensity estimation network based on multi-layer convolution;
the noise intensity estimation network is formed by directly connecting 15 3×3 convolutions in series; the input of the network is H multiplied by W multiplied by C noise images, the length and width of the noise images are H and W respectively, and the number of channels is C; the output of the network is a noise intensity predictive image of H multiplied by W multiplied by C, the length and width of the noise intensity predictive image are H and W respectively, and the number of channels is C; the noise intensity predictive image is used for predicting the noise intensity of each pixel in the noise image;
step 2: constructing a denoising network;
the Denoising network is formed by sequentially connecting four denoise blocks and a Concat layer in series, wherein each denoise Block comprises four multi-scale feature extraction units and a Concat layer;
the output of the first denoise Block is sent to each denoise Block and the Concat layer of the Denoising network; the output of the second and third de-noising blocks is sent to a Concat layer of the de-noising network in addition to the output of the subsequent de-noising blocks; the output of the fourth de-noising Block is sent to a Concat layer of the de-noising network;
the multi-scale feature extraction unit is formed by connecting a plurality of expansion convolutions; the features extracted by each multi-scale feature extraction unit are sent into a Concat layer inside a denoise Block and a subsequent multi-scale feature extraction unit;
step 3: carrying out noise intensity estimation on hyperspectral single wave bands and surrounding K wave bands of the noise image by using a noise intensity estimation network, and obtaining a noise intensity estimation graph of the estimated wave bands;
step 4: using an attention mechanism to perform high-noise suppression and low-noise enhancement on the noise intensity estimated graph of the estimated wave band obtained in the step 3;
step 5: inputting the noise intensity predictive image of the estimated wave band processed in the step 4 into the denoising network constructed in the step 2, and removing noise;
step 6: and (3) converting the wave bands of noise intensity estimation in the step (3), and repeatedly executing the steps (3) to (6) until the denoising of all the wave bands is completed, and finally, blind denoising of hyperspectral remote sensing data is achieved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110058699.7A CN112862700B (en) | 2021-01-16 | 2021-01-16 | Hyperspectral remote sensing image blind denoising method based on noise attention |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110058699.7A CN112862700B (en) | 2021-01-16 | 2021-01-16 | Hyperspectral remote sensing image blind denoising method based on noise attention |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112862700A CN112862700A (en) | 2021-05-28 |
| CN112862700B true CN112862700B (en) | 2023-10-27 |
Family
ID=76005881
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110058699.7A Active CN112862700B (en) | 2021-01-16 | 2021-01-16 | Hyperspectral remote sensing image blind denoising method based on noise attention |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112862700B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113436111B (en) * | 2021-07-21 | 2024-01-09 | 西北工业大学 | Hyperspectral remote sensing image denoising method based on network structure search |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140145801A (en) * | 2013-06-14 | 2014-12-24 | 전북대학교산학협력단 | System and method for improving quality in images using neural network |
| CN109658348A (en) * | 2018-11-16 | 2019-04-19 | 天津大学 | The estimation of joint noise and image de-noising method based on deep learning |
| CN110516728A (en) * | 2019-08-20 | 2019-11-29 | 西安电子科技大学 | Polarization SAR terrain classification method based on denoising convolutional neural networks |
| CN110570364A (en) * | 2019-08-05 | 2019-12-13 | 天津大学 | depth neural network-based sub-focal plane polarization image denoising method |
| CN111260591A (en) * | 2020-03-12 | 2020-06-09 | 武汉大学 | Image self-adaptive denoising method based on attention mechanism |
| CN111292259A (en) * | 2020-01-14 | 2020-06-16 | 西安交通大学 | Deep learning image denoising method integrating multi-scale and attention mechanism |
| CN111931738A (en) * | 2020-09-28 | 2020-11-13 | 南京邮电大学 | Neural network model pre-training method and device for remote sensing image |
| CN112233026A (en) * | 2020-09-29 | 2021-01-15 | 南京理工大学 | SAR image denoising method based on multi-scale residual attention network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10997690B2 (en) * | 2019-01-18 | 2021-05-04 | Ramot At Tel-Aviv University Ltd. | Method and system for end-to-end image processing |
-
2021
- 2021-01-16 CN CN202110058699.7A patent/CN112862700B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140145801A (en) * | 2013-06-14 | 2014-12-24 | 전북대학교산학협력단 | System and method for improving quality in images using neural network |
| CN109658348A (en) * | 2018-11-16 | 2019-04-19 | 天津大学 | The estimation of joint noise and image de-noising method based on deep learning |
| CN110570364A (en) * | 2019-08-05 | 2019-12-13 | 天津大学 | depth neural network-based sub-focal plane polarization image denoising method |
| CN110516728A (en) * | 2019-08-20 | 2019-11-29 | 西安电子科技大学 | Polarization SAR terrain classification method based on denoising convolutional neural networks |
| CN111292259A (en) * | 2020-01-14 | 2020-06-16 | 西安交通大学 | Deep learning image denoising method integrating multi-scale and attention mechanism |
| CN111260591A (en) * | 2020-03-12 | 2020-06-09 | 武汉大学 | Image self-adaptive denoising method based on attention mechanism |
| CN111931738A (en) * | 2020-09-28 | 2020-11-13 | 南京邮电大学 | Neural network model pre-training method and device for remote sensing image |
| CN112233026A (en) * | 2020-09-29 | 2021-01-15 | 南京理工大学 | SAR image denoising method based on multi-scale residual attention network |
Non-Patent Citations (1)
| Title |
|---|
| 基于低秩字典学习的高光谱遥感图像去噪;张静妙;高双喜;王晓娜;;控制工程(第06期);823-827 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112862700A (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ye et al. | Deep convolutional framelets: A general deep learning framework for inverse problems | |
| Chen et al. | Denoising hyperspectral image with non-iid noise structure | |
| Liu et al. | A 3-D atrous convolution neural network for hyperspectral image denoising | |
| CN112233026A (en) | SAR image denoising method based on multi-scale residual attention network | |
| Singer et al. | Diffusion interpretation of nonlocal neighborhood filters for signal denoising | |
| CN111369457B (en) | Remote sensing image denoising method for sparse discrimination tensor robustness PCA | |
| CN109658351B (en) | Hyperspectral image denoising method combining L0 gradient constraint and local low-rank matrix recovery | |
| CN110598584A (en) | Convolutional neural network face recognition algorithm based on wavelet transform and DCT | |
| CN111368691B (en) | Unsupervised hyperspectral remote sensing image space spectrum feature extraction method | |
| CN113379618B (en) | Optical remote sensing image cloud removing method based on residual dense connection and feature fusion | |
| CN112862700B (en) | Hyperspectral remote sensing image blind denoising method based on noise attention | |
| Liao et al. | A fast iterative kernel PCA feature extraction for hyperspectral images | |
| Dhar et al. | Accurate segmentation of complex document image using digital shearlet transform with neutrosophic set as uncertainty handling tool | |
| Deng et al. | t-Linear tensor subspace learning for robust feature extraction of hyperspectral images | |
| Zhong et al. | Multiscale feature extraction based on convolutional sparse decomposition for hyperspectral image classification | |
| Sayed et al. | Image object extraction based on curvelet transform | |
| Rakvongthai et al. | Statistical texture retrieval in noise using complex wavelets | |
| CN113436111B (en) | Hyperspectral remote sensing image denoising method based on network structure search | |
| Tu et al. | Texture pattern separation for hyperspectral image classification | |
| CN115689993A (en) | Skin cancer image segmentation method and system based on attention and multi-feature fusion | |
| CN112927151B (en) | Image denoising method based on edge enhancement and convolutional neural network | |
| Geetha et al. | Savitzky golay and kpca based optimal discrete wavelet transform architecture for image compression | |
| Hou et al. | Deep Diversity-Enhanced Feature Representation of Hyperspectral Images | |
| Kun et al. | Color image detail enhancement based on quaternion guided filter | |
| CN114495037A (en) | Video prediction method and system based on key points and Kalman filtering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |