CN104966276B - A kind of conformal projection sparse expression method of image/video scene content - Google Patents

A kind of conformal projection sparse expression method of image/video scene content Download PDF

Info

Publication number
CN104966276B
CN104966276B CN201510337089.5A CN201510337089A CN104966276B CN 104966276 B CN104966276 B CN 104966276B CN 201510337089 A CN201510337089 A CN 201510337089A CN 104966276 B CN104966276 B CN 104966276B
Authority
CN
China
Prior art keywords
msub
dictionary
image
mrow
conformal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510337089.5A
Other languages
Chinese (zh)
Other versions
CN104966276A (en
Inventor
陈小武
李健伟
邹冬青
赵沁平
高博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN201510337089.5A priority Critical patent/CN104966276B/en
Publication of CN104966276A publication Critical patent/CN104966276A/en
Application granted granted Critical
Publication of CN104966276B publication Critical patent/CN104966276B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The present invention provides a kind of conformal projection sparse expression method of image/video scene content, comprises the following steps:First, original image or video are inputted and is sampled in feature space;2nd, calculate the k nearest neighbor of each sample and set up part and completely abut against figure, calculate the distance between adjacent sample;3rd, according to conformal projection rule, it is combined with sparse expression method, dictionary of the study with conformal property;4th, original image or video are reconstructed using this dictionary.The present invention maintains the angle information between adjacent sample, obtains the stronger dictionary of ability to express to greatest extent by introducing conformal projection rule.Meanwhile, conformal projection promotes adjacent sample to be reconstructed with similar dictionary, makes dictionary more concision and compact.It has broad application prospects in image procossing, computer vision and augmented reality field.

Description

A kind of conformal projection sparse expression method of image/video scene content
Technical field
The present invention relates to image procossing, computer vision and augmented reality field, specifically a kind of image is regarded The conformal projection sparse expression method of frequency scene content.
Background technology
In the last few years, sparse expression and dictionary learning technology had largely been paid close attention to as a study hotspot, and extensively Applied to image procossing and computer vision field, such as image super-resolution, image denoising, classification and color editor.It is dilute Thin expression technology is to reconstruct the linear combination of sample in the used complete dictionary of signal, and limit the number of reconstructed sample with up to To sparse property.
At present, many researchers are directed to the research of sparse expression method, and dictionary rises in sparse expression technology To very important effect.Michal Aharon et al. proposed K-SVD dictionary learnings method in 2006 and applied to image Processing.Honglak Lee et al. proposed a kind of rapid sparse coding method in 2006, accelerated solving speed.Mairal Et al. the online dictionary learning method that was proposed in 2009 based on stochastic approximation, this method can effectively handle large data sets. The method for solving and operational efficiency that focus on sparse expression of these methods.These methods are absorbed in the re-configurability of dictionary, But it need to rely on substantial amounts of training sample.Also, the dictionary number of these methods needs manual set, it is impossible to which adjust automatically is big It is small, so as to get dictionary redundancy.Other sparse expression methods obtain certain achievement in terms of the tight ness rating and expressivity of dictionary. For example, the action attributes dictionary learning method that Qiu et al. was proposed based on maximum mutual information in 2011;Siyahjani et al. Context-aware dictionary was proposed in 2013 and for the identification and positioning of image object.These dictionary learning methods are added Otherness between classification, but the local relation in data space and contextual information are not accounted for, cause the expression energy of dictionary Power is low.And some researchs show, the partial structurtes relation between data inside is kept to strengthen fidelity in data reconstruction Degree, it is to avoid the occurrence of distortion.
Sparse expression technology is increasingly being applied to image procossing and computer vision field.For example, Elad et al. will K-SVD methods are used for image denoising;Yang et al. was proposed with sparse expression method in 2010 while learning high-resolution With the method for two dictionaries of low resolution, and for image super-resolution;Chen et al. proposed to utilize sparse expression in 2014 Technology enters the theory of edlin propagation, can handle the image/video of ultrahigh resolution and greatly reduce calculating internal memory.In addition, Sparse expression technology can also be applied in terms of recognition of face, image recovery, image classification.And the processing procedure of above-mentioned application In, the higher result of generation eye fidelity is still the emphasis of sparse expression technical research.
The content of the invention
In order to overcome the above-mentioned deficiencies of the prior art, the present invention proposes a kind of conformal projection of image/video scene content Sparse expression method, this method maintains the local angle letter between adjacent sample by introducing conformal projection, to greatest extent Breath, and obtain the stronger dictionary of ability to express.Meanwhile, conformal projection promotes adjacent sample to be reconstructed with similar dictionary, makes Dictionary more concision and compact.Finally, the reconstruction result after picture editting is made preferably to keep original partial structurtes, enhancing generation knot The visual effect and the sense of reality of fruit.
To complete goal of the invention, the technical solution adopted by the present invention is:
A kind of conformal projection sparse expression method of image/video scene content of the present invention, it is comprised the following steps that:
Step one:Input original image or video are simultaneously sampled in feature space;
Step 2:In feature space, calculate the k nearest neighbor of each sample and set up it is local completely abut against figure, then calculate The distance between adjacent sample;
Step 3:According to conformal projection rule, it is combined with sparse expression method, word of the study with conformal property Allusion quotation;
Step 4:For concrete application, original image or video are reconstructed using this dictionary, result is obtained.
Wherein, " locally completely abutting against figure " described in step 2, refers to be constituted for certain sample and its k nearest neighbor Set in, be connected between any two sample.
Wherein, " the conformal projection rule " described in step 3, is a kind of manifold learning, is specifically described as:Give Feature space M is determined to another feature space N mapping g:M → N, (xi,xj,xk) it is adjacent sample point in feature space M and structure Triangularity, (αijk) it is mapping of these sample points in feature space N.Needed to meet according to conformal projection rule:
Wherein, NiRepresent sample xiK nearest neighbor set, siRepresent the change of scale of mapping.
Wherein, dictionary of the study with conformal property is combined with sparse expression method described in step 3, specifically Step is:Conformal projection rule is combined with sparse expression algorithm, following energy theorem is obtained:
Wherein, x is input sample feature, and D is characterized dictionary, and α is reconstruction coefficients, λ1、λ2For weight coefficient, pass through iteration This energy theorem of algorithmic minimizing, finally tries to achieve the dictionary D with conformal property.
Wherein, the method can apply to the video images such as image super-resolution, video image color editor, image denoising Editor's application.
Compared with prior art, the characteristics of its is beneficial is the present invention:
1st, in sparse expression technical foundation, by introducing conformal projection rule, adjacent sample is maintained to greatest extent Between local angle information, obtain the stronger dictionary of ability to express;By conformal projection, promote adjacent sample with similar word Allusion quotation is reconstructed, and makes dictionary more concision and compact.
2nd, the more succinct and stronger dictionary of ability to express is benefited from, the present invention makes the reconstruction result after picture editting more Original partial structurtes, the visual effect and the sense of reality of enhancing generation result are kept well.
3rd, method proposed by the present invention can apply to many fields and effect is notable, including:Image super-resolution, video Color of image editor, image denoising etc..
Brief description of the drawings
Fig. 1 is the method for the invention flow chart;
Fig. 2 is the principle schematic of the present invention;
Fig. 3 is the dictionary learning total algorithm flow chart of the present invention;
Symbol description is as follows in figure:
D:The dictionary learnt under particular feature space;
A:Reconstruction coefficients;
S:Change of scale coefficient;
xi,xj,xk:The sample characteristics of the sample point of input, i.e. image/video;
αijk:It is mapped to the sample point in another space, i.e., sparse reconstruction coefficients.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, below in conjunction with the accompanying drawings, to the present invention's Method is explained in detail explanation.It should be appreciated that instantiation described herein is not used to limit only to explain the present invention The fixed present invention.
The present invention proposes a kind of conformal projection sparse expression method of image/video scene content, and the method is protected by introducing Angle mapping ruler, maintains the local angle information between adjacent sample to greatest extent, obtains more succinct and expresses energy The stronger dictionary of power;The dictionary generated using the method carries out video image editor, and its reconstruction result can preferably keep original Partial structurtes, the visual effect and the sense of reality of enhancing generation result.Meanwhile, three typical case's applications are applied this method to, including Image super-resolution, video image color editor, image denoising.
A kind of conformal projection sparse expression method of image/video scene content of the present invention, flow are as shown in figure 1, specific real Apply mode as follows:
Step one:Input original image or video are simultaneously sampled in feature space.
The original image or video of input are sampled, input sample collection X is obtained.Chosen according to no application demand Different feature spaces.For example, for image super-resolution application, image is transformed into Ycbcr colors sky from RGB color Between, the luminance channel Y of image is sampled in patch ranks.For color editor application, to RGB in pixel scale Color characteristic is sampled;For image denoising application, gray feature or RGB color feature are adopted in patch ranks Sample.
Step 2:In feature space, calculate the k nearest neighbor of each sample and set up it is local completely abut against figure, then calculate The distance between adjacent sample.
Calculate each sample x in feature space with Kd-tree methods firstiK nearest neighbor, used during calculating European Distance, in sample xiAnd its in the set of K neighbour's sample composition, connection each two sample constitutes part and completely abuts against figure; The Euclidean distance between connection sample is calculated in feature space.
Step 3:According to conformal projection rule, it is combined with sparse expression method, word of the study with conformal property Allusion quotation.
Given input sample collection X=[x1,x2,…,xN], using sparse expression method, can in the hope of excessively complete dictionary D, with And reconstruction coefficients α:
In order to improve the performance of sparse expression method, invention introduces the partial structurtes information of input data, above-mentioned Conformal f (α) is added on the basis of formula.
Conformal projection has proven to that manifold learning effect can be improved in manifold learning field.Specific method is: Given feature space M to another feature space N mapping g:M → N, (xi,xj,xk) be adjacent sample point in feature space M simultaneously It is triangle, (αijk) it is mapping of these sample points in feature space N, as shown in Figure 2.Advised according to conformal projection Then need to meet:
Wherein, NiRepresent sample xiK nearest neighbor set, siRepresent the change of scale after mapping.
Then, conformal projection rule is combined with sparse expression algorithm, obtains following energy theorem:
Wherein, x is input sample feature, and D is characterized dictionary, and α is reconstruction coefficients, λ1、λ2For weight coefficient.Pass through iteration This energy theorem of algorithmic minimizing, finally tries to achieve the dictionary D with conformal property.
Above-mentioned formula has three unknown variables (D, α, S), and wherein D is dictionary to be asked, and α is sparse reconstruction coefficients, and S is chi Degree conversion.Therefore the present invention is broken down into three subproblems:Sparse coding, dictionary updating, yardstick updates.In each subproblem During solution, only optimize a variable and fixed other two variables.This three continuous loop iterations of step are until obtaining optimal solution.
Firstly, it is necessary to which initializing variable D and S value are random matrix.In the sparse coding stage, fixed D and S value is led to Cross equation below and solve factor alpha:
Herein, the present invention solves this formula using iterative projection method.
Then, in the dictionary updating stage, fix α and S value to solve D, solution formula is:
Here each single item d in dictionary is requiredjFor unit vector, that is, meetThis formula is asked for quadratic programming Topic, each single item in dictionary can be updated item by item.
Finally, in the yardstick more new stage, fix D and α to solve S, solution formula is:
Notice each s in above-mentioned formulaiAll it is independent, therefore can be respectively solved by least square method.Ask Solution method is:
By the continuous iteration optimization of this three processes, optimal solution is finally tried to achieve.Algorithm flow chart is shown in Fig. 3.
Step 4:Original image or video are reconstructed using this dictionary, result is obtained.
The present invention provides three kinds of different applications to verify the performance of the method, including image super-resolution, video image Color editor, image denoising.
Image super-resolution application be by the Image Reconstruction of low resolution be high-resolution image.Initially set up one a pair The high-definition picture answered and low-resolution image storehouse, using above-mentioned dictionary learning method simultaneously from two dictionaries of storehouse learning. When the image of one low resolution of input, it is reconstructed using low-resolution dictionary and tries to achieve coefficient, then usage factor and height Resolution ratio dictionary reconstructs corresponding high-definition picture.
Video image color editor application is the colouring information for changing video image by interactive mode.Inputted video image Afterwards, learn its color dictionary first, when user is by paintbrush marker color on image object, corresponding color in dictionary The color of user's mark can be changed into, while this change can travel to whole video image, final color edited result is obtained.
Image denoising application is to filter out the Gaussian noise on image.An image with noise is inputted, 8*8 is gathered first The image block of size, and learn as data dictionary.Then match tracing method reconstructed image is utilized, noise filtering is obtained Image afterwards.
The dictionary tried to achieve using the present invention has good presentation skills and re-configurability, while dictionary is also more succinct. By relatively can prove that this point with conventional method.Such as traditional dictionary learning method K-SVD, the dictionary size tried to achieve is 256, the present invention can be reduced to 205, and ability to express is stronger.Can by the coefficient correlation of dictionary internal come The ability to express of this dictionary is represented, the smaller ability to express of coefficient is stronger.The phase relation for the dictionary that traditional sparse expression method is tried to achieve Number is 0.8817, and present invention introduces conformal projection after, coefficient correlation is reduced to 0.8477, illustrate that the present invention learns obtained word Allusion quotation has stronger learning ability.
Some basic explanations of the present invention are the foregoing is only, any equivalent change done according to technical scheme Change, protection scope of the present invention all should be belonged to.

Claims (5)

1. a kind of conformal projection sparse expression method of image/video scene content, it is characterised in that comprise the following steps:
(1) original image or video are inputted and is sampled in feature space;
(2) in feature space, calculate the k nearest neighbor of each sample and set up it is local completely abut against figure, then calculate adjacent sample The distance between;
(3) according to conformal projection rule, it is combined with sparse expression method, dictionary of the study with conformal property;
(4) original image or video are reconstructed using this dictionary, obtain result;
Conformal projection rule described in step (3), is a kind of manifold learning, is specifically described as:Given feature space M To another feature space N mapping g:M → N, (xi,xj,xk) it is adjacent sample point in feature space M and triangle, (αijk) it is mapping of these sample points in feature space N;Needed to meet according to conformal projection rule:
<mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>j</mi> <mo>,</mo> <mi>k</mi> <mo>&amp;Element;</mo> <msub> <mi>N</mi> <mi>i</mi> </msub> </mrow> </munder> <msup> <mrow> <mo>(</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>x</mi> <mi>j</mi> </msub> <mo>-</mo> <msub> <mi>x</mi> <mi>k</mi> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>-</mo> <msub> <mi>s</mi> <mi>i</mi> </msub> <mo>|</mo> <mo>|</mo> <msub> <mi>&amp;alpha;</mi> <mi>j</mi> </msub> <mo>-</mo> <msub> <mi>&amp;alpha;</mi> <mi>k</mi> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>,</mo> </mrow>
Wherein, NiRepresent sample xiK nearest neighbor set, siRepresent the change of scale of mapping.
2. a kind of conformal projection sparse expression method of image/video scene content according to claim 1, its feature exists In:Part described in step (2) completely abuts against figure, refers in the set that is constituted for certain sample and its k nearest neighbor, arbitrarily It is connected between two samples.
3. a kind of conformal projection sparse expression method of image/video scene content according to claim 1, its feature exists In:Dictionary of the study with conformal property is combined with sparse expression method described in step (3), is concretely comprised the following steps:Will Conformal projection rule is combined with sparse expression algorithm, obtains following energy theorem:
<mrow> <munder> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> <mrow> <mi>D</mi> <mo>,</mo> <mi>&amp;alpha;</mi> <mo>,</mo> <mi>S</mi> </mrow> </munder> <munder> <mo>&amp;Sigma;</mo> <mi>i</mi> </munder> <mo>|</mo> <mo>|</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>D&amp;alpha;</mi> <mi>i</mi> </msub> <mo>|</mo> <msubsup> <mo>|</mo> <mn>2</mn> <mn>2</mn> </msubsup> <mo>+</mo> <msub> <mi>&amp;lambda;</mi> <mn>1</mn> </msub> <munder> <mo>&amp;Sigma;</mo> <mi>i</mi> </munder> <mo>|</mo> <mo>|</mo> <msub> <mi>&amp;alpha;</mi> <mi>i</mi> </msub> <mo>|</mo> <msub> <mo>|</mo> <mn>1</mn> </msub> <mo>+</mo> <msub> <mi>&amp;lambda;</mi> <mn>2</mn> </msub> <munder> <mo>&amp;Sigma;</mo> <mi>i</mi> </munder> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>j</mi> <mo>,</mo> <mi>k</mi> <mo>&amp;Element;</mo> <msub> <mi>N</mi> <mi>i</mi> </msub> </mrow> </munder> <msup> <mrow> <mo>(</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>x</mi> <mi>j</mi> </msub> <mo>-</mo> <msub> <mi>x</mi> <mi>k</mi> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>-</mo> <msub> <mi>s</mi> <mi>i</mi> </msub> <mo>|</mo> <mo>|</mo> <msub> <mi>&amp;alpha;</mi> <mi>j</mi> </msub> <mo>-</mo> <msub> <mi>&amp;alpha;</mi> <mi>k</mi> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>,</mo> </mrow>
Wherein, x is (xi,xj,xk), it is input sample feature, D is characterized dictionary, and α is (αijk), it is reconstruction coefficients, λ1、λ2 For weight coefficient, this energy theorem is minimized by iterative algorithm, dictionary D, s with conformal property is finally tried to achievei:Mapping Change of scale, S is all siSet.
4. a kind of conformal projection sparse expression method of image/video scene content according to claim 1, its feature exists In:Methods described is applied applied to video image editor, including image super-resolution, video image color editor, image denoising.
5. a kind of conformal projection sparse expression method of image/video scene content according to claim 1, its feature exists In:By introducing conformal projection rule, the local angle information between adjacent sample is maintained to greatest extent, obtains expressing energy The stronger dictionary of power;Meanwhile, conformal projection promotes adjacent sample to be reconstructed with similar dictionary, makes dictionary more succinct tight Gather.
CN201510337089.5A 2015-06-17 2015-06-17 A kind of conformal projection sparse expression method of image/video scene content Active CN104966276B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510337089.5A CN104966276B (en) 2015-06-17 2015-06-17 A kind of conformal projection sparse expression method of image/video scene content

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510337089.5A CN104966276B (en) 2015-06-17 2015-06-17 A kind of conformal projection sparse expression method of image/video scene content

Publications (2)

Publication Number Publication Date
CN104966276A CN104966276A (en) 2015-10-07
CN104966276B true CN104966276B (en) 2017-10-20

Family

ID=54220307

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510337089.5A Active CN104966276B (en) 2015-06-17 2015-06-17 A kind of conformal projection sparse expression method of image/video scene content

Country Status (1)

Country Link
CN (1) CN104966276B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106548476B (en) * 2016-11-25 2019-11-12 天津工业大学 Lung's three-dimensional feature Method On Shape is counted using medical image

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103049340A (en) * 2012-10-26 2013-04-17 中山大学 Image super-resolution reconstruction method of visual vocabularies and based on texture context constraint
CN104268593A (en) * 2014-09-22 2015-01-07 华东交通大学 Multiple-sparse-representation face recognition method for solving small sample size problem

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8525871B2 (en) * 2008-08-08 2013-09-03 Adobe Systems Incorporated Content-aware wide-angle images

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103049340A (en) * 2012-10-26 2013-04-17 中山大学 Image super-resolution reconstruction method of visual vocabularies and based on texture context constraint
CN104268593A (en) * 2014-09-22 2015-01-07 华东交通大学 Multiple-sparse-representation face recognition method for solving small sample size problem

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Application of conformal map theory for design of 2-D ultrasonic array structure for ndt imaging application: a feasibility study";Sivaram N. Ramadas等;《IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control》;20140225;第61卷(第3期);论文第496-504页 *
"稀疏流形建模及其在人脸识别中的应用";高文娟;《中国优秀硕士学位论文全文数据库 信息科技辑》;20111215(第S2期);论文第10-13,22,31-35页 *

Also Published As

Publication number Publication date
CN104966276A (en) 2015-10-07

Similar Documents

Publication Publication Date Title
CN113469094B (en) Surface coverage classification method based on multi-mode remote sensing data depth fusion
CN105069825A (en) Image super resolution reconstruction method based on deep belief network
CN108734661B (en) High-resolution image prediction method for constructing loss function based on image texture information
CN110349087B (en) RGB-D image high-quality grid generation method based on adaptive convolution
CN110223234A (en) Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion
Luo et al. Lattice network for lightweight image restoration
CN113343775B (en) Deep learning generalization method for remote sensing image ground surface coverage classification
CN113870124B (en) Weak supervision-based double-network mutual excitation learning shadow removing method
CN105844635A (en) Sparse representation depth image reconstruction algorithm based on structure dictionary
CN112488055B (en) Video question-answering method based on progressive graph attention network
CN111652240B (en) CNN-based image local feature detection and description method
CN112686817B (en) Image completion method based on uncertainty estimation
CN112200029B (en) Remote sensing image building extraction method based on improved UNet + + network
CN103413331B (en) A kind of support edits the high resolution video image content sparse expression method propagated
CN110263865A (en) A kind of semi-supervised multi-modal multi-class image interpretation method
CN114742985A (en) Hyperspectral feature extraction method and device and storage medium
CN113744136A (en) Image super-resolution reconstruction method and system based on channel constraint multi-feature fusion
Kaur et al. Robust and efficient ‘RGB’based fractal image compression: flower pollination based optimization
CN116167920A (en) Image compression and reconstruction method based on super-resolution and priori knowledge
CN104966276B (en) A kind of conformal projection sparse expression method of image/video scene content
CN114219701A (en) Dunhuang fresco artistic style conversion method, system, computer equipment and storage medium
CN113436094B (en) Gray level image automatic coloring method based on multi-view attention mechanism
CN116152263A (en) CM-MLP network-based medical image segmentation method
CN114140317A (en) Image animation method based on cascade generation confrontation network
CN105825473A (en) Image restoration method through adaptively switching analysis sparse regularization and synthesis sparse regularization

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant