CN103116891B - A kind of remote sensing image registration method based on two-way neighborhood filtering policy - Google Patents
A kind of remote sensing image registration method based on two-way neighborhood filtering policy Download PDFInfo
- Publication number
- CN103116891B CN103116891B CN201310077992.3A CN201310077992A CN103116891B CN 103116891 B CN103116891 B CN 103116891B CN 201310077992 A CN201310077992 A CN 201310077992A CN 103116891 B CN103116891 B CN 103116891B
- Authority
- CN
- China
- Prior art keywords
- way
- point
- neighborhood
- double points
- matching double
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The invention discloses a kind of remote sensing image registration method based on two-way neighborhood filtering policy, gray feature is combined with the space structure characteristic of unique point, using the matching result based on gray feature as initial matching point pair, using the neighbour structure characteristic of unique point as constraint, by the otherness of two-way neighbour structure, iteration obtains the matching double points with identical two-way neighbour structure, and it is a little right to adopt the candidate of mistake rejecting in two-way neighborhood filtering policy recovery iterative process to mismatch.The method is used for there is larger affined transformation between image subject to registration, image subject to registration is the situation that there is similar pattern in allos image, image scene, improves the precision of registration under the condition that can participate at prosthetic.
Description
Technical field
The invention belongs to image processing field, relate to a kind of method for registering images, be specifically related to a kind of remote sensing image registration method based on two-way neighborhood filtering policy.
Background technology
Image registration, as an important preconditioning technique in graphical analysis, is widely used in the fields such as image co-registration, computer vision and target identification.The step of image registration mainly comprises: feature extraction, characteristic matching, transformation model parameter estimation, image resampling and inverse transformation.Wherein, the object of characteristic matching ensures to form reliable mapping relations between characteristics of image subject to registration, is a committed step in image registration.
Due to the singularity of remote sensing images, remote sensing images have more disturbing factor in registration process.First, remote sensing images abundance, when remote sensing images subject to registration are allos image, because imaging mechanism is different, Same Scene presents different gamma characteristics in allos image; Secondly, remote sensing images are normally remote, imaging gained under Large visual angle, there is larger affined transformation, and the possibility in scene with similar pattern are larger between the remote sensing images do not obtained in the same time.
The method of conventional images Feature Points Matching mainly contains based on gray feature with based on the large class methods of space structure two.Method based on gray feature utilizes the correlativity of gray feature to carry out characteristic matching, such as SIFT(Scale-Invariant Feature Transform, and scale invariant feature is changed) algorithm.Current most of feature extracting method does not have stability when there is larger affined transformation between image subject to registration.In addition, there is similar pattern because allos gradation of image is uncorrelated, in scene, cause the matching process degree of accuracy based on gray feature to reduce.Method for registering based on space structure mainly utilizes the characteristic on space structure to carry out characteristic matching, such as RANSAC algorithm, but because RANSAN algorithm is judging there is stronger constraint condition when to mismatch, causes more mismatching a little cannot reject, thus reducing registration accuracy.
In sum, there is the problem that there is similar pattern tripartite face in larger affined transformation, allos image, scene for remote sensing images, existing method for registering images is difficult to realize high-precision registration.
Summary of the invention
The object of the invention is to propose a kind of remote sensing image registration method based on two-way neighborhood filtering policy, gray feature is combined with the space structure characteristic of unique point, using the matching result based on gray feature as initial matching point pair, using the neighbour structure characteristic of unique point as constraint, by the otherness of two-way neighbour structure, iteration obtains the matching double points with identical two-way neighbour structure, and it is a little right to adopt the candidate of mistake rejecting in two-way neighborhood filtering policy recovery iterative process to mismatch.The method is applicable to there is larger affined transformation between image subject to registration, image subject to registration is the situation that there is similar pattern in allos image, image scene, improves the precision of registration under the condition that can participate at prosthetic.
In order to achieve the above object, technical scheme of the present invention is to provide a kind of remote sensing image registration method based on two-way neighborhood filtering policy, and it comprises the following steps:
Step 1: adopt all SIFT feature points and each self-corresponding SIFT feature vector thereof in SIFT operator extraction image subject to registration, compared by the Euclidean distance between SIFT feature vector, be that every bit in an image selects its point in another image with maximum Euclidean distance to carry out correspondence, thus form initial matching point pair between these two points, and then obtain the set that in image subject to registration, all initial matching points are right;
Step 2: for all initial matching points are to the two-way neighbour structure of structure, and it is a little right to alternatively mismatching to reject the maximum point of difference according to the two-way neighborhood difference matrix of correspondence structure, is had the matching double points set of identical two-way neighbour structure by iteration formation;
Step 3: it is a little right that the candidate reexamining all rejectings according to two-way neighborhood filtering policy mismatches, judges whether to need the candidate recovered to mismatch a little right, has and needs the candidate of recovery to mismatch a little to then turning to step 2; Nothing then iteration terminates, using remaining matching double points as final matching double points.
Step 4: adopt least square method according to final matching double points estimation affine transformation parameter, image subject to registration is carried out affine inverse transformation, obtains the image of coupling mutually.
In described step 2, the process constructing two-way neighbour structure is: respectively in each image subject to registration, using distance each point nearest before K put as k nearest neighbor point, set up the directed edge from each point to its respective k nearest neighbor point, then the two-way neighbour structure of any point is jointly made up of the k nearest neighbor point of the directed edge be connected with this point and this point.
In described step 2, the process that the matching double points set with identical two-way neighbour structure is formed is as follows:
Step 2-1, constructs each matching double points respectively and is integrated into forward direction Neighborhood matrix in two-way Neighborhood matrix
with backward Neighborhood matrix
;
Set up an office
and point
be 2 points in matching double points set described in any one, work as a little
a little
k nearest neighbor point time, the forward direction Neighborhood matrix of this matching double points set
, backward Neighborhood matrix
; Work as a little
not a little
k nearest neighbor point time, forward direction matrix and the backward matrix of this matching double points set are 0; The set that the initial matching point that described matching double points set obtains after referring to step 1 is right, or the set of the matching double points upgraded is obtained through any iteration of step 2;
Step 2-2, for forward direction Neighborhood matrix corresponding in each matching double points set
, and corresponding backward Neighborhood matrix
, carry out XOR respectively and obtain two-way neighborhood difference matrix
with
;
Step 2-3, according to
, the point selecting two-way neighborhood difference maximum in two-way neighborhood difference matrix is right
alternatively mismatch a little to rejecting, wherein
it is the right sum of current matching point;
Step 2-4, upgrades the two-way neighborhood of matching double points, judges two-way neighborhood difference matrix
with
whether is full null matrix, be not that full null matrix then turns to step 2-1; Be full null matrix then iteration stopping, residue matching double points to form the matching double points set with identical two-way neighbour structure.
In described step 3, it is as follows that two-way neighborhood filtering policy screening needs the candidate recovered to mismatch a little right process:
Step 3-1, any one candidate mismatches a little right
point in the neighbour structure at place is disallowable when mismatching a little pair as new candidate in successive iterations, and this candidate mismatches a little right
needs reexamine;
Step 3-2, mismatches the candidate of examine a little to combining with remaining matching double points respectively, and the candidate that wherein can form identical two-way neighbour structure mismatches a little to reverting to matching double points.
Compared with prior art, a kind of remote sensing image registration method based on two-way neighborhood filtering policy of the present invention, its advantage is:
(1) spatial structure characteristic of gray feature and two-way neighborhood combines by the present invention, realizes the Remote sensing image registration for there is similar pattern in large affined transformation, allos image, scene.
(2) the present invention is using two-way neighbour structure as image space restrain condition, rejects the candidate with maximum two-way neighbour structure difference and mismatches a little right, ensure that matching double points has identical two-way neighbour structure by the mode of iteration.
(3) the present invention adopts two-way neighborhood filtering policy, candidate is mismatched the situation that a centering exists neighborhood point also disallowable again to extract, the candidate wherein with matching double points with identical two-way neighbour structure is mismatched and a little reverts to matching double points, while increase matching double points, eliminate again residual mismatch a little right.
Accompanying drawing explanation
Fig. 1 is the flow chart of steps of the remote sensing image registration method that the present invention is based on two-way neighborhood filtering policy;
When Fig. 2 is Size of Neighborhood K=4 of the present invention
the two-way neighbour structure schematic diagram of point;
Two-way neighbour structure schematic diagram when Fig. 3 a is the 0th iteration during the present invention typical case implements;
Two-way neighbour structure schematic diagram when Fig. 3 b is the 10th iteration during the present invention typical case implements;
Identical two-way neighbour structure schematic diagram when Fig. 3 c is the 22nd iteration during the present invention typical case implements.
Fig. 4 a is the schematic diagram of initialization match condition during the present invention typical case implements;
Fig. 4 b is the match condition schematic diagram during the present invention typical case implements with identical two-way neighbour structure;
Fig. 4 c is the neighbour structure schematic diagram during the present invention typical case implements with identical two-way neighbour structure;
Fig. 4 d needs the candidate recovered to mismatch a little right schematic diagram during the present invention typical case implements;
Fig. 4 e is the schematic diagram of final match condition during the present invention typical case implements.
Embodiment
Below in conjunction with drawings and the specific embodiments, the invention will be further described.
The present invention proposes a kind of remote sensing image registration method based on two-way neighborhood filtering policy, is particularly useful for there is larger affined transformation between image subject to registration, image subject to registration is the situation that there is similar pattern in allos image, image scene.As shown in Figure 1, be the process flow diagram of the concrete implementation step of described method.It is the specific embodiment that application the method for the invention carries out remote sensing images A and B registration as shown in Fig. 3 a ~ Fig. 3 c, Fig. 4 a ~ Fig. 4 e.
Step one: initialization matching double points.
(1) read in image A subject to registration and image B from two image channels, utilize SIFT feature point and the 128 dimension SIFT feature vectors of each image subject to registration of SIFT feature operator extraction.
Adopt Gaussian convolution core structure difference of Gaussian image
:
（1）
Wherein, Gaussian convolution core is
,
the volume coordinate in image,
scale factor,
be constant Product-factor, during concrete enforcement, get 3.
The check point of each yardstick in difference of Gaussian image is compared with the consecutive point laterally with yardstick, the longitudinally consecutive point of adjacent yardstick, asks for extreme point, and the reference direction of gradient direction determination SIFT feature vector according to extreme point.Centered by unique point, gradient direction for principal direction get 4 × 4 neighborhood, each neighborhood chooses the gradient orientation histogram that 8 angle directions calculate each zonule, obtain 128 dimension SIFT feature vector.
(2) by being compared by the SIFT feature of image subject to registration vector Euclidean distance, the set of initial matching point is obtained
with
.For any one the SIFT feature point in image A, calculate the Euclidean distance between the SIFT feature vector of this Feature point correspondence and all SIFT feature vectors of image B, and by sorting from big to small, when maximum Euclidean distance be 1.2 times of second largest Euclidean distance and above time, using the initial matching point of SIFT feature corresponding with maximum Euclidean distance in image B point as unique point in image A.The initial matching point in image A in all unique points and image B is obtained with method described in this step, and the initial matching point of all unique points in image A in image B, and be placed in set respectively
with
in.In two images, unique point is configured to initial matching point pair with the corresponding of its initial matching point, namely show in Fig. 4 a initial matching point in image A and B subject to registration between line.
Step 2: the matching double points that there is identical two-way neighbour structure.
(1) two-way neighbour structure is constructed.Namely, initial matching point centering, using with wherein any point K the k nearest neighbor put as this point before nearest, choose K=4 in such as, embodiment shown in Fig. 2, to construct in each image from each point to the directed edge of its k nearest neighbor point separately respectively, form the two-way neighbour structure of corresponding point.
(2) the right two-way Neighborhood matrix of initial matching point is constructed
,
with
,
.With
with
for example, work as a little
a little
k nearest neighbor point time, forward direction Neighborhood matrix
, backward Neighborhood matrix
; Otherwise forward direction matrix and backward matrix are 0, its mid point
and point
for match point set
in point.Similar,
with
according to set
in some structure.
(3) to the forward direction Neighborhood matrix corresponding to two matching double points set
with
, backward Neighborhood matrix
with
carry out xor operation respectively, to form two-way neighborhood difference matrix
with
.The point that in these two two-way neighborhood difference matrix, difference is maximum is right
alternatively mismatch a little disallowable, reject criterion as follows:
（2）
Wherein
it is the right sum of current matching point.
Repeat (1) of above-mentioned steps two to (3), until two-way neighborhood difference matrix
with
when being full null matrix, iteration terminates, and makes last remaining matching double points have identical two-way neighbour structure.
See the embody rule example of Fig. 3 a ~ Fig. 3 c, be wherein the two-way neighbour structure constructed respectively in both images according to all initial matching point shown in Fig. 3 a, now not yet through iteration.Be the two-way neighbour structure of the method described according to step 2 when carrying out the 10th iteration shown in Fig. 3 b, compare Fig. 3 a and eliminate some matching double points in two images.Be two-way neighbour structure when carrying out the 22nd iteration shown in Fig. 3 c, compare Fig. 3 b and eliminate some matching double points wherein further, make to obtain identical two-way neighbour structure in two images of Fig. 3 c.Point disallowable in an iterative process is a little waited until subsequent step confirm alternatively being mismatched.
Step 3: it is a little right that the candidate adopting two-way neighborhood filtering policy to recover mistake rejecting mismatches.
(1) any one candidate mismatch a little right
point in the neighbour structure at place is disallowable when mismatching a little pair as new candidate in successive iterations, and this candidate mismatches a little right
needs reexamine.
Such as, first candidate mismatches a little to disallowable when the X time iteration, first candidate mismatches to reject in the neighbour structure at front place has a point to mismatch a little right as new candidate to disallowable when follow-up the Y time iteration, and this situation needs to mismatch a little to reexamining first candidate.
(2) candidate of examine mismatched a little to combining with remaining matching double points respectively, the candidate that wherein can form identical two-way neighbour structure mismatches a little to reverting to matching double points.
Repeat step 2 and step 3, until iteration terminates when not needing the candidate recovered to mismatch a little pair.Iteration is terminated rear remaining matching double points as final matching double points.
Namely, in residue match point after the whole iteration of step 2 terminates, it is a little right that the candidate that once only increase by group needs reexamine mismatches, judge that whether each point is still identical to the two-way neighbour structure obtained in both images after increase, if the same this group candidate mismatches a little to recovering, otherwise is just rejected.Afterwards, after step 2 in remaining matching double points, increase all candidates that confirmation needs recover in step 3 and mismatch a little to rear, re-start the iterative operation of step 2 to step 3, until do not need the candidate recovered to mismatch a little to rear end.
See the embody rule example shown in Fig. 4 a ~ Fig. 4 e.Be the schematic diagram after step one initialization coupling shown in Fig. 4 a, wherein there is the line between numerous initialization matching double points.Line after being through step 2 process shown in Fig. 4 b between remaining match point, compare Fig. 4 a quantity and significantly reduce, these remaining match points can construct the identical two-way neighbour structure shown in Fig. 4 c.Being the matching double points that the needs that detect in step 3 recover shown in Fig. 4 d, being added together by remaining matching double points in itself and Fig. 4 b, obtaining final matching double points, is exactly the line situation between final matching double points shown in Fig. 4 e.
Step 4: adopt least square method estimation affine transformation parameter according to final matching double points, image subject to registration is carried out affine inverse transformation, obtains the image pair of coupling mutually.The vector representation of affine transformation parameter method for solving is:
，
（3）
Wherein N is the sum of matching double points;
for the coefficient of affined transformation;
with
a little
coordinate before and after conversion.
So far, the method for the remote sensing image registration based on two-way neighborhood filtering policy of the present invention is completed.
Although content of the present invention has done detailed introduction by above preferred embodiment, will be appreciated that above-mentioned description should not be considered to limitation of the present invention.After those skilled in the art have read foregoing, for multiple amendment of the present invention and substitute will be all apparent.Therefore, protection scope of the present invention should be limited to the appended claims.
Claims (1)
1., based on a remote sensing image registration method for two-way neighborhood filtering policy, it is characterized in that, comprise the following steps:
Step 1: adopt all SIFT feature points and each self-corresponding SIFT feature vector thereof in SIFT operator extraction image subject to registration, compared by the Euclidean distance between SIFT feature vector, be that every bit in an image selects its point in another image with maximum Euclidean distance to carry out correspondence, thus form initial matching point pair between these two points, and then obtain the set that in image subject to registration, all initial matching points are right;
Step 2: for all initial matching points are to the two-way neighbour structure of structure, and it is a little right to alternatively mismatching to reject the maximum point of difference according to the two-way neighborhood difference matrix of correspondence structure, is had the matching double points set of identical two-way neighbour structure by iteration formation;
Wherein, the process constructing two-way neighbour structure is: respectively in each image subject to registration, using distance each point nearest before K put as k nearest neighbor point, set up the directed edge from each point to its respective k nearest neighbor point, then the two-way neighbour structure of any point is jointly made up of the k nearest neighbor point of the directed edge be connected with this point and this point;
Wherein, the process that the matching double points set with identical two-way neighbour structure is formed is as follows:
Step 2-1, constructs each matching double points respectively and is integrated into forward direction Neighborhood matrix FKNN in two-way Neighborhood matrix and backward Neighborhood matrix BKNN;
The i and some j that sets up an office is 2 points in matching double points set described in any one, when a j is the k nearest neighbor point of an i, and the forward direction Neighborhood matrix FKNN [i, j]=1 of this matching double points set, backward Neighborhood matrix BKNN [i, j]=1; When a j is not the k nearest neighbor point of an i, forward direction Neighborhood matrix and the backward Neighborhood matrix of this matching double points set are 0; The set that the initial matching point that described matching double points set obtains after referring to step 1 is right, or the set of the matching double points upgraded is obtained through any iteration of step 2;
Step 2-2, for forward direction Neighborhood matrix FKNN corresponding in each matching double points set, and corresponding backward Neighborhood matrix BKNN, carry out XOR respectively and obtain two-way neighborhood difference matrix Δ FKNN and Δ BKNN;
Step 2-3, according to
the point selecting two-way neighborhood difference maximum in two-way neighborhood difference matrix is to j
outlieralternatively mismatch a little to rejecting, wherein N is the right sum of current matching point;
Step 2-4, upgrade the two-way neighborhood of matching double points, judging whether two-way neighborhood difference matrix Δ FKNN and Δ BKNN is full null matrix, is not that full null matrix then turns to step 2-1; Be full null matrix then iteration stopping, residue matching double points to form the matching double points set with identical two-way neighbour structure;
Step 3: it is a little right that the candidate reexamining all rejectings according to two-way neighborhood filtering policy mismatches, judges whether to need the candidate recovered to mismatch a little right, has and needs the candidate of recovery to mismatch a little to then turning to step 2; Nothing then iteration terminates, using remaining matching double points as final matching double points;
Wherein, it is as follows that the candidate that two-way neighborhood filtering policy screening needs recover mismatches a little right process:
Step 3-1, any one candidate mismatches a little to j
outlierpoint in the neighbour structure at place is disallowable when mismatching a little pair as new candidate in successive iterations, and this candidate mismatches a little to j
outlierneeds reexamine;
Step 3-2, mismatches the candidate of examine a little to combining with remaining matching double points respectively, and the candidate that wherein can form identical two-way neighbour structure mismatches a little to reverting to matching double points;
Step 4: adopt least square method according to final matching double points estimation affine transformation parameter, image subject to registration is carried out affine inverse transformation, obtains the image of coupling mutually.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310077992.3A CN103116891B (en) | 2013-03-12 | 2013-03-12 | A kind of remote sensing image registration method based on two-way neighborhood filtering policy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310077992.3A CN103116891B (en) | 2013-03-12 | 2013-03-12 | A kind of remote sensing image registration method based on two-way neighborhood filtering policy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103116891A CN103116891A (en) | 2013-05-22 |
| CN103116891B true CN103116891B (en) | 2015-08-12 |
Family
ID=48415255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310077992.3A Expired - Fee Related CN103116891B (en) | 2013-03-12 | 2013-03-12 | A kind of remote sensing image registration method based on two-way neighborhood filtering policy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103116891B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103985132B (en) * | 2014-05-30 | 2017-04-19 | 中国科学院遥感与数字地球研究所 | Mismatching point iterative detection method based on K neighbor graphs |
| CN107452037B (en) * | 2017-08-02 | 2021-05-14 | 北京航空航天大学青岛研究院 | GPS auxiliary information acceleration-based structure recovery method from movement |
| CN111932593B (en) * | 2020-07-21 | 2024-04-09 | 湖南中联重科智能技术有限公司 | Image registration method, system and equipment based on touch screen gesture correction |
| CN112364879A (en) * | 2020-10-10 | 2021-02-12 | 南京轩宁信息技术有限公司 | Image matching method based on bidirectional optimal matching point pair |
-
2013
- 2013-03-12 CN CN201310077992.3A patent/CN103116891B/en not_active Expired - Fee Related
Non-Patent Citations (3)
| Title |
|---|
| A robust graph transformation matcthing for non-rigid registration;Wendy Aguilar,Yann Frauel,ec al;《Image and Vision Computing》;20090504;第2节，第3.1节 * |
| SIFT特征匹配在无人机低空遥感影像处理中的应用;陈信华;《地矿测绘》;20080625;第24卷(第2期);第1.5节，第3.2-3.3节，第4节 * |
| 刘应东,牛慧民.基于K-最近邻图的小样本KNN分类算法.《计算机工程》.2011, * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103116891A (en) | 2013-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111028277B (en) | SAR and optical remote sensing image registration method based on pseudo-twin convolution neural network | |
| Liu et al. | Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion | |
| CN102750537B (en) | Automatic registering method of high accuracy images | |
| Arrigoni et al. | Robust synchronization in SO (3) and SE (3) via low-rank and sparse matrix decomposition | |
| CN109285110B (en) | Infrared visible light image registration method and system based on robust matching and transformation | |
| CN103116891B (en) | A kind of remote sensing image registration method based on two-way neighborhood filtering policy | |
| CN104036480A (en) | Surf algorithm based quick mismatching point eliminating method | |
| Planitz et al. | The correspondence framework for 3D surface matching algorithms | |
| CN101138007A (en) | Image processing system, learning device and method, and program | |
| CN104200463A (en) | Fourier-Merlin transform and maximum mutual information theory based image registration method | |
| CN111126494B (en) | Image classification method and system based on anisotropic convolution | |
| CN112396643A (en) | Multi-mode high-resolution image registration method with scale-invariant features and geometric features fused | |
| CN111242855B (en) | Iterative depth map structure repairing method based on RGB-D SSIM structure similarity | |
| CN102915540A (en) | Image matching method based on improved Harris-Laplace and scale invariant feature transform (SIFT) descriptor | |
| CN110009670A (en) | The heterologous method for registering images described based on FAST feature extraction and PIIFD feature | |
| CN116664892A (en) | Multi-temporal remote sensing image registration method based on cross attention and deformable convolution | |
| He et al. | Linear approach for initial recovery of the exterior orientation parameters of randomly captured images by low-cost mobile mapping systems | |
| CN106651756B (en) | Image registration method based on SIFT and verification mechanism | |
| CN107886530A (en) | A kind of improved image registration algorithm based on SIFT feature | |
| Zhou et al. | Discarding wide baseline mismatches with global and local transformation consistency | |
| Dambreville et al. | A geometric approach to joint 2D region-based segmentation and 3D pose estimation using a 3D shape prior | |
| CN105184327A (en) | Vertex trisection strategy-based remote sensing image feature point matching method | |
| Chen et al. | An improved image matching method based on SURF algorithm | |
| CN117351078A (en) | Target size and 6D gesture estimation method based on shape priori | |
| Koskenkorva et al. | Quasi-dense wide baseline matching for three views |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150812 Termination date: 20180312 |