CN102306374A - Method for rebuilding super-resolution human face image by position block nonlinear mapping - Google Patents

Method for rebuilding super-resolution human face image by position block nonlinear mapping Download PDF

Info

Publication number
CN102306374A
CN102306374A CN201110252310A CN201110252310A CN102306374A CN 102306374 A CN102306374 A CN 102306374A CN 201110252310 A CN201110252310 A CN 201110252310A CN 201110252310 A CN201110252310 A CN 201110252310A CN 102306374 A CN102306374 A CN 102306374A
Authority
CN
China
Prior art keywords
resolution
image
super
block
locations
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201110252310A
Other languages
Chinese (zh)
Other versions
CN102306374B (en
Inventor
黄华
曾啸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Jiaotong University
Original Assignee
Xian Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Jiaotong University filed Critical Xian Jiaotong University
Priority to CN 201110252310 priority Critical patent/CN102306374B/en
Publication of CN102306374A publication Critical patent/CN102306374A/en
Application granted granted Critical
Publication of CN102306374B publication Critical patent/CN102306374B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Processing (AREA)

Abstract

The invention relates to a method for rebuilding a super-resolution human face image by position block nonlinear mapping. The method comprises the following steps of: segmenting the human face image for training and testing high resolution and low resolution into corresponding high-resolution position block images and low-resolution position block images which are provided with overlapped regions from structural information and detail information of a roughly-aligned human face image; rebuilding super-resolution position block images corresponding to testing low-resolution position block images at each corresponding position by using a nonlinear mapping relation which is between the high-resolution position block images and the low-resolution position block images and learned from training data; and splicing the super-resolution position block images to obtain the final super-resolution human face image. According to experiments on a standard database, the method provided by the invention has a good visual effect and high objective estimation quality.

Description

A kind of face image super-resolution reconstruction method of block of locations Nonlinear Mapping
Technical field
The invention belongs to image processing techniques, be specifically related to a kind of face image super-resolution reconstruction method of block of locations Nonlinear Mapping.
Background technology
(Super Resolution SR) is meant that (Low Resolution obtains a panel height resolution (High Resolution, HR) process of image in LR) from a width of cloth or a series of low-resolution image to image super-resolution.In recent years, video monitoring important resultant use widely on bank, airport etc.But in many cases, the facial image resolution that watch-dog obtains is too low, so that can't directly discern, so the research of face image super-resolution problem has realistic meanings.
The super-resolution algorithms of general nature image much enlightens for the face image super-resolution algorithm.But facial image is one type of special image, and its specific global structure is arranged.Therefore the general nature image super-resolution algorithm that will not examine rate global structure information is generalized to the face image super-resolution problem has certain limitation, and the effect of reconstruction also has the space of further improving.
Liu etc. have proposed the two-step approach of human face super-resolution first, are about to reconstruction two parts of human face super-resolution PROBLEM DECOMPOSITION behaviour face global information and local detail information, and global information and local detail addition are obtained final high-resolution human face image.Two-step approach is a good framework; The method of numerous human face super-resolutions is all based on this framework: to the expression of people's face global information; Main thought is to describe global characteristics with the particular constraints of facial image, all rebuilds global information through the relation of setting up between the high low-resolution image global characteristics mostly in the realization; For the expression of detail textures characteristic, main thought is to set up the image block of sample and the neighborhood relationships between the training set image block, and adopts various partial models to describe textural characteristics.
Two-step approach combines through setting up high low resolution people's face global characteristics relation and details compensation; The reconstruction recovery effects improves; But being everlasting often, the relation between high low resolution people's face global characteristics set up introduces pseudo-shadow in the reconstructed results; Reduced the performance of rebuilding, meanwhile two-step approach has been introduced the additional calculation amount, and rebuilding on the efficient also has certain decline.Based on this, Ma etc. have proposed the human face super-resolution algorithm of a kind of position-based piece reconstruct, with the global structure of the location information table traveller on a long journey face of image block, characterize detailed information with the content of image block.Yet the least square method that Ma adopts when recovering detailed information has only been considered the linear dependence component of height resolution position interblock; And ignored the nonlinear dependence component; In order to improve the performance of super-resolution rebuilding, need to seek better method and solve the problem that exists here.
Summary of the invention
The object of the present invention is to provide a kind of face image super-resolution reconstruction method of block of locations Nonlinear Mapping, to solve the problems referred to above that exist in the prior art.
In order to achieve the above object, the technical scheme taked of the present invention is:
1) at first, high low resolution facial image is divided into height resolution position piece image;
2) secondly, utilize the high-definition picture of RBF super-resolution rebuilding correspondence position piece.In the training stage; Utilize RBF by training image Nonlinear Mapping of height resolution position piece training for each position; Be used to represent the nonlinear relationship between the height resolution position piece of each position; At test phase, the nonlinear relationship of utilizing each position training to obtain estimates corresponding high-resolution position piece from test low resolution block of locations;
3) at last, the positional information according to each block of locations becomes corresponding super-resolution reconstruction image with the high-resolution position piece image mosaic of estimating to obtain.
Said step 1) comprises following steps:
1) the low resolution facial image is adopted bilinear interpolation, make that the low resolution facial image resolution sizes after the interpolation is consistent with the high-definition picture resolution sizes.
2) adopt the window of p*p size respectively high low-resolution image to be carried out piecemeal, adopting has overlapping partitioned mode, and overlapping part is arranged between the piece of adjacent position up and down, and overlaid pixel is n between the adjacent block.Wherein p is an integer, and span is 2～16, and n is an integer and 0≤n≤p/2.The height resolution position piece that same level images of positions piece in the high low-resolution image is called corresponding same position.
Said step 2) utilize the high-definition picture of RBF super-resolution rebuilding correspondence position piece, its detailed process is following:
Use the high low resolution training block of locations of q position of
Figure BDA0000087559500000031
and
Figure BDA0000087559500000032
expression respectively;
Figure BDA0000087559500000033
and
Figure BDA0000087559500000034
representes the height resolution position piece image of q first training sample of position respectively; Wherein
Figure BDA0000087559500000035
and
Figure BDA0000087559500000036
is the column vector that block of locations gradation of image value is launched into; Subscript H and L represent high resolving power and low resolution respectively; First subscript q representes q position being partitioned into according to aforesaid block of locations dividing method; Second subscript 1～m representes the 1st～m training sample; Represent to be positioned at test low resolution block of locations and high-resolution position piece to be asked after the bilinear interpolation of position q with
Figure BDA0000087559500000037
and respectively, then utilize RBF to try to achieve
Figure BDA0000087559500000039
calculating formula to be:
Figure BDA00000875595000000310
The wherein transposition computing of T representing matrix;
Figure BDA00000875595000000311
multidimensional quadric surface RBF for adopting, concrete calculation expression is:
Figure BDA00000875595000000312
W qBe the weight matrix that the training stage obtains, calculation expression is:
W q = I q H · inv ( φ + τE )
Inv representing matrix inversion operation in the above-mentioned formula, τ are a very little positive constant, and span is 0.001～0.1, and E is a unit matrix, and φ is a matrix, are obtained by training height resolution position piece image calculation:
Figure BDA0000087559500000041
The repeating step aforementioned calculation is till the high-resolution position piece image of obtaining each position.
Said step 3) becomes corresponding super-resolution reconstruction image with high-resolution position piece image mosaic, and the average of when running into overlaid pixel, getting each this grey scale pixel value of high-resolution position piece is the final gray-scale value of overlaid pixel.
The positional information that the present invention has made full use of block of locations has kept the global structure of super-resolution human face rebuilding, utilizes the Nonlinear Mapping of RBF training to estimate corresponding high-resolution block of locations by the low resolution block of locations better.Therefore method of the present invention can keep the global structure of super-resolution rebuilding people face preferably, can recover the detailed information of people's face again preferably.
Description of drawings
Fig. 1 image block synoptic diagram
Fig. 2 human face super-resolution is rebuild comparison diagram
The PSNR contrast of Fig. 3 human face super-resolution reconstructed results
The SSIM contrast of Fig. 4 human face super-resolution reconstructed results
Embodiment
For making the object of the invention, technical scheme and advantage clearer,, the present invention is done further detailed description below in conjunction with accompanying drawing and instantiation.These instances only are illustrative, and are not limitation of the present invention.
The method that the present invention proposes may further comprise the steps:
(1) high low resolution facial image is divided into height resolution position piece image:
Because the resolution sizes of high low resolution facial image is different; Direct picture position with high low resolution people's face usually relates to the sub-pix rank when being mapped; Therefore need earlier the low resolution facial image to be adopted bilinear interpolation, make that the image resolution ratio size after the interpolation is consistent with the high-definition picture resolution sizes.Because the bilinear interpolation process is not introduced extra high-frequency information, still can image after the interpolation be regarded as low-resolution image.
Then, as shown in Figure 1, adopt the window of p*p size respectively high low-resolution image to be carried out piecemeal, adopting has overlapping partitioned mode, and overlapping part is arranged between the piece of adjacent position up and down, and overlaid pixel is n between the adjacent block.Wherein p is an integer, and span is 2～16, and n is an integer and 0≤n≤p/2.The height resolution position piece that same level images of positions piece in the high low-resolution image is called corresponding same position.
(2) utilize the high-definition picture of RBF super-resolution rebuilding correspondence position piece:
Because the algorithm of each position is consistent, the Nonlinear Mapping of just trying to achieve is inconsistent, is example with q position only, introduces correlative detail.Respectively
Figure BDA0000087559500000051
and
Figure BDA0000087559500000052
indicates the position of the q-th block and low resolution training positions.
Figure BDA0000087559500000053
and
Figure BDA0000087559500000054
representes the height resolution position piece image (
Figure BDA0000087559500000055
and is the column vector that block of locations gradation of image value is launched into) of q first training sample of position respectively; Subscript H and L represent high resolving power and low resolution respectively; First subscript q representes q position being partitioned into according to aforesaid block of locations dividing method, and second subscript 1～m representes the 1st～m training sample.With
Figure BDA0000087559500000057
and
Figure BDA0000087559500000058
q, respectively, bilinear interpolation in position after the test block and the position of the unknown low-resolution high-resolution position blocks.Then utilize RBF to be in the hope of calculating formula:
Wherein T is the matrix transpose computing;
Figure BDA00000875595000000511
is multidimensional quadric surface RBF, and concrete calculation expression is:
Figure BDA0000087559500000061
W qBe the weight matrix that the training stage obtains, calculation expression is:
W q = I q H · inv ( φ + τE )
Inv representing matrix inversion operation in the above-mentioned formula, τ are a very little positive constant, and span is 0.001～0.1, and E is a unit matrix, and φ is a matrix, can be obtained by training height resolution position piece image calculation:
(3) splicing high-resolution position piece facial image:
According to the positional information of each block of locations with estimating that the high-resolution position piece that obtains is spliced into corresponding super-resolution reconstruction image.The average of when running into overlaid pixel, getting each this grey scale pixel value of high-resolution position piece is the final gray-scale value of overlaid pixel.
Fig. 2 rebuilds comparison diagram for human face super-resolution.Figure (a) is test low resolution facial image; Figure (b) proposes the facial image of method super-resolution rebuilding for the present invention; Figure (c) is the facial image of the method super-resolution rebuilding of Liu; (d) be the facial image of the method super-resolution rebuilding of Zhuang; Figure (e) is the original high resolution facial image.High-resolution human face image and the additive method that can be found out the method reconstruction that the present invention proposes by Fig. 2 contrast, and the gained visual effect is better, more near the original high resolution facial image.
Fig. 3 and Fig. 4 are the quality assessment contrast of human face super-resolution reconstructed results.Fig. 3 is the comparison diagram of Y-PSNR (PSNR), and Fig. 4 is the comparison diagram of structural similarity index (SSIM).Can find out the high-resolution human face image and the additive method contrast of the method reconstruction that the present invention proposes by Fig. 3 and Fig. 4; The gained evaluating objective quality is higher; Wherein Y-PSNR (PSNR) is higher; The error that shows method reconstructed results that the present invention proposes and original high resolution image is less, and (SSIM) is higher for the structural similarity index, shows that method that the present invention proposes has kept the structural information of people's face better.
In sum, the method for the present invention's proposition is compared the lifting that on subjective vision effect and evaluating objective quality, all has to a certain degree with existing method.
Should be appreciated that from foregoing description, under the situation that does not break away from spirit of the present invention, can make amendment and change each embodiment of the present invention.Description in this instructions is only used for illustrative, and should not be considered to restrictive.Scope of the present invention only receives the restriction of claims.

Claims (4)

1. the face image super-resolution reconstruction method of a block of locations Nonlinear Mapping is characterized in that: comprise following steps:
1) at first, high low resolution facial image is divided into height resolution position piece image;
2) secondly, utilize the high-definition picture of RBF super-resolution rebuilding correspondence position piece.In the training stage; Utilize RBF by training image Nonlinear Mapping of height resolution position piece training for each position; Be used to represent the nonlinear relationship between the height resolution position piece of each position; At test phase, the nonlinear relationship of utilizing each position training to obtain estimates corresponding high-resolution position piece from test low resolution block of locations;
3) at last, the positional information according to each block of locations becomes corresponding super-resolution reconstruction image with the high-resolution position piece image mosaic of estimating to obtain.
2. the face image super-resolution reconstruction method of block of locations Nonlinear Mapping according to claim 1 is characterized in that: said step 1) comprises following steps:
1) the low resolution facial image is adopted bilinear interpolation, make that the low resolution facial image resolution sizes after the interpolation is consistent with the high-definition picture resolution sizes.
2) adopt the window of p*p size respectively high low-resolution image to be carried out piecemeal, adopting has overlapping partitioned mode, and overlapping part is arranged between the piece of adjacent position up and down, and overlaid pixel is n between the adjacent block.Wherein p is an integer, and span is 2～16, and n is an integer and 0≤n≤p/2.The height resolution position piece that same level images of positions piece in the high low-resolution image is called corresponding same position.
3. the face image super-resolution reconstruction method of block of locations Nonlinear Mapping according to claim 1 is characterized in that: said step 2) utilize the high-definition picture of RBF super-resolution rebuilding correspondence position piece, its detailed process is following:
Use the high low resolution training block of locations of q position of
Figure FDA0000087559490000011
and
Figure FDA0000087559490000012
expression respectively;
Figure FDA0000087559490000021
and
Figure FDA0000087559490000022
representes the height resolution position piece image of q first training sample of position respectively; Wherein
Figure FDA0000087559490000023
and
Figure FDA0000087559490000024
is the column vector that block of locations gradation of image value is launched into; Subscript H and L represent high resolving power and low resolution respectively; First subscript q representes q position being partitioned into according to aforesaid block of locations dividing method; Second subscript 1～m representes the 1st～m training sample; Represent to be positioned at test low resolution block of locations and high-resolution position piece to be asked after the bilinear interpolation of position q with
Figure FDA0000087559490000025
and
Figure FDA0000087559490000026
respectively, then utilize RBF to try to achieve
Figure FDA0000087559490000027
calculating formula to be:
Figure FDA0000087559490000028
The wherein transposition computing of T representing matrix; multidimensional quadric surface RBF for adopting, concrete calculation expression is:
Figure FDA00000875594900000210
W qBe the weight matrix that the training stage obtains, calculation expression is:
W q = I q H · inv ( φ + τE )
Inv representing matrix inversion operation in the above-mentioned formula, τ are a very little positive constant, and span is 0.001～0.1, and E is a unit matrix, and φ is a matrix, are obtained by training height resolution position piece image calculation:
Figure FDA00000875594900000212
The repeating step aforementioned calculation is till the high-resolution position piece image of obtaining each position.
4. the face image super-resolution reconstruction method of block of locations Nonlinear Mapping according to claim 1; It is characterized in that: said step 3) becomes corresponding super-resolution reconstruction image with high-resolution position piece image mosaic, and the average of when running into overlaid pixel, getting each this grey scale pixel value of high-resolution position piece is the final gray-scale value of overlaid pixel.
CN 201110252310 2011-08-30 2011-08-30 Method for rebuilding super-resolution human face image by position block nonlinear mapping Expired - Fee Related CN102306374B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110252310 CN102306374B (en) 2011-08-30 2011-08-30 Method for rebuilding super-resolution human face image by position block nonlinear mapping

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110252310 CN102306374B (en) 2011-08-30 2011-08-30 Method for rebuilding super-resolution human face image by position block nonlinear mapping

Publications (2)

Publication Number Publication Date
CN102306374A true CN102306374A (en) 2012-01-04
CN102306374B CN102306374B (en) 2013-02-06

Family

ID=45380232

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110252310 Expired - Fee Related CN102306374B (en) 2011-08-30 2011-08-30 Method for rebuilding super-resolution human face image by position block nonlinear mapping

Country Status (1)

Country Link
CN (1) CN102306374B (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102902961A (en) * 2012-09-21 2013-01-30 武汉大学 Face super-resolution processing method based on K neighbor sparse coding average value constraint
CN103514580A (en) * 2013-09-26 2014-01-15 香港应用科技研究院有限公司 Method and system used for obtaining super-resolution images with optimized visual experience
WO2016019484A1 (en) * 2014-08-08 2016-02-11 Xiaoou Tang An apparatus and a method for providing super-resolution of a low-resolution image
CN106709945A (en) * 2017-01-09 2017-05-24 方玉明 Super-resolution image quality evaluation method
CN107135335A (en) * 2016-02-29 2017-09-05 致伸科技股份有限公司 Obtain the method for image and the Extraction of Image device and electronic installation of application this method
CN108171124A (en) * 2017-12-12 2018-06-15 南京邮电大学 A kind of facial image clarification method of similar sample characteristics fitting
CN110580680A (en) * 2019-09-09 2019-12-17 武汉工程大学 face super-resolution method and device based on combined learning
CN110967302A (en) * 2019-11-06 2020-04-07 清华大学 Microbial panoramic smear detection device and detection method
CN112790887A (en) * 2020-12-31 2021-05-14 深圳市维新登拓医疗科技有限公司 Oral imaging method, system and device, and readable storage medium
WO2021104394A1 (en) * 2019-11-28 2021-06-03 RealMe重庆移动通信有限公司 Image processing method and apparatus, electronic device, and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216889A (en) * 2008-01-14 2008-07-09 浙江大学 A face image super-resolution method with the amalgamation of global characteristics and local details information
CN101615290A (en) * 2009-07-29 2009-12-30 西安交通大学 A kind of face image super-resolution reconstruction method based on canonical correlation analysis
CN101635048A (en) * 2009-08-20 2010-01-27 上海交通大学 Super-resolution processing method of face image integrating global feature with local information

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216889A (en) * 2008-01-14 2008-07-09 浙江大学 A face image super-resolution method with the amalgamation of global characteristics and local details information
CN101615290A (en) * 2009-07-29 2009-12-30 西安交通大学 A kind of face image super-resolution reconstruction method based on canonical correlation analysis
CN101635048A (en) * 2009-08-20 2010-01-27 上海交通大学 Super-resolution processing method of face image integrating global feature with local information

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HUA HUANG ET.AL: "Super-Resolution Method for Face Recognition Using Nonlinear Mappings on Coherent Features", 《IEEE TRANSACTIONS ON NEURAL NETWORKS》 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102902961A (en) * 2012-09-21 2013-01-30 武汉大学 Face super-resolution processing method based on K neighbor sparse coding average value constraint
CN102902961B (en) * 2012-09-21 2015-02-18 武汉大学 Face super-resolution processing method based on K neighbor sparse coding average value constraint
CN103514580A (en) * 2013-09-26 2014-01-15 香港应用科技研究院有限公司 Method and system used for obtaining super-resolution images with optimized visual experience
CN103514580B (en) * 2013-09-26 2016-06-08 香港应用科技研究院有限公司 For obtaining the method and system of the super-resolution image that visual experience optimizes
WO2016019484A1 (en) * 2014-08-08 2016-02-11 Xiaoou Tang An apparatus and a method for providing super-resolution of a low-resolution image
CN107135335A (en) * 2016-02-29 2017-09-05 致伸科技股份有限公司 Obtain the method for image and the Extraction of Image device and electronic installation of application this method
CN106709945A (en) * 2017-01-09 2017-05-24 方玉明 Super-resolution image quality evaluation method
CN106709945B (en) * 2017-01-09 2018-03-06 方玉明 A kind of quality evaluating method for super-resolution image
CN108171124A (en) * 2017-12-12 2018-06-15 南京邮电大学 A kind of facial image clarification method of similar sample characteristics fitting
CN108171124B (en) * 2017-12-12 2022-04-05 南京邮电大学 Face image sharpening method based on similar sample feature fitting
CN110580680A (en) * 2019-09-09 2019-12-17 武汉工程大学 face super-resolution method and device based on combined learning
CN110580680B (en) * 2019-09-09 2022-07-05 武汉工程大学 Face super-resolution method and device based on combined learning
CN110967302A (en) * 2019-11-06 2020-04-07 清华大学 Microbial panoramic smear detection device and detection method
WO2021104394A1 (en) * 2019-11-28 2021-06-03 RealMe重庆移动通信有限公司 Image processing method and apparatus, electronic device, and storage medium
CN112790887A (en) * 2020-12-31 2021-05-14 深圳市维新登拓医疗科技有限公司 Oral imaging method, system and device, and readable storage medium

Also Published As

Publication number Publication date
CN102306374B (en) 2013-02-06

Similar Documents

Publication Publication Date Title
CN102306374B (en) Method for rebuilding super-resolution human face image by position block nonlinear mapping
CN101477684B (en) Process for reconstructing human face image super-resolution by position image block
CN102800071B (en) Method for reconstructing super resolution of sequence image POCS
CN102902961B (en) Face super-resolution processing method based on K neighbor sparse coding average value constraint
CN110969577A (en) Video super-resolution reconstruction method based on deep double attention network
CN102722896B (en) Adaptive compressed sensing-based non-local reconstruction method for natural image
CN103093433B (en) Natural image denoising method based on regionalism and dictionary learning
CN103136734B (en) The suppressing method of edge Halo effect during a kind of convex set projection super-resolution image reconstruction
CN105631807A (en) Single-frame image super resolution reconstruction method based on sparse domain selection
CN106952228A (en) The super resolution ratio reconstruction method of single image based on the non local self-similarity of image
Li et al. Example-based image super-resolution with class-specific predictors
CN105678697A (en) Face image super-resolution reconstruction method based on DCT domain eigen transform
CN111724307B (en) Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium
CN106651772A (en) Super-resolution reconstruction method of satellite cloud picture
CN110415169B (en) Depth map super-resolution reconstruction method, system and electronic equipment
CN105513033A (en) Super-resolution reconstruction method based on non-local simultaneous sparse representation
Yang et al. Tensor train rank minimization with hybrid smoothness regularization for visual data recovery
Mikaeli et al. Single-image super-resolution via patch-based and group-based local smoothness modeling
CN109559278B (en) Super resolution image reconstruction method and system based on multiple features study
CN104091364A (en) Single-image super-resolution reconstruction method
CN104200439A (en) Image super-resolution method based on adaptive filtering and regularization constraint
CN110689509A (en) Video super-resolution reconstruction method based on cyclic multi-column 3D convolutional network
CN113240581A (en) Real world image super-resolution method for unknown fuzzy kernel
CN111275620B (en) Image super-resolution method based on Stacking integrated learning
CN117333365A (en) Image super-resolution method based on hybrid transducer super-resolution network

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130206

Termination date: 20160830