CN101806587A - Optical three-dimensional measurement method with absolute phase measurement - Google Patents

Optical three-dimensional measurement method with absolute phase measurement Download PDF

Info

Publication number
CN101806587A
CN101806587A CN 201010161971 CN201010161971A CN101806587A CN 101806587 A CN101806587 A CN 101806587A CN 201010161971 CN201010161971 CN 201010161971 CN 201010161971 A CN201010161971 A CN 201010161971A CN 101806587 A CN101806587 A CN 101806587A
Authority
CN
China
Prior art keywords
pattern
coded
grating
optical
gray
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 201010161971
Other languages
Chinese (zh)
Other versions
CN101806587B (en
Inventor
李勇
钱义先
赵翠芳
金洪震
王辉
金可有
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Normal University CJNU
Original Assignee
Zhejiang Normal University CJNU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Normal University CJNU filed Critical Zhejiang Normal University CJNU
Priority to CN 201010161971 priority Critical patent/CN101806587B/en
Publication of CN101806587A publication Critical patent/CN101806587A/en
Application granted granted Critical
Publication of CN101806587B publication Critical patent/CN101806587B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention belongs to the technical field of an optical three-dimensional sensor technology, particularly relating to an optical three-dimensional measurement method with absolute phase measurement. The invention provides a method which adopts a single assistant coding pattern, which combines the intercepting phase distributing characteristic with the coding pattern deforming characteristic in a phase coding project, and uses 3-value gray level coding, thereby improving the decoding reliability and simplifying the decoding algorithm. By adding a sine optical grating or a logic optical grating pattern with a space gray level coding pattern, the method realizes the absolute phase measurement to compute the three-dimensional feature and the space room of an object.

Description

A kind of optical three-dimensional measuring method that adopts absolute phase to measure
Technical field
The invention belongs to the optical 3-dimensional field of sensing technologies, be specifically related to a kind of optical three-dimensional measuring method that adopts absolute phase to measure.
Background technology
Optical 3-dimensional sensing technology based on Structured Illumination is widely used in fields such as industrial detection, quality control, machine vision, video display stunt and biomedicines.One of key of this technology is the coupling of projection plane and video camera image planes corresponding point, utilizes color, gray scale, raster phase usually or utilizes the symbol of above-mentioned parameter coding or symbol sebolic addressing etc. is realized corresponding point matching.
Whether existing Structured Illumination three-dimensional sensing technology by relevant with color, can be divided into based on color with based on the encoding scheme of gray scale; By the parameter difference that is used to encode, can be divided into sinusoidal grating phase encoding scheme (claiming the sine streak projection scheme again) and light intensity encoding scheme.
The phase encoding scheme mainly contains phase measuring profilometer (DMP) and Fourier transform technology of profiling (FTP) etc.The absolute phase that needs in the phase encoding scheme to obtain grating could realize corresponding point matching, and then the three-dimensional appearance of object and locus can be measured.
Common absolute phase acquisition methods has at present:
(1) the space phase method of development.The grating of projection single-frequency is begun by a certain phase unwrapping starting point, adopts suitable algorithm to carry out phase unwrapping;
(2) time phase the method for development.The grating of two above different frequencies of projection utilizes the relation between these raster phases to carry out phase unwrapping;
(3) utilize auxiliary coding pattern to determine that the grating fringe level is inferior, launch phase place.
The branch that single width and several coding patterns are arranged again in (3) the kind method.Adopt several coding patterns can obtain reliable measurement result, coding/decoding method is simple, but Measuring Time is longer; And adopt the single width coding pattern can improve measuring speed, but ordinary solution code calculation complexity, reliability is low relatively.
Summary of the invention
The invention provides a kind of employing single width auxiliaring coding method of patterning, it uses 3 value gray-codeds in conjunction with blocking PHASE DISTRIBUTION characteristics and coding pattern deformation characteristics in the phase encoding scheme, improves decoding reliability, and has simplified decoding algorithm.Promptly adopt sinusoidal grating or Luo Qi grating pattern to add a width of cloth space gray-coded method of patterning and realize the absolute phase measurement, and then calculate the three-dimensional appearance and the locus of object.
Concrete technology path of the present invention is:
A kind of optical three-dimensional measuring method that adopts absolute phase to measure may further comprise the steps:
1), one is equipped with the projector of grating to object projective structure light pattern;
2), a video camera is taken the deformation pattern that is reflected by object in the another location;
3), deformation pattern that object is reflected resolves, and obtains the coordinate of putting on the projection plane corresponding with point on the video camera image planes;
4), calculating object measurement point coordinate according to geometric relationship carries out three-dimensionalreconstruction, the acquisition object three-dimensional image;
It is characterized in that
The described structured light patterns of step 1) comprises grating pattern and space gray-coded pattern,
This space gray-coded pattern is made up by following steps:
A), space gray-coded pattern is made of some bands corresponding with grating fringe, the width of band is identical with fringe period;
B, utilize in the band gray-value variation rule to determine a group of coded symbols along strip direction;
C), the pseudo-random sequence that is made up by this group of coded symbols is determined space gray-coded pattern.
Described pseudo-random sequence construction method comprises following steps:
A), the picked at random number equals the coded identification of length of window from a group of coded symbols of appointment, constitutes initiation sequence;
B), from this group of coded symbols 1 symbol of picked at random, add to and form new sequence after the sequence that has made up;
C), judge whether this new sequence meets the following conditions;
Subsequence more than or equal to length of window is unique,
No repeated encoding symbol in the subsequence that equals length of window arbitrarily,
As meet, check then whether whole sequence length is not less than the grating fringe number, if satisfy, make up and finish, otherwise change b) step,
As do not meet, then the firm coded identification of adding of deletion is changeed b) step.
The gray-scale value of described space gray-coded pattern comprises 3 grades.
Fig. 1 is an optical grating projection three-dimension measuring system synoptic diagram.It is made of projector and video camera.O p, O cBe respectively projector and video camera photocentre, (x P0, y P0), (x C0, y C0) be respectively the image coordinate of projector and video camera principal point.P p, P cCorresponding point for a pair of projection plane and video camera image planes.P is the measurement point on the body surface, and its coordinate is (X w, Y w, Z w).Launch sinusoidal grating or Luo Qi grating by projector, video camera is taken the deforming stripe that is reflected by body surface in the another location.By finding the solution the image absolute phase that photographs, obtain the coordinate of putting on the projection plane corresponding with point on the video camera image planes.At last by PO cO pIn the triangle that constitutes, calculate the P point coordinate according to geometric relationship, and then recover object dimensional pattern and locus.
The basic design philosophy of gray space coding pattern is:
At first the gray space coding pattern is divided into some bands corresponding with projected fringe, the width of band is identical with fringe period, i.e. the one-period of a corresponding striped of band.
Utilize in the band along the intensity profile rule of the strip direction class symbol of encoding.
Use the pseudo-random sequence that constitutes by this class symbol to determine the general structure of pattern then.
Wherein, pseudo-random sequence should satisfy following condition:
(1) in sequence, be unique more than or equal to the subsequence of length-specific (being called length of window);
(2) in equaling the subsequence of length of window, random length do not have replicator.It such as length of window 4 sequence " ABDECFADBE FDBECDABFECBDEFBDCEABCDAECFBDE ".
The coding method of symbol is: according to whether change along the strip direction gray scale, gray-scale value and change rule such as amplitude and encode.For example, adopt 3 grades of gray scales to encode, be expressed as 0,1,2.Along strip direction gray scale no change, can think that also Changing Pattern is: 0-0,1-1,2-2, symbol of every kind of gray-scale value coding, 3 symbols of can encoding.Along the periodically two-value variation of strip direction gray scale, according to the gray-scale value difference of these two variations, 3 symbols of can encoding are respectively: 0-1, and 0-2, the 1-2 two-value changes.Comprehensive above-mentioned two class situations, 6 symbols of can encoding altogether can be expressed as respectively: A (0-0), B (1-1), C (2-2), D (0-1), E (0-2), F (1-2).
Fig. 2 gives an example for coding pattern, surpasses the raster phase saltus step line that the point of π is formed by connecting by adjacent phase hit absolute value and is positioned at the band center.
The decoding algorithm scheme that the present invention adopts is as follows:
The coding pattern that actual measurement obtains as shown in Figure 3, the brighter line of color is the saltus step line among the figure.Because the phase hit line is positioned at the band center, it is less that its gray scale is influenced by adjacent ribbons, and band is out of shape with the saltus step line.Sinusoidal grating striped corresponding codes can be determined by coding pattern grey scale change situation on the analysis saltus step line.Decoding algorithm is as follows:
(1) removes of the influence of body surface reflectivity to coded image;
(2) according to threshold value coded image being carried out three values quantizes;
(3) along saltus step line computation grey level histogram, the some proportion of a certain gray scale just confirms that greater than threshold value this gray scale exists;
(4) recover code according to coding rule.
Description of drawings
Fig. 1 is an optical grating projection 3-D imaging system synoptic diagram of the present invention
The coding pattern synoptic diagram that Fig. 2 gives an example for the present invention
The actual measurement coding pattern synoptic diagram that Fig. 3 gives an example for the present invention
Fig. 4 is sine of the present invention and encode grating synoptic diagram
Fig. 5 is inventor's face expression shape change measurement result synoptic diagram
Fig. 6 is a complex scene measurement result synoptic diagram of the present invention
Embodiment
Embodiment 1
A kind of optical three-dimensional measuring method that adopts absolute phase to measure may further comprise the steps:
1), one is equipped with the projector of grating to object projective structure light pattern;
2), a video camera is taken the deformation pattern that is reflected by object in the another location;
3), deformation pattern that object is reflected resolves, and obtains the coordinate of putting on the projection plane corresponding with point on the video camera image planes;
4), calculating object measurement point coordinate according to geometric relationship carries out three-dimensionalreconstruction, the acquisition object three-dimensional image;
It is characterized in that
The described structured light patterns of step 1) comprises grating pattern and space gray-coded pattern,
This space gray-coded pattern is made up by following steps:
A), space gray-coded pattern is made of some bands corresponding with projected fringe, the width of band is identical with fringe period;
B, utilize in the band gray-value variation rule to determine a group of coded symbols along strip direction;
C), the pseudo-random sequence that is made up by this group of coded symbols is determined space gray-coded pattern.
Described pseudo-random sequence construction method comprises following steps:
A), the picked at random number equals the coded identification of length of window from a group of coded symbols of appointment, constitutes initiation sequence;
B), from this group of coded symbols 1 symbol of picked at random, add to and form new sequence after the sequence that has made up;
C), judge whether this new sequence meets the following conditions;
Subsequence more than or equal to length of window is unique,
No repeated encoding symbol in the subsequence that equals length of window arbitrarily,
As meet, check then whether whole sequence length is not less than the grating fringe number, if satisfy, make up and finish, otherwise change b) step,
As do not meet, then the firm coded identification of adding of deletion is changeed b) step.
The gray-scale value of described space gray-coded pattern comprises 3 grades.
Concrete experimental procedure is as follows:
According to pixel columns, the grating computation of Period pseudo-random sequence length of projector, adopt preceding method structure pseudo-random sequence;
According to the gray-coded pattern coding method of aforesaid space, design and each symbol corresponding codes band;
According to the code sequence of structure, coding unit is arranged in order, obtain at last and pseudo-random sequence corresponding codes pattern;
Grating and coding pattern that design is obtained project on the testee, and take deformed grating and the coding pattern of being modulated by object appearance with video camera simultaneously;
Behind intact all patterns of projection, find the solution the phase place of blocking of grating, and block phase place with the coding pattern decoding that photographs in conjunction with what obtain, calculate absolute phase, obtain the three-dimensional appearance and the locus of object at last.
The employing length of window is 3 sequence:
" ABDECFACDCEFBEADFEBCAEDAFDBFCBADEFCABEDC F ", symbol is worth gray-codeds with 3.At first carry out the dynamic process experiments of measuring with π phase shift Fourier transform technology of profiling in conjunction with coding pattern, the grating cycle is 18 pixels.Writing speed is per second 60 frame three-dimensional datas.Fig. 4 (a), Fig. 4 (b), Fig. 4 (c) overlap grating for one of design, and Fig. 4 (a) is a sinusoidal grating, and Fig. 4 (b) is the dephased sinusoidal grating of π, and Fig. 4 (c) is an encode grating.
Fig. 5 is for laughing at people's face measure of the change result in the process.Adopt five step phase-shift methods to measure the scene of forming by some isolated objects in conjunction with coding pattern then.Fig. 6 (a) is tested scene photo, and wherein wall and plaster statue are apart from about 690mm; Fig. 6 (b) is a measurement result; Fig. 6 (c) is the result behind the removal wall.As seen adopt this method can measure the three-dimensional appearance and the locus of complex scene reliably, and the bigger measurement depth of field is arranged.

Claims (3)

1. optical three-dimensional measuring method that adopts absolute phase to measure may further comprise the steps:
1), one is equipped with the projector of grating to object projective structure light pattern;
2), a video camera is taken the deformation pattern that is reflected by object in the another location;
3), deformation pattern that object is reflected resolves, and obtains the coordinate of putting on the projection plane corresponding with point on the video camera image planes;
4), calculating object measurement point coordinate according to geometric relationship carries out three-dimensionalreconstruction, the acquisition object three-dimensional image;
It is characterized in that
The described structured light patterns of step 1) comprises grating pattern and space gray-coded pattern, and this space gray-coded pattern is made up by following steps:
A), space gray-coded pattern is made of some bands corresponding with grating fringe, the width of band is identical with fringe period;
B), utilize in the band gray-value variation rule to determine a group of coded symbols along strip direction;
C), the pseudo-random sequence that is made up by this group of coded symbols is determined space gray-coded pattern.
2. optical three-dimensional measuring method according to claim 1 is characterized in that described pseudo-random sequence construction method comprises following steps:
A), the picked at random number equals the coded identification of length of window from a group of coded symbols of appointment, constitutes initiation sequence;
B), from this group of coded symbols 1 symbol of picked at random, add to and form new sequence after the sequence that has made up;
C), judge whether this new sequence meets the following conditions;
Subsequence more than or equal to length of window is unique,
No repeated encoding symbol in the subsequence that equals length of window arbitrarily,
As meet, check then whether whole sequence length is not less than the grating fringe number,
If satisfy, make up and finish, otherwise change b) step,
As do not meet, then the firm coded identification of adding of deletion is changeed b) step.
3. optical three-dimensional measuring method according to claim 1 is characterized in that described space gray-coded pattern comprises ash in class III degree value.
CN 201010161971 2010-04-29 2010-04-29 Optical three-dimensional measurement method with absolute phase measurement Expired - Fee Related CN101806587B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201010161971 CN101806587B (en) 2010-04-29 2010-04-29 Optical three-dimensional measurement method with absolute phase measurement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201010161971 CN101806587B (en) 2010-04-29 2010-04-29 Optical three-dimensional measurement method with absolute phase measurement

Publications (2)

Publication Number Publication Date
CN101806587A true CN101806587A (en) 2010-08-18
CN101806587B CN101806587B (en) 2012-12-12

Family

ID=42608478

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201010161971 Expired - Fee Related CN101806587B (en) 2010-04-29 2010-04-29 Optical three-dimensional measurement method with absolute phase measurement

Country Status (1)

Country Link
CN (1) CN101806587B (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102012892A (en) * 2010-12-13 2011-04-13 上海大学 Reconfigurable ternary optical processor
CN102243138A (en) * 2011-08-05 2011-11-16 中国科学院光电技术研究所 Focal plane detection device for projection lithography
CN102322823A (en) * 2011-09-13 2012-01-18 四川大学 Phase grade self-coding-based optical three-dimensional measurement method
WO2012071817A1 (en) * 2010-11-30 2012-06-07 东南大学 Three-dimensional measurement method based on wavelet transform
CN103398675A (en) * 2013-08-10 2013-11-20 南昌航空大学 Three-dimensional measuring method for complex large object based on stripe period correction
CN104132628A (en) * 2014-07-27 2014-11-05 四川大学 Method for achieving linear structured light three-dimensional measurement by means of phase calculation
CN104180773A (en) * 2013-05-20 2014-12-03 沈阳同联集团高新技术有限公司 Three-dimensional topography measuring device
CN104697468A (en) * 2015-03-30 2015-06-10 南昌航空大学 Optical three-dimensional measurement method based on tilted phase encoding stripes
CN106840039A (en) * 2017-03-10 2017-06-13 合肥工业大学 A kind of quick three-dimensional measuring method based on composite grating projection
CN106910213A (en) * 2017-01-23 2017-06-30 四川大学 It is a kind of based on the not visible scene three-dimensional information acquisition methods of optics for being calculated as picture
CN109945802A (en) * 2018-10-11 2019-06-28 宁波深浅优视智能科技有限公司 A kind of structural light three-dimensional measurement method
CN109974626A (en) * 2019-04-08 2019-07-05 四川大学 A kind of structural light three-dimensional measurement method based on phase-shift phase strip encoding level
CN110986828A (en) * 2019-12-13 2020-04-10 杨鑫鑫 Novel real scene three-dimensional color data acquisition and display method
TWI699525B (en) * 2019-05-21 2020-07-21 中華學校財團法人中華科技大學 Three-dimensional phase shift defect detection method and system
CN111947601A (en) * 2020-08-12 2020-11-17 上海科技大学 Projection resolving method for gray-scale pseudo-random coding structure light striations
EP3577416A4 (en) * 2017-02-06 2020-12-16 Modit3d, Inc. System and method for 3d scanning

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101504277A (en) * 2009-02-26 2009-08-12 浙江师范大学 Method for acquiring object three-dimensional image by optical three-dimensional sensing

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101504277A (en) * 2009-02-26 2009-08-12 浙江师范大学 Method for acquiring object three-dimensional image by optical three-dimensional sensing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
《中国优秀硕士学位论文全文数据库基础科学辑》 20080215 纪文丽 基于光栅投影方法的物体三维轮廓重构 第3.4节 1-3 , 第2期 2 *

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9163937B2 (en) 2010-11-30 2015-10-20 Southeast University Three-dimensional measurement method based on wavelet transform
WO2012071817A1 (en) * 2010-11-30 2012-06-07 东南大学 Three-dimensional measurement method based on wavelet transform
CN102012892A (en) * 2010-12-13 2011-04-13 上海大学 Reconfigurable ternary optical processor
CN102243138A (en) * 2011-08-05 2011-11-16 中国科学院光电技术研究所 Focal plane detection device for projection lithography
CN102322823A (en) * 2011-09-13 2012-01-18 四川大学 Phase grade self-coding-based optical three-dimensional measurement method
CN104180773A (en) * 2013-05-20 2014-12-03 沈阳同联集团高新技术有限公司 Three-dimensional topography measuring device
CN103398675B (en) * 2013-08-10 2016-06-15 南昌航空大学 A kind of complicated big three-dimensional measurement of objects method based on fringe period correction
CN103398675A (en) * 2013-08-10 2013-11-20 南昌航空大学 Three-dimensional measuring method for complex large object based on stripe period correction
CN104132628A (en) * 2014-07-27 2014-11-05 四川大学 Method for achieving linear structured light three-dimensional measurement by means of phase calculation
CN104697468A (en) * 2015-03-30 2015-06-10 南昌航空大学 Optical three-dimensional measurement method based on tilted phase encoding stripes
CN106910213B (en) * 2017-01-23 2019-09-03 四川大学 A kind of not visible scene three-dimensional information acquisition methods of optics based on calculating imaging
CN106910213A (en) * 2017-01-23 2017-06-30 四川大学 It is a kind of based on the not visible scene three-dimensional information acquisition methods of optics for being calculated as picture
US11330243B2 (en) 2017-02-06 2022-05-10 Riven, Inc. System and method for 3D scanning
EP3577416A4 (en) * 2017-02-06 2020-12-16 Modit3d, Inc. System and method for 3d scanning
CN106840039A (en) * 2017-03-10 2017-06-13 合肥工业大学 A kind of quick three-dimensional measuring method based on composite grating projection
CN109945802B (en) * 2018-10-11 2021-03-09 苏州深浅优视智能科技有限公司 Structured light three-dimensional measurement method
CN109945802A (en) * 2018-10-11 2019-06-28 宁波深浅优视智能科技有限公司 A kind of structural light three-dimensional measurement method
CN109974626A (en) * 2019-04-08 2019-07-05 四川大学 A kind of structural light three-dimensional measurement method based on phase-shift phase strip encoding level
CN109974626B (en) * 2019-04-08 2021-02-09 四川大学 Structured light three-dimensional measurement method based on phase shift amount coding stripe level
TWI699525B (en) * 2019-05-21 2020-07-21 中華學校財團法人中華科技大學 Three-dimensional phase shift defect detection method and system
CN110986828A (en) * 2019-12-13 2020-04-10 杨鑫鑫 Novel real scene three-dimensional color data acquisition and display method
CN110986828B (en) * 2019-12-13 2023-09-01 杨鑫鑫 Novel acquisition and display method for three-dimensional color data of real scene
CN111947601A (en) * 2020-08-12 2020-11-17 上海科技大学 Projection resolving method for gray-scale pseudo-random coding structure light striations

Also Published As

Publication number Publication date
CN101806587B (en) 2012-12-12

Similar Documents

Publication Publication Date Title
CN101806587B (en) Optical three-dimensional measurement method with absolute phase measurement
CN106840036B (en) A kind of diadactic structure light optimization method suitable for fast three-dimensional appearance measuring
CN101813461B (en) Absolute phase measurement method based on composite color fringe projection
CN101694375B (en) Stereoscopic vision detecting method for measuring three-dimensional morphology on strong reflection surface
CN101504277B (en) Method for acquiring object three-dimensional image by optical three-dimensional sensing
Pages et al. Optimised De Bruijn patterns for one-shot shape acquisition
CN110645919B (en) Structured light three-dimensional measurement method based on airspace binary coding
CN102322823B (en) Phase grade self-coding-based optical three-dimensional measurement method
CN109341589B (en) Grating image projection method, three-dimensional reconstruction method and three-dimensional reconstruction system
CN106643562B (en) Structural light stripes projective techniques based on time domain airspace hybrid coding
CN110174079B (en) Three-dimensional reconstruction method based on four-step phase-shift coding type surface structured light
US9633439B2 (en) Image processing system, and image processing method
Xu et al. Rapid 3D surface profile measurement of industrial parts using two-level structured light patterns
Song et al. A new phase unwrapping algorithm based on three wavelength phase shift profilometry method
CN108955571B (en) The method for three-dimensional measurement that double frequency heterodyne is combined with phase-shift coding
CN104677308A (en) Three-dimensional scanning method for high-frequency two-value strip
CN104567730B (en) A kind of method that space-time binary coding produces sinusoidal light field
CN104390608A (en) Projection grating phase method based structured light three-dimensional shape construction method
CN110849290B (en) Three-dimensional measurement method for segmented quantization coding intensity based on morphological operation
CN111207692A (en) Improved segmented step phase coding three-dimensional measurement method
CN116718133A (en) Short-distance single-point structured light three-dimensional measurement method
CN113345039B (en) Three-dimensional reconstruction quantization structure optical phase image coding method
CN107421467B (en) Structural light three-dimensional measurement method and system
CN101726259A (en) Cyclic code 3D measurement method with the combination of trapezoid grayscale and binary grayscale
US20180112975A1 (en) Fringe Projection Method, Fringe Projection Device and Computer Program Product (as translated)

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20121212

Termination date: 20150429

EXPY Termination of patent right or utility model