CN101788667A - Light amplification type three-dimensional imaging method and system - Google Patents

Light amplification type three-dimensional imaging method and system Download PDF

Info

Publication number
CN101788667A
CN101788667A CN201010100326A CN201010100326A CN101788667A CN 101788667 A CN101788667 A CN 101788667A CN 201010100326 A CN201010100326 A CN 201010100326A CN 201010100326 A CN201010100326 A CN 201010100326A CN 101788667 A CN101788667 A CN 101788667A
Authority
CN
China
Prior art keywords
light
light amplification
module
image
image processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201010100326A
Other languages
Chinese (zh)
Other versions
CN101788667B (en
Inventor
张秀达
杨超
严惠民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN2010101003263A priority Critical patent/CN101788667B/en
Publication of CN101788667A publication Critical patent/CN101788667A/en
Application granted granted Critical
Publication of CN101788667B publication Critical patent/CN101788667B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Optical Radar Systems And Details Thereof (AREA)
  • Studio Devices (AREA)

Abstract

The invention discloses a light amplification type three-dimensional imaging method and a system. A first output of a sequence scheduling control module controls a pulsed laser to emit intense pulsed lights; the intense pulsed lights irradiate at an object to be tested after penetrating through a beam expanding system; returned signal lights enter into a light amplification module for amplifying after penetrating through an imaging system; after the amplified lights are split, part of lights are received by a first vidicon after penetrating through a gainmodulation receiver, and the other parts of lights are received by a second vidicon; and the other three routes of the sequence scheduling control module respectively control the light amplification module, the gainmodulation receiver and an image processing module. Images obtained by the two vidicons are processed by the image processing module, and the distances from each pixel point of the image to the object are worked out to obtain the three-dimensional image of the object. Under the conditions that the light source power and the gate time are not changed, the invention can reduce the influence of random shot noise to signal lights and improve the precision of ranging.

Description

A kind of light amplification type three-dimensional imaging method and system
Technical field
The present invention relates to the method and system of three-dimensional imaging, especially relate to a kind of light amplification type three-dimensional imaging method and system.
Background technology
Along with the development need of human scientific-technical progress with society, high speed aircraft navigations such as aircraft, the round-the-clock target recognition and tracking of over the horizon, application such as geographical relief high precision exploration have all proposed high requirement to long-range target acquisition.The fast development of three-dimensional imaging laser radar solves problems such as long-range detection new research direction is provided for people.But traditional three-dimensional imaging laser radar system capture to such an extent that the light signal signal to noise ratio (S/N ratio) is lower, thereby the performance at aspects such as resolution of ranging is difficult to that breakthrough raising is arranged when surveying target far away.So study high performance three-dimensional imaging technology and be one of focus that people pay close attention to.
Avalanche photodide (AvalanchePhoto Diode, APD) the three-dimensional ranging technology of range finding single-point module construction of Geiger (Geiger) pattern adopted in the Lincoln laboratory of the U.S..This method is subject to technological level, and pixel count is low, and it is serious to crosstalk between pixel.People such as Denmark Andersen have invented the door gate type three-dimensional laser radar technology based on time slice (TimeSlicing).The limitation of this method need to be tens of width of cloth intensity maps could pass through to calculate and to obtain a width of cloth 3-D view.People such as the graduate Martin Laurenzis in moral method St. Louis utilize light pulse of high-quality rectangle and door gating, can obtain trapezoidal distance-The Relationship of Light intensity.Two width of cloth images can obtain 3-D view.The limitation of this method need to be the modulation quality of strict control light pulse and receptacle.
The applicant had once proposed no scanning pulsed modulation formula three-D imaging method and system's (patent No. 20051004946.3), the measuring accuracy of this system is irrelevant with the shape of emission laser pulse, but in actual conditions, it is far away to be subjected to detection of a target distance, light source power can not be excessive, the receiving optics bore can not be excessive etc. the influence of factor, make light signal that detector obtains a little less than, cause signal to noise ratio (S/N ratio) to descend, measuring accuracy reduces.
Summary of the invention
The object of the present invention is to provide a kind of light amplification type three-dimensional imaging method and system, improve three-dimensional imaging laser radar range performance.
The technical solution used in the present invention is as follows:
One, a kind of light amplification type three-dimensional imaging method:
Laser instrument produces light pulse, shining testee returns, adopt the light amplification module, the pulsed optical signals that collects is amplified, pass through beam splitting system then, with the light signal separated into two parts after amplifying, a part is received by video camera by the gain modulation receiver, and another part is directly received by another video camera, with image processing module the image that receives is handled, calculate the distance of the corresponding target of each picture element in the image, generate 3-D view.
Two, a kind of light amplification type three-dimensional imaging system:
Comprise the clock signal control module, pulsed laser, beam-expanding system, imaging system, gain modulation receiver, two video cameras and image processing module; The first via output control pulse laser instrument of clock signal control module sends pulsed light by beam-expanding system, is radiated on the testee.The light that fires back enters the light amplification module through imaging system, be divided into two-way after the light signal process beam splitting system beam split that is exaggerated, lead up to the gain modulation receiver, received by first video camera, another road is received by second video camera, the image that two video cameras obtain outputs to image processing module respectively, and second, third of clock signal control module, the four tunnel is connected with image processing module with light amplification module, gain modulation receiver respectively.
Described pulsed laser is the laser instrument of optical pulse width greater than tens nanoseconds; Described light amplification module is the video screen fluorescence lifetime image intensifier shorter than the light pulse duration, or is made up of many optical fiber with enlarging function; Described gain modulation receiver is an image intensifier; Described image processing module is personal computer, DSP digital information processing system or flush bonding processor.
The beneficial effect that the present invention has is:
The present invention amplifies flashlight by the light amplification module under the condition that does not change light source power and door gating time, reduces at random shot noise to the influence of flashlight, thereby has improved distance accuracy.
Description of drawings
Accompanying drawing is a structural principle synoptic diagram of the present invention.
Among the figure: 1, clock signal control module, 2, pulsed laser, 3, beam-expanding system, 4, imaging system, 5, the light amplification module, 6, beam splitting system, 7, the gain modulation receiver, 8, first video cameras, 9, second video cameras, 10, image processing module.
Embodiment
Below in conjunction with drawings and Examples method and system of the present invention is described further.
The principle that the present invention improves detection accuracy is:
By laser radar equation, the laser power that the single pixel-by-pixel basis of detection system is received can be expressed as:
P R = P T π D 2 β 4 πi R 2 e 2 γR - - - ( 1 )
P in the formula RBe the laser power that single pixel-by-pixel basis is received, P TBe the emission laser power, D is a receiving aperture, and β is the reflectivity of target, and γ is the proportion of goods damageds of transmitting in medium, and R is the laser radar range-to-go, and i is the pixel count of detector.
In order to suppress bias light and rear orientation light enters system, need before receiving system, add narrow band filter slice
And adopt the mode of door gating to survey.Suitable door gating time is set, guarantees the return signal of the laser pulse of reception that the gating process is complete each time.Then the receiving plane pixel receives the expectation E (N) that light pulse comprises photon number N and is:
E ( N ) = mη P R hυ - - - ( 2 )
M is the emitting times of light pulse in once surveying in the formula, and η is the quantum efficiency of receiver, and h is a Planck constant, and υ is the incident light frequency.
There is the randomness that arrives room and time at spatial transmission in light signal to receiving plane, this process is a Poisson stochastic process.Obtaining standard deviation by Poisson distribution is σ = N , σ represents the fluctuating of light signal, is referred to as shot noise.Owing to require during three-dimensional imaging the signal to noise ratio (S/N ratio) of image higher relatively, the photon number that light signal comprises is generally greater than 1000, so the error that causes of shot noise is much larger than the influence of other factors.The signal to noise ratio snr of tested light signal 1Can be expressed as:
SNR 1 = N N = N = mη P R hυ - - - ( 3 )
By (3) formula as can be known, the signal to noise ratio (S/N ratio) of measured signal is directly proportional with the square root of received optical power.The power that is to say receiving optical signals is big more, and signal to noise ratio (S/N ratio) is high more.
1. when not adding the light amplification module:
Flashlight is received by video camera by the gain modulation receiver through a part after the beam splitting system without amplification module, and another part is directly received by video camera.
If the light pulse after the beam split is respectively in the photon number that a certain pixel comprised: N 1+ σ 1And N 1+ σ 2, σ wherein 1, σ 2Represent the shot noise that this pixel light signal is additional respectively.Shot noise meets Poisson distribution, therefore has E ( σ 1 ) = E ( σ 2 ) = N 1 .
If the gain modulation function of gain modulation receiver is f (t), f (t) is at [t 1, t 2] be dull, t 1And t 2Represent the time that the door gating begins and finishes respectively.T represent light from light source through scene reflectivity to detector institute elapsed time, it and scene have following relation to detector apart from x:
x = 1 2 V C t - - - ( 4 )
V in the formula cThe velocity of propagation of light in medium for this kind wavelength.After the gain modulation receiver was advanced in a part of light pulse after the beam split, the photon number M that receives of this pixel on the video camera 1For:
M 1=f(t)(N 11) (5)
Another part light pulse after the beam split directly enters video camera, the photon number M that receives of this pixel on the video camera 2For:
M 2=N 12 (6)
(5) (6) are divided by and are obtained:
M 1 M 2 = f ( t ) ( N 1 + σ 1 ) N 1 + σ 2 ≈ f ( t ) [ 1 + σ 1 + σ 2 N 1 ] - - - ( 7 )
The uncertain factor that introduce this moment is: E ( σ 1 + σ 2 N 1 ) = 2 N 1 N 1 = 2 N 1 - - - ( 8 )
2. when adding the light amplification module:
Flashlight is received by video camera by the gain modulation receiver through a part after the beam splitting system through behind the amplification module, and another part is directly received by video camera.
Light pulse in the photon number that a certain pixel comprised is before amplifying: N 3+ σ 3, σ wherein 3The additional shot noise of expression light signal, E ( σ 3 ) = N 3 .
If the enlargement factor of light amplification module is k, behind the flashlight process amplification module, the photon number M that light pulse comprised of this pixel outgoing 3For:
M 3=k(N 33)+σ 4 (9)
σ in the formula 4Be the noise of introducing in the amplification process.Amplify because photon is received the back by photocathode, photoelectronic outgoing also is a typical Poisson process.So E ( σ 4 ) = k ( N 3 + σ 3 ) . By (3) formula as can be known: after the light amplification module, it is big that optical signal power becomes, and signal to noise ratio (S/N ratio) improves.The light signal signal to noise ratio snr of this pixel 2Can be expressed as:
SNR 2 = M 3 = k ( N 3 + σ 3 ) + σ 4 - - - ( 10 )
When k was big, signal to noise ratio (S/N ratio) was higher, the noise σ that introduces in the amplification process 4Relative signal can be ignored.So the photon number that light pulse comprised of this pixel outgoing can be reduced to M after amplifying 4:
M 4=k(N 33) (11)
After the light signal process 1:1 beam split after the amplification, the photon number that the corresponding pixel points light pulse is comprised is respectively: With σ wherein 5, σ 6Be respectively light beam after the amplification additional shot noise in transmission course.Shot noise meets Poisson distribution, therefore has E ( σ 5 ) = E ( σ 6 ) = M 4 2 .
If the gain modulation function of gain modulation receiver is g (t), g (t) is at [t 1, t 2] be dull, t 1And t 2Represent the time that the door gating begins and finishes respectively.The photon number M that receives of this pixel on the video camera 5, M 6Be respectively:
M 5 = g ( t ) [ M 4 2 + σ 5 ] - - - ( 12 )
M 6 = M 4 2 + σ 6 - - - ( 13 )
(12) (13) are divided by and are obtained:
M 5 M 6 = g ( t ) [ M 4 2 + σ 5 ] M 4 2 + σ 6 = g ( t ) ( 1 + σ 5 + σ 6 M 4 2 ) - - - ( 14 )
The uncertain factor of bringing thus is: E ( σ 5 + σ 6 M 4 2 ) = 2 M 4 2 M 4 2 ≈ 2 kN 3 2 .
When k →+during ∞, 2 kN 3 2 → 0 . Therefore, when the gain of light amplification module was big, the uncertain factor leveled off to 0.
Below specific embodiments of the present invention is elaborated: as shown in Figure 1, system of the present invention comprises: clock signal control module 1, pulsed laser 2, beam-expanding system 3, imaging system 4, light amplification module 5, beam splitting system 6, gain modulation receiver 7, two video cameras 8,9 and image processing module 10; The first via output control pulse laser instrument 2 of clock signal control module 1 sends pulsed light by beam-expanding system 3, is radiated on the testee.The light that fires back enters light amplification module 5 through imaging system 4, be divided into two-way after light signal process beam splitting system 6 beam split that are exaggerated, lead up to gain modulation receiver 7, received by first video camera 8, another road is received by second video camera 9, the image that two video cameras 8,9 obtain outputs to image processing module 10 respectively, and second, third of clock signal control module 1, the four tunnel is connected with light amplification module 5, gain modulation receiver 7 and image processing module 10 respectively.
Described pulsed laser 2 is the laser instrument of optical pulse width greater than tens nanoseconds; Described light amplification module 5 is video screen fluorescence lifetime image intensifiers shorter than the light pulse duration, or is made up of many optical fiber with enlarging function; Described gain modulation receiver 7 is image intensifiers; Described image processing module 10 is personal computer, DSP digital information processing system or flush bonding processor.
The output intensity of image intensifier is directly proportional with input light intensity and gain product.In light amplification module 5, set suitable image intensifier gain coefficient, the light signal that will contain scene information amplifies.The gain modulation function of setting image intensifier in the gain modulation receiver 7 is monotonic quantity in the door gating time.During intensity image by once or several times exposures obtain.After obtaining intensity image, can calculate range image.Generate 3-D view thus.
1, pulsed laser 2 sends pulsed light through beam-expanding system 3, the tested scene of throwing light on.The light intensity I that pixel receives when entering imaging system 4 1Can be expressed as:
I 1=Kn(P S+P N) (16)
K is a scale-up factor in the formula, and the transformational relation between expression light intensity and the luminous energy, n are the number of times of door gating in once surveying, P SBe the luminous power of back light signal, P NBe in the flashlight communication process because the time and space uncertainty that arrives causes the luminous power of shot noise correspondence.
2, flashlight enters light amplification module 5 by imaging system 4.Because by after the light amplification module, it is big that optical signal power becomes, signal to noise ratio (S/N ratio) improves, and the noise relative signal of introducing in the amplification process can be ignored.The image intensifier gain of setting light amplification module 5 is under the situation of constant, the light intensity I that this picture element receives on the video screen of light amplification module 5 2Can be expressed as:
I 2=AI 1 (17)
A is the gain of image intensifier in the formula.
3, after process beam splitting system 6 was pressed the 1:1 beam split, part signal light was received by video camera 8 by gain modulation receiver 7.The light intensity I that this pixel receives on the video camera 8 3Can be expressed as
I 3 = I 2 2 h ( t ) - - - ( 18 )
H in the formula (t) is the gain modulation function of gain modulation device image intensifier.H (t) is at [t 1, t 2] be dull, t 1And t 2Represent the time that the door gating begins and finishes respectively.
4, after process beam splitting system 6 was pressed the 1:1 beam split, another part flashlight directly entered video camera 9.The light intensity I that this pixel receives on the video camera 9 4Can be expressed as follows:
I 4 = I 2 2 - - - ( 19 )
5, two video cameras 8,9 are demarcated by experiment, determine the corresponding relation of same pixel gray scale and position after, can obtain the ratio of the light intensity of same pixel point:
I 3 I 4 = h ( t ) - - - ( 20 )
6, by formula (4) (20), and h (t) is at [t 1, t 2] be dull, so h -1(t) exist.Can obtain relative position x, can be expressed as in the corresponding target of interior this pixel of gating distance:
x = 1 2 V C h - 1 ( I 3 I 4 ) - - - ( 21 )
V in the formula CRepresent the velocity of propagation of this kind wavelength in medium.

Claims (3)

1. light amplification type three-dimensional imaging method, it is characterized in that: laser instrument produces light pulse, shining testee returns, adopt the light amplification module, the pulsed optical signals that collects is amplified, pass through beam splitting system then, with the light signal separated into two parts after amplifying, a part is received by video camera by the gain modulation receiver, another part is directly received by another video camera, with image processing module the image that receives is handled, calculated the distance of the corresponding target of each picture element in the image, generate 3-D view.
2. an a kind of light amplification type three-dimensional imaging system that implements the described method of claim 1 comprises clock signal control module (1), pulsed laser (2), beam-expanding system (3), imaging system (4), gain modulation receiver (7), two video cameras (8,9) and image processing module (10); The first via output control pulse laser instrument (2) of clock signal control module (1) sends pulsed light by beam-expanding system (3), is radiated on the testee; It is characterized in that: the light that fires back enters light amplification module (5) through imaging system (4), be divided into two-way after light signal process beam splitting system (6) beam split that is exaggerated, lead up to gain modulation receiver (7), received by first video camera (8), another road is received by second video camera (9), the image that two video cameras (8,9) obtain outputs to image processing module (10) respectively, and second, third of clock signal control module (1), the four tunnel is connected with light amplification module (5), gain modulation receiver (7) and image processing module (10) respectively.
3. a kind of light amplification type three-dimensional imaging according to claim 2 system, it is characterized in that: described pulsed laser (2) is the laser instrument of optical pulse width greater than tens nanoseconds; Described light amplification module (5) is the video screen fluorescence lifetime image intensifier shorter than the light pulse duration, or is made up of many optical fiber with enlarging function; Described gain modulation receiver (7) is an image intensifier; Described image processing module (10) is personal computer, DSP digital information processing system or flush bonding processor.
CN2010101003263A 2010-01-19 2010-01-19 Light amplification type three-dimensional imaging method and system Expired - Fee Related CN101788667B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2010101003263A CN101788667B (en) 2010-01-19 2010-01-19 Light amplification type three-dimensional imaging method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2010101003263A CN101788667B (en) 2010-01-19 2010-01-19 Light amplification type three-dimensional imaging method and system

Publications (2)

Publication Number Publication Date
CN101788667A true CN101788667A (en) 2010-07-28
CN101788667B CN101788667B (en) 2012-07-04

Family

ID=42531948

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2010101003263A Expired - Fee Related CN101788667B (en) 2010-01-19 2010-01-19 Light amplification type three-dimensional imaging method and system

Country Status (1)

Country Link
CN (1) CN101788667B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101963582A (en) * 2010-09-13 2011-02-02 深圳大学 Three-dimensional fluorescence nano microscope imaging method and system, and image equipment
CN102323595A (en) * 2011-08-08 2012-01-18 中国电子科技集团公司第十一研究所 Brightness modulation scannerless laser three-dimensional imaging method
CN102323594A (en) * 2011-08-08 2012-01-18 中国电子科技集团公司第十一研究所 Brightness-modulating non-scanning laser three-dimensional imaging system
CN103675830A (en) * 2013-12-09 2014-03-26 苏州市峰之火数码科技有限公司 Outdoor distance measurement glasses
WO2014101408A1 (en) * 2012-12-25 2014-07-03 Fu Jian Three-dimensional imaging radar system and method based on a plurality of times of integral
WO2014135127A1 (en) * 2013-03-08 2014-09-12 深圳奥比中光科技有限公司 Orthogonal vision based digital image association device
CN110431440A (en) * 2017-04-20 2019-11-08 华为技术有限公司 Light pulse peak clipper for laser radar
WO2023011091A1 (en) * 2021-08-03 2023-02-09 Congying Sui An imaging method and system based on wise-pixels with valved modulation

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100337122C (en) * 2005-03-25 2007-09-12 浙江大学 Pulse modulation type three-dimensional image-forming method and system containing no scanning device

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101963582A (en) * 2010-09-13 2011-02-02 深圳大学 Three-dimensional fluorescence nano microscope imaging method and system, and image equipment
CN101963582B (en) * 2010-09-13 2012-03-14 深圳大学 Three-dimensional fluorescence nano microscope imaging method and system, and image equipment
CN102323595A (en) * 2011-08-08 2012-01-18 中国电子科技集团公司第十一研究所 Brightness modulation scannerless laser three-dimensional imaging method
CN102323594A (en) * 2011-08-08 2012-01-18 中国电子科技集团公司第十一研究所 Brightness-modulating non-scanning laser three-dimensional imaging system
CN102323594B (en) * 2011-08-08 2013-06-05 中国电子科技集团公司第十一研究所 Brightness-modulating non-scanning laser three-dimensional imaging system
WO2014101408A1 (en) * 2012-12-25 2014-07-03 Fu Jian Three-dimensional imaging radar system and method based on a plurality of times of integral
WO2014135127A1 (en) * 2013-03-08 2014-09-12 深圳奥比中光科技有限公司 Orthogonal vision based digital image association device
CN103675830A (en) * 2013-12-09 2014-03-26 苏州市峰之火数码科技有限公司 Outdoor distance measurement glasses
CN110431440A (en) * 2017-04-20 2019-11-08 华为技术有限公司 Light pulse peak clipper for laser radar
CN110431440B (en) * 2017-04-20 2021-08-20 华为技术有限公司 Optical pulse chopper for laser radar
US11598848B2 (en) 2017-04-20 2023-03-07 Huawei Technologies Co., Ltd. Optical pulse clipper for lidar
WO2023011091A1 (en) * 2021-08-03 2023-02-09 Congying Sui An imaging method and system based on wise-pixels with valved modulation

Also Published As

Publication number Publication date
CN101788667B (en) 2012-07-04

Similar Documents

Publication Publication Date Title
CN101788667B (en) Light amplification type three-dimensional imaging method and system
CN103760567A (en) Passive imaging system with distance measuring function and distance measuring method thereof
Albota et al. Three-dimensional imaging laser radars with Geiger-mode avalanche photodiode arrays
US7667824B1 (en) Range gated shearography systems and related methods
Bastos et al. An overview of LiDAR requirements and techniques for autonomous driving
EP1962107A2 (en) High-speed laser ranging system including a fiber laser
CN101776760A (en) Laser three-dimensional imaging device based on single-photon detector
CN109541627A (en) The adaptive range gate laser radar of dual wavelength
US7148974B1 (en) Method for tracking the location of mobile agents using stand-off detection technique
CN104931974A (en) Light source modulation and demodulation-based ICMOS high-speed 3D imaging laser radar
Hao et al. Development of pulsed‐laser three‐dimensional imaging flash lidar using APD arrays
CN101806899A (en) Striped tube UV laser imaging radar system for carrying out four-dimensional imaging on terrain and imaging method thereof
CN106680829A (en) Linear-array real-time imaging pulse laser radar device
CN106772426B (en) System for realizing remote laser high-sensitivity single photon imaging
Tan et al. Long-range daytime 3D imaging lidar with short acquisition time based on 64× 64 Gm-APD array
Sakib LiDAR with Pulsed Time of Flight
CN201611390U (en) Optical amplification three dimensional imaging system
Gordon et al. Advanced 3D imaging lidar concepts for long range sensing
Katz et al. Design considerations of CMOS Si photomultiplier for long range LIDAR
CN214122466U (en) Collector, distance measurement system and electronic equipment
CN114019521A (en) Area array laser radar multimode data acquisition method
JPS58223079A (en) Measurement collation system for remote object
Xia Flash LiDAR Single Photon Imaging Over 50 KM
Huang et al. Design and performance of a fiber array coupled multi-channel photon counting, 3D imaging, airborne lidar system
Bronzi et al. 3D Sensor for indirect ranging with pulsed laser source

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20120704

Termination date: 20220119

CF01 Termination of patent right due to non-payment of annual fee