CN103110429B - The optical calibration method of ultrasound probe - Google Patents
The optical calibration method of ultrasound probe Download PDFInfo
- Publication number
- CN103110429B CN103110429B CN201210191466.5A CN201210191466A CN103110429B CN 103110429 B CN103110429 B CN 103110429B CN 201210191466 A CN201210191466 A CN 201210191466A CN 103110429 B CN103110429 B CN 103110429B
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- coordinate
- ultrasound probe
- positioning device
- optical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 239000000523 sample Substances 0.000 title claims abstract description 45
- 230000003287 optical effect Effects 0.000 title claims abstract description 38
- 238000002604 ultrasonography Methods 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 28
- 239000011159 matrix material Substances 0.000 claims abstract description 30
- 230000009466 transformation Effects 0.000 claims abstract description 16
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000013507 mapping Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Ultra Sonic Daignosis Equipment (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a kind of scaling method of ultrasound probe, there are following steps: 1. paste feature identification point on ultrasonic probe, 2. fixing stereo calibration template and optical positioning device, 3. hand-held ultrasound scanning probe stereo calibration template, optical positioning device is utilized to obtain the spatial attitude of ultrasonic probe while each collection ultrasonoscopy, obtain initial data, then utilize method of least square to demarcate ultrasonic probe.Compared to traditional electromagnetism scaling method, tool has the following advantages: calibration result is more accurate; In calculating process, only have a unknown number (unknown transformation matrix), reduce the complexity of calibrated and calculated; Can be applied on the ultrasonic probe of various model, simple equipments, applied widely; Adopt optical positioning device to obtain ultrasonic probe attitude, without the need to considering the impact of power line, what make to demarcate is wider; The problem of electromagnetic compatibility is there is not in optical positioning device in surgical navigational environment.
Description
Technical field
The present invention relates to a kind of optical calibration method, particularly relate to a kind of optical calibration method of ultrasound probe.
Background technology
The existing Manual three-dimensional ultrasonoscopy scaling method based on magnetic locator: fix an electromagnetic receiver on ultrasonic probe, electromagnetic positioners is utilized to obtain the spatial information of this receptor relative to emitter, set up ultrasound image coordinates system I, receptor coordinate system R, transmitter coordinate system T, template coordinate system C tetra-coordinate systems, calibration formula be P (C)=
ct
t.
tt
r.
rt
ip (I), wherein
tt
rrepresent the transformation matrix of electromagnetic receiver to emitter, the ultrasound image coordinates P (I) of known spatial point, corresponding template coordinate P (C) and at every turn measurement
tt
r, utilize the spatial mappings relation that method of least square obtains between receptor and ultrasonoscopy
ct
twith
rt
i.
There is following problem in the Manual three-dimensional ultrasonoscopy scaling method based on magnetic locator:
(1) the N shape calibrating template designed is the calibrating template of two dimension in space, and the calibrating template coordinate obtained lacks the spatial information of the third dimension;
(2) calibration formula P (C)=
ct
t*
tt
r*
rt
i* the unknown quantity in P (I) has
ct
twith
rt
itwo, calculate comparatively complicated, need to gather several uncalibrated images;
(3) acquiescence receptor coordinate system replaces ultrasonic probe coordinate system, may there is mapping fault;
(4) need Motionless electromagnetic receptor on ultrasonic probe, active receiving limits, and needs for the ultrasonic probe of different model the electromagnetic positioners being equipped with different size, and make equipment complicated, adaptability is not high;
(5) there is electromagnetic compatibility problem in electromagnetic location equipment in surgical navigational application.
Summary of the invention
The present invention is directed to the proposition of above problem, and a kind of ultrasound probe scaling method of development, there are following steps:
S1. two pieces of identical cameras that optical axis is parallel to each other are chosen, as optical positioning device, using the photocentre of side camera as coordinate origin, the photocentre connecting line of two pieces of cameras is X-axis, using the optical axis of this camera as Z axis, set up rectangular coordinate system in space, as optical positioning device coordinate system C;
S2. choose and fix two identical rectangular frames, make two faces relative in 2 described virtual cuboids of rectangular frame Special composition, use each end points that many are demarcated straight line and connect described two rectangular frames, as stereo calibration template, choose a summit in described two rectangular frames as initial point, set up rectangular coordinate system in space, as stereo calibration template coordinate system M;
S3. choose the ultrasound probe of a 2D, this ultrasound probe is pasted at least three characteristic points, sets up rectangular coordinate system in space, as ultrasound probe coordinate system T;
S4. ultrasound probe scanning calibrating template is used, hyperacoustic plane of scanning motion is made to demarcate each demarcation straight line of straight line through described many, record each plane of scanning motion and the spatial point of demarcating straight line intersection, calculate the coordinate of each spatial point in described stereo calibration template coordinate system M;
S5. set ultrasound image coordinates system I, utilize the coordinate of each spatial point in ultrasonography described in the acquisition of ultrasonoscopy treatment technology, be denoted as P (I)=(u
i, v
i, 0,1)
t;
S6. by described optical positioning device coordinate system C, stereo calibration template coordinate system M, ultrasound probe coordinate system T and ultrasound image coordinates system I, formula is obtained:
P(M)＝
MT
C·
CT
T(i)·
TT
I·P(I)，
In formula: P (M) is the coordinate of spatial point in stereo calibration template coordinate system M; P (I) is the image coordinate of corresponding point in ultrasonoscopy;
Dimension is the matrix of 4 × 4
mt
cfor the transformation matrix of coordinate system C to coordinate system M, this matrix is fixed value;
Dimension is the matrix of 4 × 4
ct
ti () is the transformation matrix of coordinate system T to coordinate system C, each uncalibrated image gathers a corresponding matrix
ct
t(i);
Dimension is the matrix of 4 × 4
tt
ifor required transformation matrix;
If object function f (i)=| P
m(i)-
mt
c.
ct
t(i)
tt
ip
i(i) |
2, when making f (i)=0
tt
ibe required by demarcation, gather several uncalibrated images, after stacking data, use the method optimization of method of least square to obtain
tt
i, complete the demarcation of ultrasound probe.
The photocentre distance of described two pieces of cameras and parallax range scalable, optical axis included angle scalable.The number of described connecting line is at least 7.
In described step S3, before scanning, described calibrating template is immersed in the water of 50 DEG C.In described step S6, gather at least 2 width images.
Owing to have employed technique scheme, the optical calibration method of ultrasound probe provided by the invention, tool has the following advantages:
1. design a kind of stereo calibration template, by calibrating template coordinate by two-dimensional expansion to three-dimensional, the constraints in calibrated and calculated process can be strengthened like this, make calibration result more accurate;
2. calibration formula P (M)=
mt
c.
ct
t(i)
tt
iunknown quantity in P (I) only has
tt
i, reduce the complexity of calibrated and calculated;
3. directly utilize ultrasonic probe coordinate system and other coordinate systems to convert, explicit physical meaning, and three the signature identification point strong adaptabilities pasted on probe, can be applied on the ultrasonic probe of various model, simple equipments, applied widely;
4. adopt optical positioning device to obtain ultrasonic probe attitude, without the need to considering the impact of power line, what make to demarcate is wider;
5. there is not the problem of electromagnetic compatibility in optical positioning device in surgical navigational environment.
Accompanying drawing explanation
In order to the technical scheme of clearer explanation embodiments of the invention or prior art, introduce doing one to the accompanying drawing used required in embodiment or description of the prior art simply below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the transformation relation schematic diagram of space coordinates of the present invention;
Fig. 2 is stereo calibration template schematic diagram of the present invention;
Fig. 3 is signature identification point on ultrasonic probe of the present invention and Coordinate Setting schematic diagram thereof;
Fig. 4 is stereo calibration template Coordinate Setting schematic diagram of the present invention;
Fig. 5 is the Coordinate calculation method schematic diagram of spatial point E of the present invention in stereo calibration template coordinate system.
Detailed description of the invention
For making the object of embodiments of the invention, technical scheme and advantage clearly, below in conjunction with the accompanying drawing in the embodiment of the present invention, clear complete description is carried out to the technical scheme in the embodiment of the present invention, obviously, described embodiment is a part of embodiment of the present invention, instead of whole embodiment.Based on the embodiment in the present invention, those of ordinary skill in the art, not making other embodiments all obtained under creative work prerequisite, belong to protection scope of the present invention.
Embodiment 1:
As Figure 1-5: a kind of ultrasound probe scaling method:
One. the setting of the coordinate system C of optical positioning device: two pieces that choose that optical axis is parallel to each other identical cameras, as optical positioning device.As shown in Figure 1, two pieces of identical cameras, are equipped with the camera lens of same model, preferably, are fixed on horizontally disposed fixed mount.
Further, in order to meet different calibration ranges, two pieces of described cameras are flexibly connected with described fixed mount, and namely the photocentre distance of two pieces of cameras and parallax range are set to adjustable, the optical axis included angle also scalable of two pieces of cameras, can meet the demarcation demand of different azimuth simultaneously.
After fixing optical instrument, common, select with left camera photocentre for axle center, the baseline of two pieces of cameras is X-axis line, using the optical axis of this left camera as Z axis, sets up rectangular coordinate system, as optical positioning device coordinate system C.
Two. stereo calibration template and Coordinate Setting thereof: as shown in Figure 2: select two on all four rectangular frames or rectangular panel, two on all four rectangular frames of size are selected in the present embodiment, to a framework be connected in being vertically oppositely arranged, the wherein orthographic projection of rectangular frame A on another rectangular frame B is overlapped completely with rectangular frame B, also can be expressed as two described rectangular frames in optical positioning device coordinate system, form two faces relative in a space virtual cuboid.Demarcate straight line, connect the summit of described two rectangular panel for selected many.Due to the needs of computing, at least need 7 to demarcate straight line, adopt the mode that broken line N shape connects, all summits of two rectangular frames described in connection.Concrete connected mode, as shown in Figure 2: in six faces of Virtual space cuboid, except the face at described two rectangular frame places, in remaining 4 rectangular surfaces, have three faces at least, there are three summits demarcated straight line and connect face, each place, and three straight lines are N font.Using O point as zero, set up rectangular coordinate system in space, as stereo calibration template coordinate system M.
Three. the signature identification point on ultrasonic probe and Coordinate Setting thereof, as Fig. 3 shows, choose the ultrasound probe of a 2D, this ultrasound probe is pasted at least three characteristic points, preferably, characteristic point can be Visible Light Characteristics point and infrared LED characteristic point etc., sets up rectangular coordinate system in space, as ultrasound probe coordinate system T.
Four. each ultrasonic depth finder, all has the coordinate of oneself at its scanning plane, be set as ultrasound image coordinates system I in the present invention.
Five. use ultrasound probe scanning calibrating template, when work, described calibrating template is put into the water of water temperature about 50 DEG C, hand-held 2D ultrasonic probe scans this template, and the plane of scanning motion and calibration line intersect at D, E, F, G, H, M, N seven spatial point, as Fig. 4 shows.
Utilize the coordinate of each spatial point in ultrasonography coordinate system I described in the acquisition of ultrasonoscopy treatment technology, be denoted as P (I)=(u
i, v
i, 0,1)
t.
D, E, F, G, H, M, N of obtaining seven spatial point, the coordinate of each point in volume template coordinate system M is set to P (M)=(x
i, y
i, z
i, 1)
t, for coplanar 3 of D, E, F, similar to triangle EFC, there is the coordinate of E point in stereo calibration template to be (x according to the principle of similar triangles
e, y
e, z
e, 1)
t,
Wherein
The geometry information of known stereo calibration template OA, OB, OC, DE, DF can measure and obtain in ultrasonoscopy, so just can obtain the coordinate of an E in stereo calibration template coordinate system, every width uncalibrated image can obtain the coordinate as E, G, M 3, as shown in Figure 4.
If the corresponding point image coordinate of spatial point in ultrasonoscopy is P (I)=(u
i, v
i, 0,1)
t, this image coordinate can utilize the technical limit spacing of ultrasonoscopy process.
Because stereo calibration template is static relative to optical positioning device in calibration process, so transformation matrix
mt
cconstant, only need record the transformation matrix that the coordinate of O, A, C tri-spatial point in optical positioning device coordinate system can be asked
mt
c;
While the ultrasonic uncalibrated image of each collection, utilize optical positioning device to obtain the coordinate of three characteristic points on ultrasonic probe, can transformation matrix be obtained
ct
t(i), the corresponding transformation matrix of every width uncalibrated image.
By above-mentioned condition obtain calibration formula P (M)=
mt
c.
ct
t(i)
tt
ip (I), in formula, P (M) is the coordinate of spatial point in stereo calibration template coordinate system; P (I) is the image coordinate of corresponding point in ultrasonoscopy; Dimension is the matrix of 4 × 4
mt
cfor the transformation matrix of coordinate system C to coordinate system M, this matrix is fixed value; Dimension is the matrix of 4 × 4
ct
ti () is the transformation matrix of coordinate system T to coordinate system C, each uncalibrated image gathers a corresponding matrix
ct
t(i); Dimension is the matrix of 4 × 4
tt
ifor required transformation matrix.
If object function f (i)=| P
m(i)-
mt
c.
ct
t(i)
tt
ip
i(i) |
2, when making f (i)=0
tt
ibe required by demarcation, gather several uncalibrated images (at least 2 width image), after stacking data, engineering can use the method optimization of method of least square to obtain
tt
i.
The above; be only the present invention's preferably detailed description of the invention; but protection scope of the present invention is not limited thereto; anyly be familiar with those skilled in the art in the technical scope that the present invention discloses; be equal to according to technical scheme of the present invention and inventive concept thereof and replace or change, all should be encompassed within protection scope of the present invention.
Claims (4)
1. an optical calibration method for ultrasound probe, is characterized in that having following steps:
S1. two pieces of identical cameras that optical axis is parallel to each other are chosen, as optical positioning device, using the photocentre of side camera as coordinate origin, the photocentre connecting line of two pieces of cameras is X-axis, using the optical axis of this camera as Z axis, set up rectangular coordinate system in space, as optical positioning device coordinate system C;
S2. choose and fix two identical rectangular frames, make two faces relative in 2 described virtual cuboids of rectangular frame Special composition, use each end points that many are demarcated straight line and connect described two rectangular frames, as stereo calibration template, choose a summit in described two rectangular frames as initial point, set up rectangular coordinate system in space, as stereo calibration template coordinate system M;
S3. choose the ultrasound probe of a 2D, this ultrasound probe is pasted at least three characteristic points, sets up rectangular coordinate system in space, as ultrasound probe coordinate system T;
S4. ultrasound probe scanning calibrating template is used, hyperacoustic plane of scanning motion is made to demarcate each demarcation straight line of straight line through described many, record each plane of scanning motion and the spatial point of demarcating straight line intersection, calculate the coordinate of each spatial point in described stereo calibration template coordinate system M;
S5. set ultrasound image coordinates system I, utilize the coordinate of each spatial point in ultrasonography described in the acquisition of ultrasonoscopy treatment technology, be denoted as P (I)=(u
i, v
i, 0,1)
t;
S6. by described optical positioning device coordinate system C, stereo calibration template coordinate system M, ultrasound probe coordinate system T and ultrasound image coordinates system I, formula is obtained:
P(M)＝
MT
C·
CT
T(i)·
TT
I·P(I)，
In formula: P (M) is the coordinate of spatial point in stereo calibration template coordinate system M; P (I) is the image coordinate of corresponding point in ultrasonoscopy;
Dimension is the matrix of 4 × 4
mt
cfor the transformation matrix of coordinate system C to coordinate system M, this matrix is fixed value;
Dimension is the matrix of 4 × 4
ct
ti () is the transformation matrix of coordinate system T to coordinate system C, each uncalibrated image gathers a corresponding matrix
ct
t(i);
Dimension is the matrix of 4 × 4
tt
ifor required transformation matrix;
If object function f (i)=| P
m(i)-
mt
c.
ct
t(i)
tt
ip
i(i) |
2, when making f (i)=0
tt
ibe required by demarcation, gather several uncalibrated images, after stacking data, use the method optimization of method of least square to obtain
tt
i, complete the demarcation of ultrasound probe; P
mi () represents the coordinate of target at coordinate system M; P
ii () represents the coordinate of target at coordinate system I;
The number of described connecting line is at least 7.
2. the optical calibration method of a kind of ultrasound probe according to claim 1, is further characterized in that: the photocentre distance of described two pieces of cameras and parallax range scalable, optical axis included angle scalable.
3. the optical calibration method of a kind of ultrasound probe according to claim 1, is further characterized in that: in described step S3, before scanning, is immersed by described calibrating template in the water of 50 DEG C.
4. the optical calibration method of a kind of ultrasound probe according to claim 1, is further characterized in that: in described step S6, gathers at least 2 width images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210191466.5A CN103110429B (en) | 2012-06-11 | 2012-06-11 | The optical calibration method of ultrasound probe |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210191466.5A CN103110429B (en) | 2012-06-11 | 2012-06-11 | The optical calibration method of ultrasound probe |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103110429A CN103110429A (en) | 2013-05-22 |
| CN103110429B true CN103110429B (en) | 2015-12-16 |
Family
ID=48408829
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210191466.5A Expired - Fee Related CN103110429B (en) | 2012-06-11 | 2012-06-11 | The optical calibration method of ultrasound probe |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103110429B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103584885B (en) * | 2013-11-20 | 2015-02-25 | 哈尔滨工程大学 | Free arm ultrasound calibration method based on positioning and navigation puncture needle |
| CN103983933B (en) * | 2014-05-08 | 2017-09-19 | 工业和信息化部电子第五研究所 | Plate level RF current probe frequency demarcating method and system and device |
| WO2016046588A1 (en) * | 2014-09-24 | 2016-03-31 | B-K Medical Aps | Transducer orientation marker |
| CN106560163B (en) * | 2015-09-30 | 2019-11-29 | 合肥美亚光电技术股份有限公司 | The method for registering of operation guiding system and operation guiding system |
| CN105534596B (en) * | 2015-12-31 | 2018-03-09 | 精微视达医疗科技（武汉）有限公司 | A kind of method and target wire body mould for locus calibration between binocular optical probe and Ultrasonic-B probe |
| CN105769244B (en) * | 2016-03-22 | 2020-04-03 | 上海交通大学 | Calibration device for ultrasonic probe calibration |
| CN106344153B (en) * | 2016-08-23 | 2019-04-02 | 深圳先进技术研究院 | A kind of flexible puncture needle needle point autotracker and method |
| CN106725595B (en) * | 2016-12-05 | 2023-04-21 | 华南理工大学 | Electromagnetic positioning and B-ultrasonic integrated probe calibration device and calibration method thereof |
| CN107462392A (en) * | 2017-09-01 | 2017-12-12 | 南京航空航天大学 | High Precision Automatic scaling method based on virtual template |
| CN108830905A (en) * | 2018-05-22 | 2018-11-16 | 苏州敏行医学信息技术有限公司 | The binocular calibration localization method and virtual emulation of simulating medical instrument cure teaching system |
| CN110554095B (en) * | 2019-08-16 | 2022-07-26 | 上海工程技术大学 | Three-dimensional ultrasonic probe calibration device and method |
| CN113048878B (en) * | 2019-12-27 | 2023-08-29 | 苏州因确匹电子科技有限公司 | Optical positioning system and method and multi-view three-dimensional reconstruction system and method |
| CN112472294B (en) * | 2020-12-15 | 2023-01-20 | 山东威高医疗科技有限公司 | Method for acquiring spatial positions of different ultrasonic equipment probes in electromagnetic navigation system |
| CN113940699A (en) * | 2021-10-11 | 2022-01-18 | 电子科技大学 | Ultrasonic probe self-positioning device and self-positioning method thereof |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5921992A (en) * | 1997-04-11 | 1999-07-13 | Radionics, Inc. | Method and system for frameless tool calibration |
| CN101630409B (en) * | 2009-08-17 | 2011-07-27 | 北京航空航天大学 | Hand-eye vision calibration method for robot hole boring system |
| CN102207371B (en) * | 2011-03-16 | 2012-07-04 | 西南石油大学 | Three-dimensional point coordinate measuring method and measuring apparatus thereof |
| CN102319117B (en) * | 2011-06-16 | 2013-04-03 | 上海交通大学医学院附属瑞金医院 | Large vessel internal intervention implant system based on magnetic navigation fusion real-time ultrasonic information |
-
2012
- 2012-06-11 CN CN201210191466.5A patent/CN103110429B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN103110429A (en) | 2013-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103110429B (en) | The optical calibration method of ultrasound probe | |
| CN108663681B (en) | Mobile robot navigation method based on binocular camera and two-dimensional laser radar | |
| CN103292710B (en) | A kind of distance measurement method applying binocular vision vision range finding principle | |
| CN107976669B (en) | Device for determining external parameters between camera and laser radar | |
| CN102607457B (en) | Measuring device and measuring method for large three-dimensional morphology based on inertial navigation technology | |
| Pandey et al. | Extrinsic calibration of a 3d laser scanner and an omnidirectional camera | |
| CN101577002B (en) | Calibration method of fish-eye lens imaging system applied to target detection | |
| CN109446892B (en) | Human eye attention positioning method and system based on deep neural network | |
| CN101581569A (en) | Calibrating method of structural parameters of binocular visual sensing system | |
| CN102519434B (en) | Test verification method for measuring precision of stereoscopic vision three-dimensional recovery data | |
| CN103115615B (en) | Fully-automatic calibration method for hand-eye robot based on exponential product model | |
| CN103759669A (en) | Monocular vision measuring method for large parts | |
| CN102506711B (en) | Line laser vision three-dimensional rotate scanning method | |
| CN109141226A (en) | The spatial point coordinate measuring method of one camera multi-angle | |
| CN102022989A (en) | Robot calibration method based on exponent product model | |
| CN101425185B (en) | Method for demarcating small-scale vision measuring video camera based on composite planar target drone | |
| CN102072706A (en) | Multi-camera positioning and tracking method and system | |
| CN104517291A (en) | Pose measuring method based on coaxial circle characteristics of target | |
| CN102081798B (en) | Epipolar rectification method for fish-eye stereo camera pair | |
| CN109242918A (en) | A kind of helicopter-mounted binocular stereo vision scaling method | |
| CN104634248A (en) | Revolving shaft calibration method under binocular vision | |
| CN103606147A (en) | Coordinate system transformation and calibration method of multiple measurement cameras with different fields of view | |
| CN106403838A (en) | Field calibration method for hand-held line-structured light optical 3D scanner | |
| Liu et al. | A high-accuracy pose measurement system for robotic automated assembly in large-scale space | |
| CN104807405A (en) | Three-dimensional coordinate measurement method based on light ray angle calibration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20151216 Termination date: 20200611 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |