CN1161600C - Structure-light 3D double-visual calibrating point generating method nad device - Google Patents
Structure-light 3D double-visual calibrating point generating method nad device Download PDFInfo
- Publication number
- CN1161600C CN1161600C CNB011156554A CN01115655A CN1161600C CN 1161600 C CN1161600 C CN 1161600C CN B011156554 A CNB011156554 A CN B011156554A CN 01115655 A CN01115655 A CN 01115655A CN 1161600 C CN1161600 C CN 1161600C
- Authority
- CN
- China
- Prior art keywords
- transfer table
- dimensional
- calibration point
- coordinate
- sighting device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The present invention relates to a method and a device for generating a three-dimensional double-visual sense index point of structured light for detection. The device is composed of a bidirectional photoelectric aiming device and a three-dimensional mobile station. The mobile station is moved to cause that hole hearts of aiming holes on both sides of the bidirectional photoelectric aiming device are respectively within the collection range of a sample of the bidirectional photoelectric aiming device and are coincided with the light strip center of the structured light of a laser projector on the corresponding side, the coordinates of hole heart objects are obtained by the mobile station, and the coordinates of hole heart images are obtained by a CCD video camera; the CCD video camera is used for obtaining the axial direction of the bidirectional photoelectric aiming device and an axis of the three-dimensional mobile station in a Z direction. In this way, the installation deflection angle of the bidirectional photoelectric aiming device is obtained. The precision of the image coordinates is 0.015 pixel, and the precision of the object coordinates is 0.005mm.
Description
The present invention relates to method and device that a kind of structural light three-dimensional double vision feel calibration point that is used to detect takes place.
Vision-based detection is widely used in the integrality of workpiece, the measurement of surface smoothness; The automatic detection of microelectronic component (IC chip, PC plate, BGA) etc.; The detection of soft, easy crisp parts; The detection of various mould 3D shapes; Robotic vision guiding etc.In many visible detection methods, structured light three-dimensional vision with its wide range, big visual field, measuring speed is fast, optical strip image is easy to extract and characteristics such as degree of precision have obtained using more and more widely in industrial environment.The large-sized object of forming by the vision sensor array (as aviation component etc.) space three-dimensional size multisensor vision detection system the most attractively.This system flexibility is good, is measured as contactlessly, and dynamic response is fast, can satisfy to produce " beat " short requirements in a large number, and whole measuring process is increasingly automated.
For the multisensor vision detection system of being made up of the vision sensor array, its key issue is to set up the relation of the global coordinate system of sensor coordinate system and detection system, i.e. global calibration.And one of gordian technique of global calibration is the generation of calibration point.
The practical calibration point method for generation that is used for many visual structure of structural light three-dimensional parameter calibration mainly is a fiber elongation method at present.This method is that the optical plane of each vision sensor all produces an intersection point with filament at some filaments of measured zone pull-up, and this intersection point is calibration point.The coordinate of each calibration point and between mutual coordinate relation obtain by two electronic theodolites.But this kind method exists following shortcoming:
1. can not realize the extracted with high accuracy of calibration point picture position.The precision that while two electronic theodolites obtain the calibration point article coordinate is lower, generally about 0.05mm.
2. under two vision sensors situation staggered relatively,, therefore caused measuring error because " blind area " problem can not directly produce calibration point, but realize by moving of two electronic theodolites.
3. need repeatedly the conversion of coordinate system, in the transfer process of coordinate system, must lose some measuring accuracy.
Therefore, because the existence of the problems referred to above, many vision-based detection of structural light three-dimensional precision is about 0.5-1mm at present, and the precision of calibration point is lower.
The objective of the invention is, provide a kind of easy to operate, the calibration point method for generation and the device thereof that are used for structural light three-dimensional double vision feel global calibration that precision is high.
Technical solution of the present invention is, with the high-precision three-dimensional transfer table is base, can on space X, three directions of Y, Z, move by controller control, fix a two-way photoelectricity sighting device on its loading installed surface, sighting device is made up of two conical sight holes and two phototriodes and amplifying circuit, gland, pressure pad and screw, phototriode is fixed in the sight hole in the gland, and joins with amplifying circuit on the pressure pad, and gland and pressure pad are lumped together by set screws.The method for generation of its calibration point is:
(1) two-way photoelectricity sighting device is fixed on the three-dimensional transfer table, connects the transfer table power supply, and then connect the laser instrument of right (or left side) side-looking feel detection system and the power supply of ccd video camera, emission line structured light;
(2) the control transfer table moves, make the two-way photoelectricity sighting device right side (or left side) side sight hole be positioned at the starting point place of the sample collection scope of corresponding right (or left side) side-looking feel detection system, and make the right side (or left side) the side sight hole of two-way photoelectricity sighting device be positioned at the striation zone of structured light;
(3) then along transfer table Y to the fine setting transfer table, make the output voltage maximum of the two-way photoelectricity sighting device right side (or a left side) sidelight electricity triode, at this moment, the striation center overlaps with right sight hole center, can obtain its current space three-dimensional article coordinate by transfer table; Close laser instrument then, images acquired is accurately extracted the picture coordinate of the hole heart, so just obtains a data of calibration point to (space article coordinate and picture coordinate);
(4) to moving predetermined step-length, repeating step (3) extracts the picture coordinate of the sight hole hole heart, and reads the three-dimensional article coordinate of the coordinate of three-dimensional transfer table as the sight hole hole heart control transfer table along transfer table Z, and it is right so just to obtain another data of calibration point;
(5) repeating step (4) is until collecting enough data of calibration point at predetermined Z in acquisition range.
(6) to moving predetermined step-length, repeating step (3) extracts the picture coordinate of the sight hole hole heart, and reads the three-dimensional article coordinate of the coordinate of three-dimensional transfer table as the sight hole hole heart control transfer table along transfer table X, and it is right so just to obtain another data of calibration point;
(7) repeating step (6) is until collecting enough data of calibration point at predetermined X in acquisition range.
(8) repeating step (4), (5), (6), (7) are up to collect enough data of calibration point in whole calibrating point acquisition plane scope.
Advantage of the present invention is, adopts a two-way photoelectronic collimating device, cooperates a high accuracy three-dimensional transfer table, and adopts effective two-way photoelectricity sighting device alignment error modification method, obtained to be used for the structural light three-dimensional double vision and to feel the calibration point of global calibration; Device of the present invention is formed the electric sighting device of bi-directional light by two conical sight holes and two phototriodes and amplifying circuit, corresponding gland, pressure pad and screw.Position between two sight holes on this sighting device is closed when tying up to design and has been preestablished, and is guaranteed by high-accuracy machining.Suppose that optical plane is incident in sight hole with α (α is the angle in incident light and sight hole axle center) angle respectively from both sides, and to establish the optical plane cross section be Gaussian distribution, its width approximates the diameter of sight hole.When the output voltage of silicon phototriode in the two-way photoelectricity sighting device is maximum, optical plane center and sight hole center inregister.Adjust optical plane and be incident in sight hole with α=0 as far as possible.Sight hole adopts conical, can reduce being in the light of hole wall, and it is more sharp-pointed to make that aiming edge, hole becomes, thereby can accurately extract the hole center image coordinate of sight hole.Precision is the three-dimensional transfer table of 5 μ m, and the moving range on space X, three directions of Y, Z is 300mm.It is controlled by a controller, can move on three directions in space with minimum step 5 μ m.Because at the calibration point of having gathered the right side vision detection system, when gathering the calibration point of left side vision detection system again, the initial point of transfer table is not reset, and promptly in the calibration point gatherer process, only sets a true origin when initial, set up coordinate system, then all three-dimensional article coordinates that obtain after moving of transfer table all are the article coordinate under this unique coordinate system, and are therefore, easy and simple to handle, the detection error is little, the precision height.Of the present invention being skillfully constructed, easy to operate, the structural design advantages of simple of its implement device, the image coordinate precision of calibration point are 0.15 pixel, the overall article coordinate precision of calibration point is 0.015mm.
Fig. 1 is the prior art synoptic diagram;
Fig. 2 is a generating means synoptic diagram of the present invention;
Fig. 3 is the two-way photoelectricity sighting device of a present invention synoptic diagram;
Fig. 4 is the three-dimensional transfer table synoptic diagram of the present invention;
Fig. 5 installs schematic top plan view for the present invention is desirable;
Fig. 6 is a mounting shift angle schematic top plan view of the present invention;
Fig. 7 is mounting shift angle of the present invention (θ) acquisition methods synoptic diagram.
Fig. 3 is the structural drawing of two-way photoelectricity sighting device.Among the figure, 17 is left circles taper sight hole, is used for aiming at and reception left side structure striation.19 is the left side phototriode, is used for producing photocurrent under illumination.21 is the left side gland, and 23 is the left side pressure pad.25 is the left side screw, and 27 is the left side amplifying circuit, and photocurrent is converted to voltage output; 18 is right circles taper sight hole, is used for aiming at and receiving right side structure striation; 20 is the right side phototriode, is used for producing photocurrent under illumination; 24 is the right side pressure pad, and 22 is the right side gland.26 is the right side screw, and 28 is the right side amplifying circuit, and photocurrent is converted to voltage output.
Fig. 4 is used for cooperating with two-way photoelectricity sighting device, and the three-dimensional transfer table of the space article coordinate of calibration point is provided, and wherein 29 is the X-axis guide rail.30 is the Y-axis guide rail.31 is the Z axis rail.
Fig. 2 is the structured flowchart of the three-dimensional double vision feel of total light calibration point generating means.The definition of three dimensional space coordinate system as shown in FIG..Wherein 9,10 is laser projecting apparatus.11,12 is ccd video camera, realizes obtaining of scene image.13 is two-way photoelectricity sighting device.14 is three-dimensional transfer table.15 is image pick-up card.16 is computing machine, control and data processing.
The method for generation of calibration point is as follows:
(1) two-way photoelectricity sighting device 13 is fixed on the three-dimensional transfer table 14, connects the transfer table power supply, and then connect the laser instrument 10 of right side vision detection system and the power supply of ccd video camera 12, emission line structured light;
(2) control transfer table 14 moves, and makes two-way photoelectricity sighting device 13 right side sight holes 18 be positioned at the starting point place of the sample collection scope of corresponding right side vision detection system, and makes the right side sight hole 18 of two-way photoelectricity sighting device 13 be positioned at the striation zone of structured light;
(3) then along transfer table Y to the fine setting transfer table, make the output voltage maximum of two-way photoelectricity sighting device 13 right side phototriodes 20, at this moment, the striation center overlaps with the center of right sight hole 18, can obtain its current space three-dimensional article coordinate by transfer table.Close laser instrument 10 then, images acquired is accurately extracted the picture coordinate of the hole heart, so just obtains a data of calibration point to (space article coordinate and picture coordinate);
(4) to moving predetermined step-length, repeating step (3) extracts the picture coordinate of the sight hole hole heart, and reads the three-dimensional article coordinate of the coordinate of three-dimensional transfer table as the sight hole hole heart control transfer table along transfer table Z, and it is right so just to obtain another data of calibration point;
(5) repeating step (4) is until collecting enough data of calibration point at predetermined Z in acquisition range;
(6) the three-dimensional transfer table of control along X to moving predetermined step-length, repeating step (4), (5);
(7) repeating step (6), until collect enough data of calibration point in the predetermined acquisition range in right side, the data that obtain are all imported computing machine 16, turn off the laser instrument 10 on right side and the power supply of ccd video camera 12; Whole data of calibration point have so just been obtained corresponding to the right side vision sensor.
(8) open the laser instrument 9 in left side and the power supply of ccd video camera 11, emission line structured light;
(9) control transfer table 14 moves, and makes two-way photoelectricity sighting device 13 be positioned at the starting point place of the sample collection scope of corresponding left side vision detection system, makes the left side sight hole 17 of two-way photoelectricity sighting device 13 be positioned at the striation zone of structured light;
(10) then along the position of transfer table Y to fine setting transfer table 14, make the output voltage maximum of two-way photoelectricity sighting device 13 left side phototriodes 19, at this moment, the striation center overlaps with left side sight hole 17 centers, can obtain its current article coordinate by transfer table.Close laser instrument 9 then, images acquired is accurately extracted the picture coordinate of the hole heart, can obtain a data of calibration point like this to (space article coordinate and picture coordinate);
(11) the control transfer table along transfer table Z to moving predetermined step-length, repeating step (10);
(12) repeating step (11) is until collecting enough data of calibration point at predetermined Z in acquisition range;
(13) the three-dimensional transfer table of control along X to moving predetermined step-length, repeating step (10), (11), (12);
(14) repeat the step (13) suddenly, until in the predetermined acquisition range in left side, collecting enough data of calibration point.Turn off the laser instrument 9 in left side and the power supply of ccd video camera 11; The data that obtain are all imported computing machine 16.
(15) transfer table is resetted, get back to initial initial point place.
So far, whole data of calibration point have been obtained corresponding to left and right sides double vision sense sensor.
When obtaining the calibration point of left side vision detection system, the article coordinate that three-dimensional transfer table provides is actually the three-dimensional article coordinate at sight hole center, right side.Since about two the hole heart of two sight holes be in together in the center symmetry perpendicular of two-way photoelectricity sighting device, and the position dimension between them closes when tying up to design given and guaranteed by the machining of precision, therefore the hole heart three-dimensional article coordinate of left side sight hole need could obtain by following further conversion, also is the unification of world coordinates.
In the ideal case, the axis of two-way photoelectricity sighting device with by the Z of transfer table to the Z of the article coordinate system that shifting axle determined to parallel axes, as shown in Figure 5, wherein 32 is the loading installed surface of three-dimensional transfer table.33 is two-way photoelectricity sighting device.Relative space position relation between the sight hole of then two-way photoelectricity sighting device both sides is as follows:
And under non-ideality, the axis of the two-way photoelectricity sighting device in back is installed is had angle theta to the Z of the article coordinate system that shifting axle determined to axis with Z by transfer table, see Fig. 6, wherein 32 is the loading installed surface of three-dimensional transfer table.33 is two-way photoelectricity sighting device.
Among Fig. 6, P
1, P
2Be respectively two sight hole hole heart points.Relative space position relation between the heart of sight hole hole, then two-way photoelectricity sighting device both sides is as follows:
In formula (1), (2), (X
1, Y
1, Z
1), (X
2, Y
2, Z
2) be respectively when gathering the left side calibration point three-dimensional article coordinate of the right side sight hole hole heart and sight hole hole, the left side heart.H be sight hole hole, the both sides heart transfer table X to spatial altitude poor, L is along the axial width of sight hole.
Two-way photoelectricity sighting device mounting shift angle (θ) acquisition methods as shown in Figure 7, wherein 32 is the loading installed surface of three-dimensional transfer table.33 is two-way photoelectricity sighting device.34 is gauge point.Concrete grammar is as follows:
(1) be positioned over a ccd video camera, its optical axis is perpendicular to the YOZ plane.Gather piece image, can record the axial of two-way photoelectricity sighting device like this;
(2) on two-way photoelectricity sighting device surface, paste a gauge point,, be designated as position A by the position that ccd video camera obtains gauge point;
(3) make three-dimensional transfer table move certain distance, obtain the position of gauge point again by ccd video camera, be designated as position B along the Z axle;
(4) A, B line are three-dimensional transfer table Z-direction, and so far mounting shift angle can obtain.
Because mounting shift angle θ, h and L are known quantity, calculate the three-dimensional article coordinate (X of sight hole hole, the left side heart according to (2) formula
2, Y
2, Z
2), so just realized the unification of world coordinates at an easy rate, and the precision of calibration point is very high.
Claims (3)
1, a kind of structural light three-dimensional double vision feel calibration point method for generation that is used to detect is characterized in that:
(1) two-way photoelectricity sighting device is fixed on the three-dimensional transfer table, connects the transfer table power supply, and then connect right side or the laser instrument of left side vision detection system and the power supply of ccd video camera, emission line structured light;
(2) the control transfer table moves, make two-way photoelectricity sighting device right side or left side sight hole be positioned at the starting point place of the sample collection scope of corresponding right side or left side detection system, and make the right side of two-way photoelectricity sighting device or the striation zone that the left side sight hole is positioned at structured light;
(3) then along Y to the fine setting transfer table, make the output voltage maximum of two-way photoelectricity sighting device right side or left side phototriode, at this moment, the striation center overlaps with right sight hole center, can obtain its current space three-dimensional article coordinate by transfer table; Close laser instrument then, images acquired is accurately extracted the picture coordinate of the hole heart, so just obtains a space article coordinate and picture coordinate data of calibration point is right;
(4) to moving predetermined step-length, repeating step (3) extracts the picture coordinate of the sight hole hole heart, and reads the three-dimensional article coordinate of the coordinate of three-dimensional transfer table as the sight hole hole heart control transfer table along transfer table Z, and it is right so just to obtain another data of calibration point;
(5) repeating step (4) is until collecting enough data of calibration point at predetermined Z in acquisition range.
(6) to moving predetermined step-length, repeating step (3) extracts the picture coordinate of the sight hole hole heart, and reads the three-dimensional article coordinate of the coordinate of three-dimensional transfer table as the sight hole hole heart control transfer table along transfer table X, and it is right so just to obtain another data of calibration point;
(7) repeating step (6) is until collecting enough data of calibration point at predetermined X in acquisition range;
(8) repeating step (4), (5), (6), (7) are up to collect enough data of calibration point in whole calibrating point acquisition plane scope.
2, structural light three-dimensional double vision feel calibration point method for generation according to claim 1 is characterized in that the method for obtaining photoelectronic collimating device mounting shift angle (θ) is:
(1) places a ccd video camera, make its optical axis well-behaved, gather piece image, can record the axial of two-way photoelectricity sighting device so directly in the YOZ plane;
(2) on two-way photoelectricity sighting device surface, paste a gauge point,, be designated as position A by the position that ccd video camera obtains gauge point;
(3) make three-dimensional transfer table move certain distance, obtain the position of gauge point again by ccd video camera, be designated as position B along the Z axle;
(4) A, B line are three-dimensional transfer table Z-direction, and so far mounting shift angle can obtain.
3, a kind of structural light three-dimensional double vision feel calibration point generating means, it is characterized in that, with the high-precision three-dimensional transfer table is base, can on space X, three directions of Y, Z, move by controller control, fix a two-way photoelectricity sighting device on the loading installed surface of its top, sighting device is made up of two conical sight holes and two phototriodes and amplifying circuit, gland, pressure pad and screw, phototriode is fixed in the sight hole in the gland and joins with amplifying circuit on the pressure pad, and gland and pressure pad are lumped together by set screws.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB011156554A CN1161600C (en) | 2001-04-30 | 2001-04-30 | Structure-light 3D double-visual calibrating point generating method nad device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB011156554A CN1161600C (en) | 2001-04-30 | 2001-04-30 | Structure-light 3D double-visual calibrating point generating method nad device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1384351A CN1384351A (en) | 2002-12-11 |
| CN1161600C true CN1161600C (en) | 2004-08-11 |
Family
ID=4662135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB011156554A Expired - Fee Related CN1161600C (en) | 2001-04-30 | 2001-04-30 | Structure-light 3D double-visual calibrating point generating method nad device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1161600C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100451537C (en) * | 2006-11-16 | 2009-01-14 | 天津大学 | Measuring method of large scale space coordinates based on automatic vision guiding and examining apparatus |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1297796C (en) * | 2003-07-02 | 2007-01-31 | 西安交通大学 | Chromatographic scanning three-dimension measuring method and device linear array photoelectric sensor |
| CN100384220C (en) * | 2006-01-17 | 2008-04-23 | 东南大学 | Video camera rating data collecting method and its rating plate |
| CN101308012B (en) * | 2008-05-29 | 2010-06-23 | 上海交通大学 | Double monocular white light three-dimensional measuring systems calibration method |
| CN101493318B (en) * | 2008-09-16 | 2010-10-06 | 北京航空航天大学 | Rudder deflection angle synchronization dynamic measurement system and implementing method thereof |
| CN101630409B (en) * | 2009-08-17 | 2011-07-27 | 北京航空航天大学 | Hand-eye vision calibration method for robot hole boring system |
| CN103175851B (en) * | 2011-12-21 | 2015-01-07 | 北京兆维电子(集团)有限责任公司 | Calibrating tool and calibrating method for multi-camera scanning system |
| CN103344655B (en) * | 2013-06-28 | 2016-12-28 | 云南电力试验研究院（集团）有限公司电力研究院 | A kind of digital X-ray digital detector automatic positioning equipment |
| CN105014675B (en) * | 2014-06-20 | 2016-08-17 | 北京信息科技大学 | A kind of narrow space intelligent mobile robot vision navigation system and method |
| CN104990518A (en) * | 2015-07-13 | 2015-10-21 | 深圳市易尚展示股份有限公司 | FPGA-based three-dimension scanning control device and method |
| CN105136124A (en) * | 2015-07-16 | 2015-12-09 | 延锋汽车饰件系统宁波有限公司 | Automobile ornament vision mistake-proofing system |
| CN106736867B (en) * | 2016-11-17 | 2019-08-20 | 天津大学 | Machine tool chief axis double structure light quick calibrating method |
| CN109883443B (en) * | 2019-02-19 | 2021-06-18 | 北京工业大学 | Line structure optical sensor spatial attitude calibration method |
-
2001
- 2001-04-30 CN CNB011156554A patent/CN1161600C/en not_active Expired - Fee Related
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100451537C (en) * | 2006-11-16 | 2009-01-14 | 天津大学 | Measuring method of large scale space coordinates based on automatic vision guiding and examining apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1384351A (en) | 2002-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1161600C (en) | Structure-light 3D double-visual calibrating point generating method nad device | |
| CN100585615C (en) | Detection system | |
| CN201373735Y (en) | Three-dimensional image coordinate measuring instrument | |
| CN107345789B (en) | PCB hole position detection device and method | |
| CN102135393A (en) | Measurement system for motion parameters of high-speed motion object | |
| CN101852607A (en) | Rotary laser visual linear array space identification and positioning system | |
| CN107726980B (en) | Calibration method of linear laser displacement sensor based on four-axis measuring machine | |
| CN103759639A (en) | Precision positioning platform position detection method based on area array CCD | |
| CN1256414A (en) | Surface three-dimensional appearance testing method and equipment | |
| CN203217498U (en) | Six-degree-of-freedom optomagnetic three-dimensional mouse device | |
| CN1948896A (en) | Dynamic 3D laser scanning testing head | |
| CN1865844A (en) | Portable railless-structured optical three-D scanning measuring system and measuring method therefor | |
| CN111953912A (en) | Method and device for detecting spatial position of high-speed moving light spot | |
| CN100359286C (en) | Method for improving laser measuring accuracy in image processing | |
| US20070045566A1 (en) | Substrate Alignment Using Linear Array Sensor | |
| US5751011A (en) | System for punching holes in a spinnerette | |
| CN103528528A (en) | Compact type precise laser triangular range finder | |
| CN1236277C (en) | Overall calibrating method for multi-vision sensor detecting system | |
| CN1268892C (en) | PSD based three-dimensional measuring method | |
| CN206803969U (en) | A kind of dot laser coplane degree test device | |
| CN106556343A (en) | A kind of wind-power electricity generation equipment hubless feature parameter Fast measurement system and method | |
| CN207215021U (en) | A kind of camera optical axis offset correction device based on binocular stereo vision | |
| CN105928484A (en) | Elevator guide rail automatic measurement system based on binocular vision | |
| CN111028298B (en) | Convergent binocular system for rigid coordinate system space transformation calibration | |
| CN212620593U (en) | Integrated PCB line width and line distance measuring device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C19 | Lapse of patent right due to non-payment of the annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |