CN106091981A - Region projection method for measuring optical three-dimensional contour for HDR object - Google Patents
Region projection method for measuring optical three-dimensional contour for HDR object Download PDFInfo
- Publication number
- CN106091981A CN106091981A CN201610368146.0A CN201610368146A CN106091981A CN 106091981 A CN106091981 A CN 106091981A CN 201610368146 A CN201610368146 A CN 201610368146A CN 106091981 A CN106091981 A CN 106091981A
- Authority
- CN
- China
- Prior art keywords
- projection
- phase
- region
- bar graph
- phase shift
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/254—Projection of a pattern, viewing through a pattern, e.g. moiré
Abstract
The invention discloses the region projection method for measuring optical three-dimensional contour for HDR object, first to measured object projection binary-coding bar graph and phase shift figure thereof, obtain projection template, then the respective pixel of this projection template with multifrequency sine striped phase shift figure to be projected is multiplied, obtains region projection figure;Then, region projection figure is projected to measured object again, employing phase shift algorithm calculating phase diagram, and give the reflectance of space invariance, obtain the bar graph strengthened;Finally, utilize multifrequency striped that phase diagram solution is wrapped up, and repair the phase place in the region that is blocked line by line, obtain after giving reflectance repairing bar graph, utilize this bar graph to calculate the three-D profile data of measured object.Present invention uses region projection figure, on the premise of not increasing hardware system complexity, eliminate the impact overflowed HDR method, effectively solve the problem that when structural light three-dimensional profile measurement measures HDR object, precision is low.
Description
Technical field
The invention belongs to photoelectric measurement field, be particularly used for the region projection optical 3-dimensional wheel of HDR object
Wide measuring method.
Background technology
Optical three-dimensional contour is measured technology and is always the focus of three-dimensional sensing and metering field research, is widely used in industry
The aspects such as detection, reverse engineering, human body three-dimensional modeling, historical relic's protection.Owing to it has, noncontact, non-demolition, speed is fast, precision
High so that it is become ideal profile topography measurement means.
Structural light three-dimensional contour measuring method, according to the light field projected on object and optical projection system, acquisition system with
The geometrical relationship of measured object benchmark demodulates the three-dimensional information of measured object field.For being suitable for the measurement of HDR object, it is necessary to
Solve to overflow and the impact of reflectivity spatial change.General solution is divided into hardware and software two class.Hardware approach typically exists
On the basis of structured-light system, add and regulate the device of light intensity according to measured object surface reflectivity, it is possible to reduce because reflectance is empty
Between change the impact brought, and prevent from overflowing, but such method system be complicated, and realize difficulty.Software improves not to be increased firmly
On the basis of part, by regulation time of exposure, projection brightness or the two regulation simultaneously, then by the image co-registration under different condition
For piece image, to eliminate the impact of reflectance change, but such method does not accounts for the impact that camera overflows, and causes measurement more
The precision of HDR object is relatively low.
In sum, need to choose a kind of suitably method, in the case of not increasing hardware complexity, can eliminate excessive
Go out and the impact of reflectance, and obtain the three-D profile of HDR object.Therefore, the most necessary research one can be coordinated
Or solve the measuring three-dimensional profile method of the problems referred to above.
Summary of the invention
It is an object of the invention to overcome above-mentioned deficiency, it is provided that a kind of region projection optics for HDR object
Measuring three-dimensional profile method, it can solve HDR object and measure on the premise of not increasing hardware system complexity
The problem that precision is low.
In order to achieve the above object, the present invention comprises the following steps:
Step one, to measured object projection binary-coding bar graph and phase shift figure thereof, it is thus achieved that projection template;
Step 2, is multiplied the respective pixel of this projection template with multifrequency sine striped phase shift figure to be projected, obtains district
Territory projection;
Step 3, projects to measured object again by region projection figure, uses phase shift algorithm to calculate phase diagram, and gives space
Constant reflectance, obtains the bar graph strengthened;
Step 4, utilizes multifrequency striped to wrap up phase diagram solution, and repairs the phase place in the region that is blocked line by line, gives reflection
Obtain after rate repairing bar graph, utilize this bar graph to calculate the three-D profile data of measured object.
In described step one, it is thus achieved that projection template comprises the following steps:
The first step, to measured object projection binary-coding bar graph and phase shift figure thereof, calculates peak frequency striped phase shift figure
Mean flow rate figure, and its thresholding is processed, determine overflow area;
Second step, each pixel intensity derivative to phase shift step number in calculation code figure, according to maximum position in derivative
Each code pattern is carried out binaryzation, and the binary code value in calculation overflow district, determines the projected area that overflow area is corresponding, it is thus achieved that projection
Template.
In described second step, according to each pixel intensity derivative to phase shift step number, detect whether to have what spilling caused
Pseudo-maximum, if there is pseudo-maximum, after being eliminated, carries out binaryzation further according to remaining maximum position to this pixel,
So can eliminate the impact overflowed binaryzation;If there is not pseudo-maximum, directly according to maximum position binaryzation;According to
The binary code value in binaryzation result calculation overflow district, determines the projected area of correspondence, it is thus achieved that projection template;Required projection in template
The coefficient in district is 0, and other regions are 1.
In described step 4, repair phase place process first according to threshold process or manually iris wipe and determine occlusion area, use Fu
In the phase place of leaf series approximation occlusion area the i-th row outside continuation certain length, repair occlusion area the i-th row according to Coefficients of Approximation
Phase place, and then i+1 line phase is repaired, by that analogy, until the phase place reparation of all occlusion areas completes.
Repair specifically comprising the following steps that of phase place process
The first step, first according to threshold process or manually iris wipe and determine occlusion area;
Second step, determines the high-low limit point Y in this regionuAnd Yl, and set blocked area often go outside continuation length L and
The exponent number of Fourier space;
3rd step, repairs y=YuLine phase: determine y=Yu, the right boundary x of the occlusion area of rowlAnd xr, take out
[xl-L,xr+ L] phase place, and approach by Fourier space, repair occlusion area y=Y according to the Coefficients of Approximation obtaineduRow
Phase place;
4th step, repairs y=y-1 line phase, repeats above-mentioned repair process, until by all for occlusion area phase place reparations
Complete.
In described step 4, by the bar graph repaired is carried out center extraction, calculate the three-D profile number of measured object
According to.
Compared with prior art, the present invention uses binary-coding bar graph and phase shift figure thereof, it is to avoid because of saturated and reflection
The problem of the binarization error that rate affects and causes, thus obtain reliable decoded result, and then accurately obtain required projective module
Plate;Use region projection figure, it is to avoid overflow and saturated impact, it is thus achieved that reliably after phase diagram, give space homogeneous anti-
Penetrate rate, obtain high-quality bar graph, it is ensured that follow-up certainty of measurement;Use again image repair technology, the bar to blocked area
Stricture of vagina is recovered, it is hereby achieved that complete bar graph, it is ensured that the acquisition of object comprehensive data, effectively solve structure light three
The problem that when dimension profile measurement measures HDR object, precision is low.
Accompanying drawing explanation
Fig. 1 is measurement apparatus schematic diagram of the present invention;
Fig. 2 is binary-coding bar graph and the phase shift figure thereof of the present invention;
Fig. 3 is that the present invention projects template acquisition flow chart;
Fig. 4 is that the present invention projects template and the schematic diagram of region projection figure;Wherein (a) is projection template, and (b) is original throwing
Shadow figure, (c) is region projection figure;
Fig. 5 is that occlusion area of the present invention repairs flow chart.
Detailed description of the invention
The present invention will be further described below in conjunction with the accompanying drawings.
1) projection template obtains
Measurement apparatus schematic diagram such as Fig. 1, projection light source projects binary-coding bar graph and phase shift figure thereof (such as figure to measured object
2), calculate the mean flow rate figure of peak frequency striped phase shift figure after shooting, and its thresholding is processed, determine overflow area;Then,
Calculation code figure overflows the mean flow rate of peripheral region, and asks for this mean flow rate derivative to phase shift step number, at this time can
Two kinds of results occur:
The most above-mentioned derivative does not exist pseudo-maximum, but presents randomness, at this moment without to Overflow handling;
When b. there is two obvious pseudo-maximum, because it forms interference to binaryzation, should give elimination
According to above two result, the binaryzation of code pattern should process respectively:
When result in a. occurs, it is not necessary to eliminating pseudo-maximum, in direct calculation code figure, the brightness of pixel is to moving one's steps mutually
The derivative of number, determines maximum position therein, to this pixel binaryzation;When result in b. occurs, calculate the brightness of pixel
Derivative to phase shift step number, after eliminating pseudo-maximum, according to remaining maximum position, to this pixel binaryzation.
Afterwards, according to the code pattern of binaryzation, the binary code value in calculation overflow district, the projected area of correspondence is determined, it is thus achieved that
Projection template: the coefficient of above-mentioned zone is 0, and other regions are 1, projection template obtains flow chart such as Fig. 3;
2) respective pixel of this projection template with multifrequency sine striped phase shift figure to be projected is multiplied, obtains region projection
Figure, projection template and the schematic diagram of region projection figure, such as Fig. 4;
3) then, region projection figure is projected to measured object again, during shooting, chooses bigger time of exposure, dark to ensure
The striped image quality in district, the phase diagram of employing phase shift algorithm calculating shooting striped, and give the reflectance of space invariance, obtain
The bar graph strengthened, can eliminate the impact of reflectivity spatial change;
4) subsequently, utilize multifrequency striped that phase diagram solution is wrapped up, and the phase diagram after solving parcel is repaired line by line and is blocked
The phase place in region, obtains after giving reflectance repairing bar graph;
5) phase place process is repaired, such as Fig. 5:
A. occlusion area determines: first according to threshold process or manually iris wipe and determine occlusion area;
B. parameter sets: (repairs similar by column) as a example by repairing line by line, determines the high-low limit point Y in this regionuAnd Yl,
And set blocked area and often go length L of outside continuation and the exponent number of Fourier space;
C. y=Y is repaireduLine phase: determine y=Yu, the right boundary x of the occlusion area of rowlAnd xr, take out [xl-L,xr
+ L] phase place, and approach by Fourier space, repair occlusion area y=Y according to the Coefficients of Approximation obtaineduThe phase place of row;
D. repair y=y-1 line phase, repeat above-mentioned repair process, until all for occlusion area phase place reparations are completed;
E. to repair phase place give step 3) in reflectance, obtain repair bar graph;
6) last, the bar graph repaired is carried out center extraction, calculates the three-D profile data of measured object.
Claims (6)
1. for the region projection method for measuring optical three-dimensional contour of HDR object, it is characterised in that include following step
Rapid:
Step one, to measured object projection binary-coding bar graph and phase shift figure thereof, it is thus achieved that projection template;
Step 2, is multiplied the respective pixel of this projection template with multifrequency sine striped phase shift figure to be projected, obtains region and throws
Shadow figure;
Step 3, projects to measured object again by region projection figure, uses phase shift algorithm to calculate phase diagram, and gives space invariance
Reflectance, obtain strengthen bar graph;
Step 4, utilizes multifrequency striped to wrap up phase diagram solution, and repairs the phase place in the region that is blocked line by line, after giving reflectance
Obtain repairing bar graph, utilize this bar graph to calculate the three-D profile data of measured object.
Region projection method for measuring optical three-dimensional contour for HDR object the most according to claim 1, its
It is characterised by, in described step one, it is thus achieved that projection template comprises the following steps:
The first step, to measured object projection binary-coding bar graph and phase shift figure thereof, calculates the average of peak frequency striped phase shift figure
Luminance graph, and its thresholding is processed, determine overflow area;
Second step, each pixel intensity derivative to phase shift step number in calculation code figure, according to maximum position in derivative to respectively
Code pattern carries out binaryzation, and the binary code value in calculation overflow district, determines the projected area that overflow area is corresponding, it is thus achieved that projective module
Plate.
Region projection method for measuring optical three-dimensional contour for HDR object the most according to claim 2, its
It is characterised by, in described second step, according to each pixel intensity derivative to phase shift step number, detects whether to have what spilling caused
Pseudo-maximum, if there is pseudo-maximum, after being eliminated, carries out binaryzation further according to remaining maximum position to this pixel,
So can eliminate the impact overflowed binaryzation;If there is not pseudo-maximum, directly according to maximum position binaryzation;According to
The binary code value in binaryzation result calculation overflow district, determines the projected area of correspondence, it is thus achieved that projection template;Required projected area be
Number is 0, and other regions are 1.
Region projection method for measuring optical three-dimensional contour for HDR object the most according to claim 1, its
It is characterised by, in described step 4, repairs phase place process first according to threshold process or manually iris wipe and determine occlusion area, use Fu
In the phase place of leaf series approximation occlusion area the i-th row outside continuation certain length, repair occlusion area the i-th row according to Coefficients of Approximation
Phase place, and then i+1 line phase is repaired, by that analogy, until the phase place reparation of all occlusion areas completes.
Region projection method for measuring optical three-dimensional contour for HDR object the most according to claim 1, its
It is characterised by, repairs specifically comprising the following steps that of phase place process
The first step, first according to threshold process or manually iris wipe and determine occlusion area;
Second step, determines the high-low limit point Y in this regionuAnd Yl, and set blocked area and often go in length L of outside continuation and Fu
The exponent number of leaf-size class number;
3rd step, repairs y=YuLine phase: determine y=Yu, the right boundary x of the occlusion area of rowlAnd xr, take out [xl-L,
xr+ L] phase place, and approach by Fourier space, repair occlusion area y=Y according to the Coefficients of Approximation obtaineduThe phase place of row;
4th step, repairs y=y-1 line phase, repeats above-mentioned repair process, until all for occlusion area phase places having been repaired
Become.
Region projection method for measuring optical three-dimensional contour for HDR object the most according to claim 1, its
It is characterised by, in described step 4, by the bar graph repaired is carried out center extraction, calculates the three-D profile number of measured object
According to.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610368146.0A CN106091981B (en) | 2016-05-27 | 2016-05-27 | Region projection method for measuring optical three-dimensional contour for high dynamic range object |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610368146.0A CN106091981B (en) | 2016-05-27 | 2016-05-27 | Region projection method for measuring optical three-dimensional contour for high dynamic range object |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106091981A true CN106091981A (en) | 2016-11-09 |
CN106091981B CN106091981B (en) | 2018-09-04 |
Family
ID=57230799
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610368146.0A Active CN106091981B (en) | 2016-05-27 | 2016-05-27 | Region projection method for measuring optical three-dimensional contour for high dynamic range object |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106091981B (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108253907A (en) * | 2018-02-01 | 2018-07-06 | 深圳市易尚展示股份有限公司 | Method for three-dimensional measurement and device based on Hilbert transform phase error correction |
CN108645354A (en) * | 2018-07-06 | 2018-10-12 | 四川大学 | The structured light three-dimensional imaging method and system of high reflective subject surface |
CN109489585A (en) * | 2018-12-06 | 2019-03-19 | 广西师范大学 | Based on the method for three-dimensional measurement for improving multifrequency fringe structure light |
CN109579741A (en) * | 2018-11-01 | 2019-04-05 | 南京理工大学 | A kind of Full-automatic multimould state three-dimensional colour measurement method based on multi-angle of view |
CN110440714A (en) * | 2019-09-05 | 2019-11-12 | 南昌航空大学 | A kind of phase unwrapping package method based on multifrequency and binary system striped |
CN110645919A (en) * | 2019-08-23 | 2020-01-03 | 安徽农业大学 | Structured light three-dimensional measurement method based on airspace binary coding |
CN111402149A (en) * | 2020-03-06 | 2020-07-10 | 四川大学 | Fringe pattern restoration method based on convolutional neural network denoising regularization |
CN111473745A (en) * | 2020-06-23 | 2020-07-31 | 南京理工大学智能计算成像研究院有限公司 | Light-emitting surface microscopic three-dimensional measurement method based on multi-frequency phase shift scheme |
CN111998799A (en) * | 2020-08-20 | 2020-11-27 | 四川大学 | Multi-frequency projection three-dimensional surface shape measurement motion area detection method and system |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1999028704A1 (en) * | 1997-12-02 | 1999-06-10 | Universita' Degli Studi Di Brescia | Process for the measurement of three dimensional (3d) profiles by means of structured light projection |
CN1554925A (en) * | 2003-12-19 | 2004-12-15 | 上海交通大学 | High precision combined optical grating device for optical 3D measurement |
JP3914638B2 (en) * | 1997-09-09 | 2007-05-16 | シーケーディ株式会社 | Shape measuring device |
JP2009042015A (en) * | 2007-08-08 | 2009-02-26 | Ckd Corp | Three-dimensional measuring device and board inspection machine |
CN102175179A (en) * | 2011-02-23 | 2011-09-07 | 东南大学 | Method and device for three-dimensionally reestablishing surface contour of human body |
CN102607465A (en) * | 2012-03-13 | 2012-07-25 | 西安交通大学 | Phase unwrapping method based on colored phase shift stripe secondary encoding |
CN102944187A (en) * | 2012-10-18 | 2013-02-27 | 北京航空航天大学 | Method for acquiring phase position of fast bright reflection surface on basis of color stripe permutation projection |
-
2016
- 2016-05-27 CN CN201610368146.0A patent/CN106091981B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3914638B2 (en) * | 1997-09-09 | 2007-05-16 | シーケーディ株式会社 | Shape measuring device |
WO1999028704A1 (en) * | 1997-12-02 | 1999-06-10 | Universita' Degli Studi Di Brescia | Process for the measurement of three dimensional (3d) profiles by means of structured light projection |
CN1554925A (en) * | 2003-12-19 | 2004-12-15 | 上海交通大学 | High precision combined optical grating device for optical 3D measurement |
JP2009042015A (en) * | 2007-08-08 | 2009-02-26 | Ckd Corp | Three-dimensional measuring device and board inspection machine |
CN102175179A (en) * | 2011-02-23 | 2011-09-07 | 东南大学 | Method and device for three-dimensionally reestablishing surface contour of human body |
CN102607465A (en) * | 2012-03-13 | 2012-07-25 | 西安交通大学 | Phase unwrapping method based on colored phase shift stripe secondary encoding |
CN102944187A (en) * | 2012-10-18 | 2013-02-27 | 北京航空航天大学 | Method for acquiring phase position of fast bright reflection surface on basis of color stripe permutation projection |
Non-Patent Citations (2)
Title |
---|
LIJUN LI等: "Surface Reconstruction Based on Computer Stereo Vision Using Structured Light Projection", 《2009 INTERNATIONAL CONFERENCE ON INTELLIGENT HUMAN-MACHINE SYSTEMS AND CYBERNETICS》 * |
林焕等: "基于格雷码-相移的双目三维测量方法研究", 《电子学报》 * |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108253907A (en) * | 2018-02-01 | 2018-07-06 | 深圳市易尚展示股份有限公司 | Method for three-dimensional measurement and device based on Hilbert transform phase error correction |
CN108645354A (en) * | 2018-07-06 | 2018-10-12 | 四川大学 | The structured light three-dimensional imaging method and system of high reflective subject surface |
CN108645354B (en) * | 2018-07-06 | 2019-10-18 | 四川大学 | The structured light three-dimensional imaging method and system of high reflective subject surface |
CN109579741A (en) * | 2018-11-01 | 2019-04-05 | 南京理工大学 | A kind of Full-automatic multimould state three-dimensional colour measurement method based on multi-angle of view |
CN109579741B (en) * | 2018-11-01 | 2020-06-26 | 南京理工大学 | Full-automatic multi-mode three-dimensional color measurement method based on multiple visual angles |
CN109489585A (en) * | 2018-12-06 | 2019-03-19 | 广西师范大学 | Based on the method for three-dimensional measurement for improving multifrequency fringe structure light |
CN110645919A (en) * | 2019-08-23 | 2020-01-03 | 安徽农业大学 | Structured light three-dimensional measurement method based on airspace binary coding |
CN110440714A (en) * | 2019-09-05 | 2019-11-12 | 南昌航空大学 | A kind of phase unwrapping package method based on multifrequency and binary system striped |
CN110440714B (en) * | 2019-09-05 | 2021-01-08 | 南昌航空大学 | Phase unwrapping method based on multi-frequency and binary stripes |
CN111402149A (en) * | 2020-03-06 | 2020-07-10 | 四川大学 | Fringe pattern restoration method based on convolutional neural network denoising regularization |
CN111402149B (en) * | 2020-03-06 | 2022-07-08 | 四川大学 | Fringe pattern restoration method based on convolutional neural network denoising regularization |
CN111473745A (en) * | 2020-06-23 | 2020-07-31 | 南京理工大学智能计算成像研究院有限公司 | Light-emitting surface microscopic three-dimensional measurement method based on multi-frequency phase shift scheme |
CN111998799A (en) * | 2020-08-20 | 2020-11-27 | 四川大学 | Multi-frequency projection three-dimensional surface shape measurement motion area detection method and system |
Also Published As
Publication number | Publication date |
---|---|
CN106091981B (en) | 2018-09-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106091981A (en) | Region projection method for measuring optical three-dimensional contour for HDR object | |
CN109141232B (en) | Online detection method for disc castings based on machine vision | |
CN110476056B (en) | Visual detection method, detection equipment and robot | |
KR20190028794A (en) | GPU-based TFT-LCD Mura Defect Detection Method | |
CN103411533B (en) | Structured light self-adaptation multiexposure method | |
CN104101611A (en) | Mirror-like object surface optical imaging device and imaging method thereof | |
Lim et al. | Edge detection and measurement of nose radii of cutting tool inserts from scanned 2-D images | |
CN106996748A (en) | A kind of wheel footpath measuring method based on binocular vision | |
CN104111038A (en) | Method for using phase fusion algorithm to repair phase error caused by saturation | |
CN104568987A (en) | Method for inspecting transparent body, and apparatus for inspecting transparent body | |
CN109631797B (en) | Three-dimensional reconstruction invalid region rapid positioning method based on phase shift technology | |
CN112848281A (en) | Light compensation method for photocuring 3D printer | |
CN105354816A (en) | Electronic element positioning method and apparatus | |
CN113804418B (en) | Laser position adjusting method and device | |
CN107367245A (en) | Null Spot detection and elimination method in optical three-dimensional contour measurement | |
Wang et al. | Highly reflective surface measurement based on dual stereo monocular structured light system fusion | |
CN106813569A (en) | A kind of automobile tire 3-D positioning method based on line-structured light | |
Zhixin et al. | Adaptive centre extraction method for structured light stripes | |
TWI480508B (en) | Method and device of measuring position information of spatial image | |
CN111189840A (en) | Paper defect detection method with near-field uniform illumination | |
CN106767484A (en) | Fabric size rate of change automatic testing method, apparatus and system | |
Xue et al. | Reparability measurement of vision sensor in active stereo visual system | |
CN109447959A (en) | Tire static load mark measurement method | |
JP2012093234A (en) | Three-dimensional shape measurement device, three-dimensional shape measurement method, structure manufacturing method, and structure manufacturing system | |
JP7156225B2 (en) | Attached matter detection device and attached matter detection method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |