CN104596443A - Light plane equation fitting locating calibration method based on inherent characteristics of three-line laser - Google Patents
Light plane equation fitting locating calibration method based on inherent characteristics of three-line laser Download PDFInfo
- Publication number
- CN104596443A CN104596443A CN201510038858.1A CN201510038858A CN104596443A CN 104596443 A CN104596443 A CN 104596443A CN 201510038858 A CN201510038858 A CN 201510038858A CN 104596443 A CN104596443 A CN 104596443A
- Authority
- CN
- China
- Prior art keywords
- laser
- plane
- feature point
- lambda
- dispersed feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 84
- 238000005070 sampling Methods 0.000 claims description 26
- 230000003287 optical effect Effects 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 5
- 238000012797 qualification Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 3
- 238000013178 mathematical model Methods 0.000 abstract description 7
- 238000005259 measurement Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 7
- 230000009897 systematic effect Effects 0.000 description 4
- 238000002513 implantation Methods 0.000 description 3
- 230000001427 coherent effect Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000035800 maturation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
Abstract
The invention provides a light plane equation fitting locating calibration method based on inherent characteristics of a three-line laser and belongs to the field of fitting calibration methods that the space attitude equation of the three light planes emitted by the three-line laser is precisely fitted in a camera virtual space coordinate system. The calibration method especially provides the three real inherent characteristics that the adjacent angle sizes of the three laser planes are known, the three laser planes are all intersected at the same intersecting line and the projection light bars of the three laser planes are all parallel to each other as the constraint condition of a mathematical model of the projection features of the special three-line laser. Therefore, the objective function and the constraint condition which are capable of reflecting the three crucial characteristics completely are obtained, the possibility of the solution of the objective function is achieved, and a whole set of feasible methods that a series of physical characteristics of the projection planes of the three-line laser in the real world are accurately transplanted to the mathematical mode of the virtual camera coordinator system can be finally provided.
Description
Technical field
The invention belongs to the fitting calibrating method field of the equal Accurate Curve-fitting of spatial attitude equation in the coordinate system of camera Virtual Space of three optical planes sent by three laser line generators, be specifically related to a kind of optic plane equations matching positioning and demarcating method based on three laser line generator inherent characteristics.
Background technology
Vision measurement technology can adapt to new standard that modern industry proposes workpiece configurations size detection and requirement well, is a kind of contactless profile measurement means having both precision and efficiency.The basic thought of vision measuring method is: be incident upon on object to be detected with the laser plane that spatial attitude is known, and with camera, the projection of laser plane on object appearance profile is taken pictures, by detecting unique point image coordinate, obtain unique point three-dimensional coordinate corresponding under camera coordinates system according to known camera internal parameter and laser plane equation, and then obtain the three-dimensional information of profiling object surface.Become in the process of known quantity making the spatial attitude of laser plane, how just by the process such as modeling, demarcation, the laser plane that laser instrument is projected in real physical world becomes known mathematical space equation under the Virtual Space coordinate system of camera, is the emphasis of technical field of visual measurement research all the time.
At present, for solving and determining that the method for single linear structural laser space plane equation has been tending towards ripe, its method is various, their general character is all by demarcating the camera internal parameter and target three-dimensional information that obtain, solve volume coordinate (x, y, z) corresponding to multiple dispersed feature points on laser striation using the dispersed feature point as matching optical plane, utilize multiple non-colinear dispersed feature point can obtain the space equation of any one laser light plane.
Such as, be in " quick calibrating method of multi-linear structured light vision sensors " literary composition of 201310352766.1 at number of patent application, the acquisition methods containing one group of required dispersed feature point volume coordinate (x, y, z) is solved in mode just at its formula (12).
And for example, the Sun Q that the present inventor also delivers on academic journal Public Library of Science, Hou Y, Tan Q, in Li G (2014) A Flexible Calibration Method Using the PlanarTarget with a Square Pattern for Line Structured Light Vision System.PLoS ONE9 (9): e106911.doi:10.1371/journal.pone.0106911 mono-literary composition, propose in its chapters and sections 2.2Subpixelcenter localization of the light stripe and solve laser striation central point pixel coordinate (x
p, y
p) corresponding coordinate (X under camera coordinates system
c, Y
c, Z
c) extracting method, can obtain continuously according to the method and more accurate optical losses point pixel coordinate, therefore also can as obtaining one group of dispersed feature point volume coordinate (x required on laser striation, y, z) method, said method is referred to as dispersed feature point volume coordinate extracting method B.
By any one known dispersed feature point volume coordinate (x above-mentioned, y, z) acquisition methods, we all can carry out matching and solve a space plane equation approximately through these discrete points by obtaining one group of known dispersed feature point volume coordinate, thus under the camera virtual coordinate system established, obtain a laser plane spatial attitude equation, by the laser instrument projection plane in real physical world, under being transplanted to virtual camera coordinates system.Repeat said process, second, the 3rd can also be obtained further, to such an extent as to more virtual laser space plane equations newly.By this known method, can also realize further: by the multiple laser planes projected by multi-line laser in real physical world all under the Virtual Space coordinate system of camera rough split become the calibration process of known mathematical space equation.
But, this mode of the multiple optical plane of matching under same camera virtual coordinate system roughly can not meet the accuracy requirement of vision measurement to the coordinate setting of laser space plane equation far away, simply by the laser space plane equation model calibration technique repeated application for single line laser device in the Light-plane calibration of multi-line laser, there is obvious defect to the approximating method realizing the split under the same coordinate system of multiple optical plane.The existing laser space plane equation simulated by dispersed feature point is also accurate not, because the systematic error such as metrical error, Algorithm Error can hardly be avoided, therefore obtained laser space plane equation can not strictly overlap with the laser instrument projection plane in real world.Therefore, even if under identical optical plane spatial attitude equation demarcation condition, if first by obtained first and second laser space plane equation in virtual camera coordinates system downcrossings, and form the words of Article 1 intersection, then in the same way, when the 3rd laser space plane equation is also fitted under same virtual camera coordinates system, then under error interference, 3rd laser space plane equation can intersect at the new intersection of two other with the first two laser plane respectively respectively, and do not overlap with the intersection of the first and second laser space plane equations.
On the other hand, up-to-date research finds, multiple line structure light laser as shown in Figure 1 a kind ofly can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form three laser line generators of three parallel striations at same projection plane, it has unusual significance in the vision measuring method that some is special, because three laser line generators can send three linear structural laser planes from same linear light sources according to identical angle α, and the angle α between its each laser plane is all very real symmetric known quantities, therefore, if first laser plane a that three laser line generators are sent, the space equation of second laser plane b and the 3rd laser plane c solves respectively and by three's Accurate Curve-fitting in camera coordinates system, finally obtain under the Virtual Space coordinate system of camera, three virtual laser plane space equation all meet at same intersection L, then can migrate among the camera Virtual Space coordinate system of vision measurement imaging system by having the known intrinsic real physical characteristics of its three laser plane angle α of aforementioned three laser line generators, and in follow-up vision measurement process, accuracy and the efficiency of measurement is significantly improved by this key property.And in aforementioned invention 201310352766.1 " quick calibrating method of a multi-linear structured light vision sensors " literary composition, also describe a kind of scaling method of multi-linear structured light vision sensors under camera virtual coordinate system, in theory, the method can be directly used in and can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form the calibration process of three laser line generators of three parallel striations at same projection plane.But its method does not utilize required laser instrument from three laser planes that same line source sends, angle this true and known intrinsic physical characteristics all identical of adjacent two laser planes, therefore cannot revise further and reduce systematic error.In addition, the scaling method that aforementioned invention 201310352766.1 " quick calibrating method of multi-linear structured light vision sensors " is introduced also must utilize the Cross ration invariability principle of the intrinsic gridline on laser projection striation and directivity grid target, could assist dispersed feature point volume coordinate (x, y, z) acquisition and the demarcation of multiple laser space plane equations under same camera coordinates system.But due to the limited amount of grid lines intrinsic on directivity grid, therefore also therefore relatively less by the quantity of its crossing with laser projection striation obtainable dispersed feature point, and less dispersed feature point quantity must be unfavorable for the simulated coordinate precision of laser space plane equation in the virtual system of camera.
In addition, the Sun Q that the present inventor publishes on academic journal Public Library of Science, Hou Y, Tan Q, in Li G (2014) A Flexible Calibration Method Using the PlanarTarget with a Square Pattern for Line Structured Light Vision System.PLoS ONE9 (9): e106911.doi:10.1371/journal.pone.0106911 mono-literary composition, a kind of method that camera internal parameter is demarcated also is also disclosed at the chapters and sections 1.Calibration ofcamera model of chapter 1 Methods, referred to as camera internal parameter calibration method A.
Summary of the invention
In order to solve the existing laser space plane equation model calibration technique for single line laser device, its mode with split roughly under same camera virtual coordinate system the mode of the multiple optical plane of matching can not meet the accuracy requirement of vision measurement to the coordinate setting of laser space plane equation far away, and the existing laser space plane equation scaling method for multiple line structure light laser fails to make full use of the intrinsic important physical of multiple line structure light laser, and, also the acquisition of dispersed feature point volume coordinate and the demarcation of multiple laser space plane equations under same camera coordinates system must could be assisted by the Cross ration invariability principle of the intrinsic gridline on laser projection striation and directivity grid target.But due to the limited amount of grid lines intrinsic on directivity grid, cause by the quantity of its crossing with laser projection striation obtainable dispersed feature point also therefore relatively less, therefore cannot revise and reduce systematic error, be unfavorable for the technical matters improving the fitting calibrating precision of laser space plane equation in camera virtual coordinate system further, the invention provides a kind of optic plane equations matching positioning and demarcating method based on three laser line generator inherent characteristics.
The technical scheme that technical solution problem of the present invention is taked is as follows:
Based on an optic plane equations matching positioning and demarcating method for three laser line generator inherent characteristics, it is characterized in that, the method comprises the steps:
Step one: select and a kind ofly can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form three laser line generators of three parallel striations at same projection plane, and from the intrinsic performance index parameter of dispatching from the factory of this three laser line generator, obtain the angle value parameter alpha of its contiguous two laser plane angles;
Step 2: complete the demarcation that camera internal parameter is carried out according to camera internal parameter calibration method A;
Step 3: the set R obtaining the N group dispersed feature point spatial value for first laser plane (a) in three laser line generators, it specifically comprises following sub-step:
Step 3.1: one group of required dispersed feature point volume coordinate (x is carried out to the laser striation (a-1) of first laser plane (a) of three laser line generators described in step one in target plane (K) according to dispersed feature point volume coordinate extracting method B, y, z) sampling and extraction, thus obtain and to sample the first group of dispersed feature point volume coordinate obtained for above the project laser striation (a-1) that formed of the aforementioned target plane of first laser plane (a) under initial attitude (K):
wherein i=1,2 ..., n, n represent the number of the dispersed feature point at laser striation (a-1) up-sampling, coordinate figure (x, y, z) respective superscript (a) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a);
Step 3.2: the inclination angle changing laser projection target plane (K), and repeat the process of step 3.1, thus obtain and to sample the second group of dispersed feature point volume coordinate obtained for above the project laser striation (a-2) that formed of the target plane of aforementioned first laser plane (a) under new attitude (K '):
wherein i=(n+1), (n+2), ..., 2n, n represents the number of the dispersed feature point at laser striation (a-2) up-sampling, the respective superscript (a) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a);
Step 3.3: by N (N>=3, N is natural number) inclination angle of secondary change laser projection target plane repeat the process of step 3.2, to sample the N group dispersed feature point spatial value obtained until obtain the laser striation (a-N) projecting formed in N number of new attitude of target plane (K) for aforementioned first laser plane (a):
wherein i=((N-1) × n+1), ((N-1) × n+2) ..., N × n, n represents the number of the dispersed feature point at laser striation (a-N) up-sampling, coordinate figure (x, y, z) respective superscript (a) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a);
Step 3.4: the N group dispersed feature point altogether step 3.1 to step 3.3 obtained respectively is integrated and become the set R of a non-colinear dispersed feature point:
I=1 in formula (1), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to N time at laser striation (a-1), (a-2) ... (a-N) sum of the dispersed feature point of up-sampling, the respective superscript (a) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a); N represents the change total degree of target planar inclination;
Step 4: adopt and the identical method of step 3, obtain respectively for second laser plane (b) in three laser line generators, integrate become the set of a non-colinear dispersed feature point by amounting to N group dispersed feature point
and for the 3rd laser plane (c) in three laser line generators, integrate become the set of a non-colinear dispersed feature point by amounting to N group dispersed feature point
J=1 in formula (2), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to N time at laser striation (b-1), (b-2) ... (b-N) sum of the dispersed feature point of up-sampling, the respective superscript (b) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (b); N represents the change total degree of target planar inclination;
K=1 in formula (3), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to N time at laser striation (c-1), (c-2) ... (c-N) sum of the dispersed feature point of up-sampling, the respective superscript (c) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (c); N represents the change total degree of target planar inclination;
Step 5: according to the math equation of multiple space plane equations under three-dimensional coordinate system, defines one and represents that three space planes all intersect at the math equation group expression formula of same intersection:
In formula (4), (x, y, z) represents the spatial value of the one group of dispersed feature point met on three space planes of system of equations;
Multinomial coefficient (A to be solved
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2, λ) then jointly define three space planes spatial attitude coefficient separately that expression meets system of equations (4);
First equation in formula (4) by formula described in step 3 (1) simulate, the laser space plane equation of first laser plane (a) of three laser line generators;
Second equation in formula (4) by formula described in step 4 (2) simulate, the laser space plane equation of second laser plane (b) of three laser line generators;
The 3rd equation in formula (4) by formula described in step 4 (3) simulate, the laser space plane equation of the 3rd laser plane (c) of three laser line generators;
The 3rd equation in formula (4) represents the intersection of the laser space plane equation of first laser plane (a) and second laser plane (b) in the 3rd the strict through type of laser plane (c) (4) simultaneously;
Step 6: the set utilizing the one group of non-colinear dispersed feature point volume coordinate for aforementioned first laser plane (a) obtained by step 3
i=1,2 ..., (N × n);
The set of the one group of non-colinear dispersed feature point volume coordinate for aforementioned second laser plane (b) obtained by step 4
j=1,2 ..., (N × n);
And the set of the one group of non-colinear dispersed feature point volume coordinate for aforementioned 3rd laser plane (c) to be obtained by step 4
k=1,2 ..., (N × n) is common as fitting data, all intersect at the math equation group expression formula (4) of same intersection as fitting function using by the determined expression of step 5 three space planes, and set up unified objective function according to known least square method:
Step 7: the performance index parameter of dispatching from the factory using intrinsic for three laser line generators described in step one: the angle value parameter alpha of contiguous two laser plane angles as known constraint condition, to the respective spatial attitude multinomial coefficient (A of three laser planes in the determined objective function of step 6 (5)
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2, λ) and do the restriction of further mathematics, thus obtain the constraint expression formula including angular definitions condition:
Step 8: by known Lenvenberg-Marquardt nonlinear optimization algorithm, formula (5) and qualifications expression formula (6) thereof are solved simultaneously, thus solve (A
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2value λ), and complete to utilize and a kind ofly can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form the inherent characteristic of three laser line generators of three parallel striations at same projection plane, the fitting calibrating process of the equal Accurate Curve-fitting of spatial attitude equation in the coordinate system of camera Virtual Space of three optical planes that three laser line generators are sent.
The invention has the beneficial effects as follows: scaling method of the present invention propose especially known for the contiguous angle amount of three laser planes, three laser planes all intersected at same intersection, three laser planes all parallel these three the real inherent characteristics of projection striation simultaneously as the constraint condition of the mathematical model of described special three laser line generator projection properties, obtain objective function and the constraint condition thereof that overallly can reflect these three key features thus, and bring possibility for solving of this objective function.Finally give a whole set of by the series of physical feature of three laser line generator projection plane in real world all simultaneously accurate implantation to the feasible method under the mathematical model of virtual camera coordinates system.In addition, present invention overcomes adopt old for solving and determining the method for single linear structural laser space plane equation laser plane equation multiple directly to split under same camera virtual coordinate system time, its roughly under same camera virtual coordinate system the mode of the multiple optical plane of matching can not meet the problem of vision measurement to the accuracy requirement of laser space plane equation coordinate setting far away, and improve the error of multi-line structured light fitting calibrating method, make stated accuracy obtain larger raising.
Accompanying drawing explanation
Fig. 1 can send three all identical laser planes of contiguous angle from same line source and its three laser planes can form the position relationship schematic diagram of three laser planes of three laser line generators of three parallel striations at same projection plane.
Fig. 2 is the parallel relation schematic diagram of three the projection striations of three laser planes in same target plane described in Fig. 1.
Fig. 3 is the vertical view of Fig. 2.
Fig. 4 be the laser plane of three in Fig. 2 is simplified respectively and retain three laser planes project with target plane respectively after form the schematic diagram of three laser striations parallel to each other.
Fig. 5 changes the spatial attitude of target plane and inclination angle and the position relationship schematic diagram of newly-generated three laser planes projection striation thereon on Fig. 2 basis.
Fig. 6 is the vertical view of Fig. 5.
Fig. 7 be the laser plane of three in Fig. 5 is simplified respectively and retain three laser planes project with target plane respectively after form the schematic diagram of three laser striations parallel to each other.
Embodiment
Below in conjunction with accompanying drawing, the present invention is described in further details.
Three laser line generators as shown in Figure 1 can send three all identical laser planes of contiguous angle from same line source, and its first laser plane a, second laser plane b and the 3rd laser plane c can form three parallel striations on same projection target plane K, laser striation a-1, laser striation b-1 namely shown in Fig. 2 and laser striation c-1.The STR-660-20-L01 type three wire configuration light laser that this laser instrument selects COHERENT company to manufacture.As shown in Figure 3, when this laser instrument dispatches from the factory, the angle value parameter alpha of contiguous two intrinsic angles of laser plane is 11.7 °.
Fig. 4 be the laser plane of three in Fig. 2 is simplified respectively and only retain three laser planes project with target plane respectively afterwards form the schematic diagram of three laser striations parallel to each other, Fig. 5 is then the spatial attitude changing target plane K on Fig. 2 basis, and is K at inclination angle
'new target plane on the newly-generated projection of three laser planes striation, that is: the position relationship schematic diagram of laser striation a-2, laser striation b-2 and laser striation c-2.Keep the position of three laser line generators and attitude all constant, and continue the space obliquity and attitude changing target plane K, then can also obtain N (N >=3, N is natural number) spatial attitude of secondary change target plane K time, to be projected striation, that is: laser striation a-N, laser striation b-N and a laser striation c-N by three laser planes of laser instrument newly-generated three laser planes of institute that project under the current pose of the target plane K of this correspondence.
A kind of optic plane equations matching positioning and demarcating method based on three laser line generator inherent characteristics of the present invention comprises the steps:
Step one: select and a kind ofly can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form three laser line generators of three parallel striations at same projection plane, the STR-660-20-L01 type three wire configuration light laser such as selecting COHERENT company to manufacture, obtains angle value parameter alpha=11.7 ° of its contiguous two laser plane angles from the performance index parameter of dispatching from the factory that this three laser line generator is intrinsic;
Step 2: according to the Sun Q that academic journal PublicLibrary of Science publishes, Hou Y, Tan Q, in Li G (2014) A Flexible Calibration Method Using the Planar Target with aSquare Pattern for Line Structured Light Vision System.PLoS ONE 9 (9): e106911.doi:10.1371/journal.pone.0106911 mono-literary composition, described in the chapters and sections 1.Calibrationof camera model of chapter 1 Methods, method completes the demarcation carried out camera internal parameter,
Step 3: the set R obtaining the N group dispersed feature point spatial value for first laser plane a in three laser line generators, it specifically comprises following sub-step:
Step 3.1: what propose in its chapters and sections 2.2Subpixel center localizationof the light stripe according to academic journal described in step 2 solves laser striation central point pixel coordinate (x
p, y
p) corresponding coordinate (X under camera coordinates system
c, Y
c, Z
c) more exact method, one group of required dispersed feature point volume coordinate (x is carried out to the laser striation (a-1) of first laser plane a on target plane K of three laser line generators described in step one, y, z) sampling and extraction, thus obtain and on the target plane K under initial attitude, to project laser striation (a-1) formed sample the first group of dispersed feature point volume coordinate obtained for aforementioned first laser plane a:
wherein i=1,2 ..., n, n represent the number of the dispersed feature point at laser striation (a-1) up-sampling, and the respective superscript (a) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane a; To herein
i=1 ..., the sampling number object n value in n is set to 30, then can obtain one group for laser striation (a-1) and comprise 30 dispersed feature point spatial values
i=1,2 ..., the set of the point of 30;
Step 3.2: as shown in Figures 5 to 7, change the inclination angle of laser projection target plane K, the spatial attitude inclination angle of laser projection target plane is made to become K ', repeat the process of step 3.1, thus to sample the second group of dispersed feature point volume coordinate obtained for above the project laser striation (a-2) that formed of the target plane K ' of aforementioned first laser plane a under new attitude:
wherein i=(n+1), (n+2) ..., 2n, n represent the number of the dispersed feature point at laser striation (a-2) up-sampling, coordinate figure (x, y, z) respective superscript a all represents that this dispersed feature point is under the jurisdiction of first laser plane a;
To herein
i=(n+1), (n+2), ..., sampling number object n value in 2n is still set to 30, then can carry out for the laser striation (a-2) on the target plane K ' of aforementioned first laser plane a under new attitude sampling and the extraction that second group comprises 30 dispersed feature point spatial values: can obtain new one group and comprise 30 dispersed feature point spatial values for laser striation (a-2):
i=31,32..., 60, the set of point;
Step 3.3: by N (N>=3, N is natural number) inclination angle of secondary change laser projection target plane repeat the process of step 3.2, to sample the N group dispersed feature point spatial value obtained until obtain the laser striation (a-N) projecting formed in N number of new attitude of target plane (K) for aforementioned first laser plane (a):
wherein i=((N-1) × n+1), ((N-1) × n+2) ..., N × n, n represents the number of the dispersed feature point at laser striation (a-N) up-sampling, coordinate figure (x, y, z) respective superscript (a) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a);
To herein
i=((N-1) × n+1), ((N-1) × n+2) ..., sampling number object n value in N × n is still set to 30 and makes N=9, then the laser striation (a-9) that can project generated under the 9th attitude of target plane K for aforementioned first laser plane a carries out sampling and the extraction that the 9th group comprises 30 dispersed feature point spatial values: can obtain new one group and comprise 30 dispersed feature point spatial values for laser striation (a-9)
i=(8 × 30+1), (8 × 30+2) ..., the set of the point of (9 × 30);
Step 3.4: the nine groups of dispersed feature points altogether step 3.1 to step 3.3 obtained respectively are integrated and become the set R of a non-colinear dispersed feature point:
I=1 in formula (1), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to N time at laser striation (a-1), (a-2) ... (a-N) sum of the dispersed feature point of up-sampling, the respective superscript a of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane a; N represents the change total degree of target planar inclination;
That is: when the obliquity and attitude of target plane K successively changes N=9 time, and corresponding for first of laser instrument the laser plane a attitude each time at target plane K change project afterwards generate laser striation (a-1), (a-2) ... (a-N) when the sampled point number n on is all set to 30, then the set that obtained nine groups can be comprised respectively 30 dispersed feature points merges jointly becomes a non-colinear set comprising 270 dispersed feature point volume coordinates
i=1,2 ..., (9 × 30); Can obtain for first laser plane a and include the new set R of 270 dispersed feature points;
Step 4: adopt and the identical method of step 3, obtain and for second laser plane b in three laser line generators, integrate and become one for second laser plane b by amounting to nine groups of dispersed feature points and include the set of 270 non-colinear dispersed feature points
J=1 in formula (2), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to nine times at laser striation (b-1), (b-2) ... (b-N) sum of the dispersed feature point of up-sampling, the respective superscript (b) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (b); N represents the change total degree of target planar inclination; That is: when the obliquity and attitude of target plane K successively changes N=9 time, and corresponding for first of laser instrument the laser plane b attitude each time at target plane K change project afterwards generate laser striation (b-1), (b-2) ... (b-N) when the sampled point number n on is all set to 30, then the set that obtained nine groups can be comprised respectively 30 dispersed feature points merges jointly becomes a non-colinear set comprising 270 dispersed feature point volume coordinates:
j=1,2 ..., (9 × 30);
That is: when the obliquity and attitude of target plane K successively changes N=9 time, and corresponding for second of laser instrument the laser plane b attitude each time at target plane K change project afterwards generate laser striation (b-1), (b-2) ... (b-N), when the sampled point number n on is all set to 30, can obtains for second laser plane b and include the new S set of 270 dispersed feature points;
Adopt and the identical method of step 3, can also obtain and for the 3rd laser plane c in three laser line generators, integrate and become one for the 3rd laser plane c by amounting to nine groups of dispersed feature points and include the set of 270 non-colinear dispersed feature points
k=1,2..., (9 × 30) ... (3);
K=1 in formula (3), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to nine times at laser striation (c-1), (c-2) ... (c-N) sum of the dispersed feature point of up-sampling, the respective superscript (c) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (c); N represents the change total degree of target planar inclination; That is: when the obliquity and attitude of target plane K successively changes N=9 time, and corresponding for the 3rd of laser instrument the laser plane c attitude each time at target plane K change project afterwards generate laser striation (c-1), (c-2) ... (c-N), when the sampled point number n on is all set to 30, can obtains for the 3rd laser plane c and include the new set T of 270 dispersed feature points;
Step 5: according to the math equation of multiple space plane equations under three-dimensional coordinate system, defines one and represents that three space planes all intersect at the math equation group expression formula of same intersection:
In formula (4), (x, y, z) represents the spatial value of the one group of dispersed feature point met on three space planes of system of equations;
Multinomial coefficient (A to be solved
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2, λ) then jointly define three space planes spatial attitude coefficient separately that expression meets system of equations (4);
First equation in formula (4) by formula described in step 3 (1) simulate, the laser space plane equation of first laser plane a of three laser line generators;
Second equation in formula (4) by formula described in step 4 (2) simulate, the laser space plane equation of second laser plane b of three laser line generators;
The 3rd equation in formula (4) by formula described in step 4 (3) simulate, the laser space plane equation of the 3rd laser plane c of three laser line generators;
The 3rd equation in formula (4) represents the intersection of the laser space plane equation of first laser plane a and second laser plane b in the 3rd the strict through type of laser plane c (4) simultaneously;
Step 6: simultaneously utilize the one group of non-colinear dispersed feature point volume coordinate set for aforementioned first laser plane a obtained by step 3
i=1 ..., (9 × 30) and the one group of non-colinear dispersed feature point volume coordinate set for aforementioned second laser plane b obtained by step 4
j=1 ..., (9 × 30) and the one group of non-colinear dispersed feature point volume coordinate set for aforementioned 3rd laser plane c obtained by step 4
k=1, ..., (9 × 30) are common as fitting data, all intersect at the math equation group expression formula (4) of same intersection as fitting function, and set up unified objective function according to known least square method using by the determined expression of step 5 three space planes:
Step 7: the performance index parameter of dispatching from the factory using intrinsic for three laser line generators described in step one: angle value parameter alpha=11.7 of contiguous two laser plane angles ° as known constraint condition, to the respective spatial attitude multinomial coefficient (A of three laser planes in the determined objective function of step 6 (5)
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2, λ) and do the restriction of further mathematics, thus obtain the constraint expression formula including angular definitions condition:
Step 8: by known Lenvenberg-Marquardt nonlinear optimization algorithm, formula (5) and qualifications expression formula (6) thereof are solved simultaneously, thus solve (A
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2value λ), and complete to utilize and a kind ofly can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form the inherent characteristic of three laser line generators of three parallel striations at same projection plane, the fitting calibrating process of the equal Accurate Curve-fitting of spatial attitude equation in the coordinate system of camera Virtual Space of three optical planes that three laser line generators are sent.
By solving while formula (5) and qualifications expression formula (6) thereof, the non trivial solution obtained is inevitable have expressed following implication accurately, simultaneously:
First: represented by plane equation group three known planes after trying to achieve solution, a, b, c tri-rays as shown in Fig. 3 or Fig. 6 must be formed; Three Projection Line Segments of its three rays in same target plane are inevitable parallel to each other;
Second: represented by plane equation group three known planes after trying to achieve solution, accurately intersect at same intersection L while of inevitable;
3rd: represented by plane equation group three known planes after trying to achieve solution, the angle between its contiguous two planes must equal known quantity α;
It can thus be appreciated that, represented by plane equation group three known planes after trying to achieve solution are necessarily aforementioned can send three all identical laser planes of contiguous angle from same line source and its three laser planes can form the desirable mathematical model of three laser line generators of three parallel striations at same projection plane, namely achieve by the series of physical feature of three laser line generator projection plane in real world all simultaneously accurate implantation under the mathematical model of virtual camera coordinates system.
Optic plane equations matching positioning and demarcating method of the present invention directly solves laser striation central point pixel coordinate (x disclosed in the present inventor is on academic journal Public Library of Science
p, y
p) corresponding coordinate (X under camera coordinates system
c, Y
c, Z
c) the central point pixel coordinate of more exact method to the projection striation pattern of laser plane in camera photos extract, the optical losses obtained is the central point of each pixel on continuous striation pattern, the data volume of the dispersed feature point that can be obtained by this maturation method and precision all far away higher than data volume and the coordinate accuracy of the dispersed feature point adopting the method for the extremely limited directivity grid of intrinsic mesh lines bar quantity to obtain, thus provide reliable guarantee from the angle of Data Source for revising further and reducing systematic error.
On the other hand, the present invention takes full advantage of and a kind ofly can send three all identical laser planes of contiguous angle from same line source and its three laser planes can form the inherent characteristic of three laser line generators of three parallel striations at same projection plane: its high-precision contiguous laser plane angle is the known quantity that its index of dispatching from the factory provides, and three laser planes that this laser instrument sends projected from same accurate line source truly.
The most important thing is, scaling method of the present invention propose especially known for the contiguous angle amount of three laser planes, three laser planes all intersected at same intersection, three laser planes all parallel these three the real inherent characteristics of projection striation simultaneously as the constraint condition of the mathematical model of described special three laser line generator projection properties, objective function and the constraint condition thereof that overallly can reflect these three key features could be obtained thus, and bring possibility for solving of this objective function.Finally give a whole set of by the series of physical feature of three laser line generator projection plane in real world all simultaneously accurate implantation to the feasible method under the mathematical model of virtual camera coordinates system.
In addition, present invention overcomes adopt old for solving and determining the method for single linear structural laser space plane equation laser plane equation multiple directly to split under same camera virtual coordinate system time, its roughly under same camera virtual coordinate system the mode of the multiple optical plane of matching can not meet the problem of vision measurement to the accuracy requirement of laser space plane equation coordinate setting far away, and improve the error of multi-line structured light fitting calibrating method, make stated accuracy obtain larger raising.
Claims (1)
1., based on the optic plane equations matching positioning and demarcating method of three laser line generator inherent characteristics, it is characterized in that, the method comprises the steps:
Step one: select and a kind ofly can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form three laser line generators of three parallel striations at same projection plane, and from the intrinsic performance index parameter of dispatching from the factory of this three laser line generator, obtain the angle value parameter alpha of its contiguous two laser plane angles;
Step 2: complete the demarcation to camera internal parameter according to camera internal parameter calibration method A;
Step 3: the set R obtaining the N group dispersed feature point spatial value for first laser plane (a) in three laser line generators, it specifically comprises following sub-step:
Step 3.1: one group of required dispersed feature point volume coordinate (x is carried out to the laser striation (a-1) of first laser plane (a) of three laser line generators described in step one in target plane (K) according to dispersed feature point volume coordinate extracting method B, y, z) sampling and extraction, thus obtain and to sample the first group of dispersed feature point volume coordinate obtained for above the project laser striation (a-1) that formed of the aforementioned target plane of first laser plane (a) under initial attitude (K):
wherein i=1,2 ..., n, n represent the number of the dispersed feature point at laser striation (a-1) up-sampling, coordinate figure (x, y, z) respective superscript (a) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a);
Step 3.2: the inclination angle changing laser projection target plane (K), and repeat the process of step 3.1, thus obtain and to sample the second group of dispersed feature point volume coordinate obtained for above the project laser striation (a-2) that formed of the target plane of aforementioned first laser plane (a) under new attitude (K '):
wherein i=(n+1), (n+2), ..., 2n, n represents the number of the dispersed feature point at laser striation (a-2) up-sampling, the respective superscript (a) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a);
Step 3.3: by N (N>=3, N is natural number) inclination angle of secondary change laser projection target plane repeat the process of step 3.2, to sample the N group dispersed feature point spatial value obtained until obtain the laser striation (a-N) projecting formed in N number of new attitude of target plane (K) for aforementioned first laser plane (a):
wherein i=((N-1) × n+1), ((N-1) × n+2) ..., N × n, n represents the number of the dispersed feature point at laser striation (a-N) up-sampling, coordinate figure (x, y, z) respective superscript (a) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a);
Step 3.4: the N group dispersed feature point altogether step 3.1 to step 3.3 obtained respectively is integrated and become the set R of a non-colinear dispersed feature point:
I=1 in formula (1), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to N time at laser striation (a-1), (a-2) ... (a-N) sum of the dispersed feature point of up-sampling, the respective superscript (a) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (a); N represents the change total degree of target planar inclination;
Step 4: adopt and the identical method of step 3, obtain respectively for second laser plane (b) in three laser line generators, integrate become the set of a non-colinear dispersed feature point by amounting to N group dispersed feature point
and for the 3rd laser plane (c) in three laser line generators, integrate become the set of a non-colinear dispersed feature point by amounting to N group dispersed feature point
J=1 in formula (2), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to N time at laser striation (b-1), (b-2) ... (b-N) sum of the dispersed feature point of up-sampling, the respective superscript (b) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (b); N represents the change total degree of target planar inclination;
K=1 in formula (3), 2, ..., N × n, N × n represent in step 3.1 to step 3.3 amount to N time at laser striation (c-1), (c-2) ... (c-N) sum of the dispersed feature point of up-sampling, the respective superscript (c) of coordinate figure (x, y, z) all represents that this dispersed feature point is under the jurisdiction of first laser plane (c); N represents the change total degree of target planar inclination;
Step 5: according to the math equation of multiple space plane equations under three-dimensional coordinate system, defines one and represents that three space planes all intersect at the math equation group expression formula of same intersection:
In formula (4), (x, y, z) represents the spatial value of the one group of dispersed feature point met on three space planes of system of equations;
Multinomial coefficient (A to be solved
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2, λ) then jointly define three space planes spatial attitude coefficient separately that expression meets system of equations (4);
First equation in formula (4) by formula described in step 3 (1) simulate, the laser space plane equation of first laser plane (a) of three laser line generators;
Second equation in formula (4) by formula described in step 4 (2) simulate, the laser space plane equation of second laser plane (b) of three laser line generators;
The 3rd equation in formula (4) by formula described in step 4 (3) simulate, the laser space plane equation of the 3rd laser plane (c) of three laser line generators;
The 3rd equation in formula (4) represents the intersection of the laser space plane equation of first laser plane (a) and second laser plane (b) in the 3rd the strict through type of laser plane (c) (4) simultaneously;
Step 6: the set utilizing the one group of non-colinear dispersed feature point volume coordinate for aforementioned first laser plane (a) obtained by step 3
i=1,2 ..., (N × n);
The set of the one group of non-colinear dispersed feature point volume coordinate for aforementioned second laser plane (b) obtained by step 4
j=1,2 ..., (N × n);
And the set of the one group of non-colinear dispersed feature point volume coordinate for aforementioned 3rd laser plane (c) to be obtained by step 4
k=1,2 ..., (N × n) is common as fitting data, all intersect at the math equation group expression formula (4) of same intersection as fitting function using by the determined expression of step 5 three space planes, and set up unified objective function according to known least square method:
Step 7: the performance index parameter of dispatching from the factory using intrinsic for three laser line generators described in step one: the angle value parameter alpha of contiguous two laser plane angles as known constraint condition, to the respective spatial attitude multinomial coefficient (A of three laser planes in the determined objective function of step 6 (5)
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2, λ) and do the restriction of further mathematics, thus obtain the constraint expression formula including angular definitions condition:
Step 8: by known Lenvenberg-Marquardt nonlinear optimization algorithm, formula (5) and qualifications expression formula (6) thereof are solved simultaneously, thus solve (A
1, B
1, C
1, D
1, A
2, B
2, C
2, D
2value λ), and complete to utilize and a kind ofly can send three all identical laser planes of contiguous angle from same line source, and its three laser planes can form the inherent characteristic of three laser line generators of three parallel striations at same projection plane, the fitting calibrating process of the equal Accurate Curve-fitting of spatial attitude equation in the coordinate system of camera Virtual Space of three optical planes that three laser line generators are sent.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510038858.1A CN104596443B (en) | 2015-01-26 | 2015-01-26 | Light plane equation fitting locating calibration method based on inherent characteristics of three-line laser |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510038858.1A CN104596443B (en) | 2015-01-26 | 2015-01-26 | Light plane equation fitting locating calibration method based on inherent characteristics of three-line laser |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104596443A true CN104596443A (en) | 2015-05-06 |
CN104596443B CN104596443B (en) | 2017-02-01 |
Family
ID=53122387
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510038858.1A Expired - Fee Related CN104596443B (en) | 2015-01-26 | 2015-01-26 | Light plane equation fitting locating calibration method based on inherent characteristics of three-line laser |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104596443B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105157604A (en) * | 2015-08-07 | 2015-12-16 | 天津大学 | Rapid calibration method of outfield multi-beam line-structured light vision measuring system |
CN109709508A (en) * | 2019-01-17 | 2019-05-03 | 电子科技大学 | A kind of optics AOA localization method based on sensor node |
CN112665530A (en) * | 2021-01-05 | 2021-04-16 | 银昌龄 | Light plane recognition device corresponding to projection line, three-dimensional measurement system and method |
CN112954633A (en) * | 2021-01-26 | 2021-06-11 | 电子科技大学 | Parameter constraint-based dual-network architecture indoor positioning method |
CN113983933A (en) * | 2021-11-11 | 2022-01-28 | 易思维(杭州)科技有限公司 | Calibration method of multi-line laser sensor |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11316113A (en) * | 1998-04-30 | 1999-11-16 | Asahi Glass Co Ltd | Form measuring device |
CN1508511A (en) * | 2002-12-17 | 2004-06-30 | 北京航空航天大学 | Method for calibrating structure optical vision sensor |
CN101943563A (en) * | 2010-03-26 | 2011-01-12 | 天津大学 | Rapid calibration method of line-structured light vision sensor based on space plane restriction |
CN103411553A (en) * | 2013-08-13 | 2013-11-27 | 天津大学 | Fast calibration method of multiple line structured light visual sensor |
CN103884271A (en) * | 2012-12-20 | 2014-06-25 | 中国科学院沈阳自动化研究所 | Direct calibration method for line structured light vision sensor |
-
2015
- 2015-01-26 CN CN201510038858.1A patent/CN104596443B/en not_active Expired - Fee Related
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11316113A (en) * | 1998-04-30 | 1999-11-16 | Asahi Glass Co Ltd | Form measuring device |
CN1508511A (en) * | 2002-12-17 | 2004-06-30 | 北京航空航天大学 | Method for calibrating structure optical vision sensor |
CN101943563A (en) * | 2010-03-26 | 2011-01-12 | 天津大学 | Rapid calibration method of line-structured light vision sensor based on space plane restriction |
CN103884271A (en) * | 2012-12-20 | 2014-06-25 | 中国科学院沈阳自动化研究所 | Direct calibration method for line structured light vision sensor |
CN103411553A (en) * | 2013-08-13 | 2013-11-27 | 天津大学 | Fast calibration method of multiple line structured light visual sensor |
Non-Patent Citations (3)
Title |
---|
QIUCHENG SUN ECT.AL: "《A Flexible Calibration Method Using the Planar Target with a Square Pattern for Line Structured Light Vision System》", 《PUBLIC LIBRARY OF SCIENCE》 * |
刘宁 等: "《一种用于计算三维视觉测量中线结构光平面的新型算法》", 《光子学报》 * |
陈天飞 等: "《基于主动视觉标定线结构光传感器中的光平面》", 《光学精密工程》 * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105157604A (en) * | 2015-08-07 | 2015-12-16 | 天津大学 | Rapid calibration method of outfield multi-beam line-structured light vision measuring system |
CN105157604B (en) * | 2015-08-07 | 2017-07-11 | 天津大学 | The quick calibrating method of outfield multi beam Vision Measuring System With Structured Light Stripe |
CN109709508A (en) * | 2019-01-17 | 2019-05-03 | 电子科技大学 | A kind of optics AOA localization method based on sensor node |
CN109709508B (en) * | 2019-01-17 | 2020-09-01 | 电子科技大学 | Optical AOA positioning method based on sensor nodes |
CN112665530A (en) * | 2021-01-05 | 2021-04-16 | 银昌龄 | Light plane recognition device corresponding to projection line, three-dimensional measurement system and method |
CN112665530B (en) * | 2021-01-05 | 2022-09-23 | 银昌龄 | Light plane recognition device corresponding to projection line, three-dimensional measurement system and method |
CN112954633A (en) * | 2021-01-26 | 2021-06-11 | 电子科技大学 | Parameter constraint-based dual-network architecture indoor positioning method |
CN112954633B (en) * | 2021-01-26 | 2022-01-28 | 电子科技大学 | Parameter constraint-based dual-network architecture indoor positioning method |
CN113983933A (en) * | 2021-11-11 | 2022-01-28 | 易思维(杭州)科技有限公司 | Calibration method of multi-line laser sensor |
CN113983933B (en) * | 2021-11-11 | 2022-04-19 | 易思维(杭州)科技有限公司 | Calibration method of multi-line laser sensor |
Also Published As
Publication number | Publication date |
---|---|
CN104596443B (en) | 2017-02-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN113014906B (en) | 3D scene reconstruction method, system and computer program storage medium | |
CN103453849B (en) | The complex curved surface parts method for three-dimensional measurement that many optical sensors are collaborative and system | |
CN104596443A (en) | Light plane equation fitting locating calibration method based on inherent characteristics of three-line laser | |
Gould | Resolution of the MACHO-LMC-5 Puzzle: the jerk-parallax microlens degeneracy | |
CN100388319C (en) | Multi-viewpoint attitude estimating and self-calibrating method for three-dimensional active vision sensor | |
CN105678076B (en) | The method and device of point cloud measurement data quality evaluation optimization | |
Toschi et al. | On the evaluation of photogrammetric methods for dense 3D surface reconstruction in a metrological context | |
CN101986350A (en) | Monocular structured light-based three-dimensional modeling method | |
CN103258345A (en) | Method for extracting parameters of tree branches based on ground laser radar three-dimensional scanning | |
CN103606147B (en) | Multiple stage is not total to visual field and measures the coordinate system conversion scaling method of camera | |
CN113916130B (en) | Building position measuring method based on least square method | |
CN105550428A (en) | Bridge security evaluation method based on TLS (three-dimensional laser scanning) technique | |
Gronle et al. | View and sensor planning for multi-sensor surface inspection | |
CN105115560A (en) | Non-contact measurement method for cabin capacity | |
CN109211132A (en) | A kind of photogrammetric method for obtaining tall and big object deformation information of unmanned plane high-precision | |
CN107391794B (en) | Typhoon continuous three-dimensional wind field inversion method | |
CN105469391A (en) | Cloud shadow detection method and cloud shadow detection system | |
CN103162620A (en) | Image processing device and image processing method | |
Dvoynishnikov et al. | Optoelectronic method of contactless reconstruction of the surface profile of complexly shaped three-dimensional objects | |
CN104299262A (en) | Three-dimensional cloud simulating method based on speed field flow line | |
Sitnik et al. | Opto-numerical methods of data acquisition for computer graphics and animation systems | |
Shen et al. | The principle analysis of light ray optimization algorithm | |
CN115719363A (en) | Environment sensing method and system capable of performing two-dimensional dynamic detection and three-dimensional reconstruction | |
Hendriatiningsih et al. | 3D Model Based on Terrestrial Laser Scanning (TLS) Case study: The Cangkuang Temple, Garut District, West Java, Indonesia. | |
CN114663600A (en) | Point cloud reconstruction method and system based on self-encoder |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170201 |
|
CF01 | Termination of patent right due to non-payment of annual fee |