CN114463422B - Method and system for image measurement correction - Google Patents
Method and system for image measurement correction Download PDFInfo
- Publication number
- CN114463422B CN114463422B CN202210387332.4A CN202210387332A CN114463422B CN 114463422 B CN114463422 B CN 114463422B CN 202210387332 A CN202210387332 A CN 202210387332A CN 114463422 B CN114463422 B CN 114463422B
- Authority
- CN
- China
- Prior art keywords
- calibration
- error
- coordinate system
- measuring instrument
- image measuring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Abstract
A method and system for image measurement correction, comprising the steps of: s1, placing a calibration plate with a plurality of calibration points distributed in rows and columns on the object carrying table surface of the image measuring instrument, coinciding the central symmetry points of the calibration points with the original point of the coordinate system of the image measuring instrument, and collecting coordinate data of the calibration points on the calibration plate; s2, rotating the calibration plate and translating the calibration plate according to the fact that the central symmetry point of the calibration points is used as a fixed point for multiple times, and collecting coordinate data of the calibration points after the calibration plate is rotated and translated each time; s3, determining error E General assembly Calculating an error factor of the surveying instrument based on a least square method according to the coordinate data of the plurality of calibration points of step S1 and step S2; the data of the calibration point under different states are obtained through multiple times of rotation measurement, and the error of the part belonging to the image measuring instrument is separated from the error through the data processing of the calibration point, so that the subsequent measurement is more accurate.
Description
Technical Field
The present application relates to the field of image measurement technologies, and in particular, to a method and a system for image measurement correction.
Background
The image measurement technology needs to be carried on related hardware equipment, and the purpose of measuring the sizes of the components is achieved through cooperation between the image measurement technology and the hardware equipment. Since the image measurement technology is mounted on the related hardware device, the error compensation value between the obtained component size and the actual size needs to be obtained through correction before the component size is measured by using the image measurement technology, so that the measurement error is avoided.
In the prior art, a correction method of the two-dimensional image measuring instrument is to correct the marking plate for multiple measurements to obtain error factors, but the error factors include the error factors of the two-dimensional image measuring instrument and other error factors, so that other error factors still exist in the subsequent measurement of the dimension of the component, and the error of the dimension measurement of the component is caused.
Disclosure of Invention
Object of the application
In view of the above, an object of the present application is to provide a method and a system for image measurement correction, so as to solve the problem that in the correction method for a two-dimensional image measuring instrument in the prior art, an error factor is obtained by correcting multiple measurements of a marking plate, but the error factor includes the error factor of the two-dimensional image measuring instrument and other error factors, and therefore, other error factors still exist in the subsequent measurement of the component size, which results in an error in the measurement of the component size.
(II) technical scheme
The application discloses a method for image measurement correction, which comprises the following steps:
s1, placing a calibration plate with a plurality of calibration points distributed in rows and columns on the object carrying table surface of the image measuring instrument, coinciding the central symmetry points of the calibration points with the origin of the coordinate system of the image measuring instrument, and collecting the coordinate data of the calibration points on the calibration plate;
s2, rotating the calibration plate and translating the calibration plate for multiple times by taking the central symmetry point of the calibration points as a fixed point, and collecting coordinate data of the calibration points after rotating the calibration plate and translating the calibration plate each time;
s3, determining error E General assembly An error factor of the image measuring instrument is calculated based on a least square method according to the coordinate data of the plurality of calibration points of the step S1 and the step S2.
In a possible implementation manner, the step S2 is specifically:
s21, rotating the calibration board by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 1 Secondly, collecting coordinate data of the calibration points on the calibration plate for the second time;
s22, taking the rotation angle as theta 1 The calibration plate is rotated by the degree and returns to the position state before the step S21, the calibration plate is translated by a preset distance along one coordinate axis direction of the coordinate system of the image measuring instrument, and the coordinate data of the calibration points on the calibration plate are collected for the third time;
s23, rotating the calibration board by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 2 Acquiring coordinate data of the calibration points on the calibration plate for the fourth time;
s24, translating the coordinate system of the image measuring instrument in the direction opposite to the translation distance of one coordinate axis of the coordinate system of the image measuring instrument, and collecting the coordinate data of the calibration points on the calibration plate for the fifth time;
and S25, translating the preset distance along the other coordinate axis direction of the coordinate system of the image measuring instrument, and collecting coordinate data of the calibration point on the calibration plate for the sixth time.
In a possible implementation manner, the step S3 is specifically: s31, determining error E General assembly Composition of said error E General assembly Including a first type of error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the fourth type: deviation error factor E between coordinate system of image measuring instrument and coordinate system of calibration plate 4 (ii) a Wherein E Total = E 1 +E 2 +E 4 (ii) a The deviation comprises distance deviation and angle deviation of the origin of the coordinate system of the image measuring instrument and the coordinate system of the calibration plate;
and S32, constructing a calculation model based on a least square method, establishing model calculation error factors according to the coordinate data of the calibration points, wherein the error factors comprise error factors along the X-axis direction of the coordinate system of the image measuring instrument and error factors along the Y-axis direction of the coordinate system of the image measuring instrument, and calculating the error factors along the X-axis direction of the coordinate system of the image measuring instrument and the error factors along the Y-axis direction of the coordinate system of the image measuring instrument according to the least square method.
In one possible embodiment, the calculation model of the least squares method is:
wherein:
representing the X coordinate value in the coordinate data;
representing the Y coordinate value in the coordinate data;
representing the error of the system error factor in the X-axis direction;
representing the error of the system error factor Y-axis direction;
representing the error of the measurement error factor in the X-axis direction;
representing the error of the measurement error factor Y-axis direction;
a lateral deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate;
a longitudinal deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate; theta represents an included angle between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; i is
An order identity matrix;
is a single column matrix with elements all of 1,
is a matrix
Transposing; o is a zero matrix;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing; the other elements are the nominal scale of the index points,
the number of the index points.
In a possible embodiment, the row spacing of the row-column distribution is equal to the column spacing; the preset distance is equal to the row spacing or the column spacing; the line spacing or the row spacing is 10mm +/-1.1 mm; theta is described 1 Specifically 90 degrees plus or minus 0.6 degrees theta 2 Specifically 180 degrees ± 0.6 degrees.
As a second aspect of the present application, there is also provided a system for image measurement correction, including a calibration board having a plurality of calibration points, a calibration point data acquisition module, and an error factor calculation module; the plurality of calibration points are distributed in a central symmetry manner, the plurality of calibration points are distributed in rows and columns, and the row spacing of the row and column distribution is equal to the column spacing;
the calibration point data acquisition module is used for acquiring coordinate data of the calibration points of the calibration plate in different states; the different states comprise that the central symmetrical points of the plurality of calibration points are coincided with the origin of the coordinate system of the image measuring instrument for the first time, and the calibration plate is rotated and translated for multiple times by taking the central symmetrical points of the plurality of calibration points as fixed points;
the error factor calculation module is used for determining an error E General (1) And calculating an error factor of the image measuring instrument based on a least square method according to the coordinate data of the plurality of calibration points.
In one possible embodiment, the rotating the calibration board and translating the calibration board a plurality of times with a central symmetry point of the plurality of calibration points as a fixed point includes: rotating the calibration plate by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 1 Degree; the rotation angle is theta 1 Rotating the calibration plate by a preset angle to return to a state that the calibration plate is placed on the object carrying table of the image measuring instrument, and translating the calibration plate by a preset distance along one coordinate axis direction of a coordinate system of the image measuring instrument; rotating the calibration plate with the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 2 Degree; translating the preset distance along the direction opposite to the translation distance of one coordinate axis of the coordinate system of the image measuring instrument; and translating the preset distance along the other coordinate axis direction of the coordinate system of the image measuring instrument.
In a possible embodiment, the error E is determined by the method General assembly The method for calculating the error factor of the measuring instrument based on the least square method according to the coordinate data of the plurality of calibration points comprises the following steps: determining an error E General assembly Composition of said error E General assembly Including a first type of error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the third type: noise error factor E 3 (ii) a Error factors of the fourth type: the central symmetry point of the calibration point and the origin of the coordinate system of the image measuring instrument can not be in absolute coincidence, and the transverse direction and the longitudinal direction of the calibration point distributed in a determinant form have certain angular rotation with the coordinate system of the measuring instrument, namely the transverse direction and the longitudinal direction and the coordinate system of the image measuring instrumentHas error factor E of included angle of coordinate axes 4 (ii) a Wherein E Total = E 1 +E 2 +E 3 +E 4 (ii) a For error factor E in error 4 Performing equivalent replacement, the error factor E 4 Replacing the deviation between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate, wherein the deviation comprises the distance deviation and the angle deviation of the origin points of the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; and constructing a calculation model based on a least square method, establishing a model according to the coordinate data of the calibration points to calculate the error factors, wherein the error factors comprise error factors along the X-axis direction of the coordinate system of the image measuring instrument and error factors along the Y-axis direction of the coordinate system of the image measuring instrument, and the error factors along the X-axis direction of the coordinate system of the image measuring instrument and the error factors along the Y-axis direction of the coordinate system of the image measuring instrument are calculated according to the least square method.
In one possible embodiment, the calculation model of the least squares method is:
wherein:
representing the X coordinate value in the coordinate data;
representing the Y coordinate value in the coordinate data;
representing the error of the system error factor in the X-axis direction;
representing the error of the system error factor Y-axis direction;
representative measurementError factor X-axis direction error;
representing the error of the Y-axis direction of the measurement error factor;
a lateral deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate;
a longitudinal deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate; theta represents an included angle between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; i is
An order identity matrix;
is a single column matrix with elements all of 1,
is a matrix
Transposing; o is a zero matrix;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing; the other elements are the nominal scale of the index points,
the number of the index points.
In a possible embodiment, the row spacing of the row-column distribution is equal to the column spacing; the preset distance is equal to the row spacing or the column spacing; the line spacing or the row spacing is 10mm +/-1.1 mm; theta is described 1 Specifically 90 degrees plus or minus 0.6 degrees theta 2 Specifically 180 degrees ± 0.6 degrees.
(III) advantageous effects
Placing a calibration plate with a plurality of calibration points distributed in rows and columns on the object carrying table surface of an image measuring instrument, coinciding the central symmetry points of the calibration points with the original point of the coordinate system of the image measuring instrument, and collecting the coordinate data of the calibration points on the calibration plate; rotating the calibration plate and translating the calibration plate according to the fact that the central symmetry point of the calibration points is used as a fixed point for multiple times, and collecting coordinate data of the calibration points after the calibration plate is rotated and translated each time; the data of the calibration points in different states are obtained through measurement, and the error of the part belonging to the image measuring instrument is separated from the error through the data processing of the calibration points, so that the subsequent measurement is more accurate.
Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objectives and other advantages of the present application may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining and illustrating the present application and should not be construed as limiting the scope of the present application.
FIG. 1 is a system flow diagram of the present application;
FIG. 2 is a diagram of a calibration plate of the present application;
FIG. 3 is a system block diagram of the present application;
wherein: 1. calibrating the plate; 2. a calibration point data acquisition module; 3. an error factor calculation module; 4. an error factor correction module; 11. and (6) calibrating the points.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the above description of the present application, it should be noted that the terms "one side", "the other side", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings or orientations or positional relationships that the products of the application usually place when using, are only used for convenience of description and simplification of description, and do not indicate or imply that the devices or elements that are referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present application. Furthermore, the terms "first," "second," and the like are used merely to distinguish one description from another, and are not to be construed as indicating or implying relative importance.
Further, the term "identical" and the like do not mean that the components are absolutely required to be identical, but may have slight differences. The term "perpendicular" merely means that the positional relationship between the components is more perpendicular than "parallel", and does not mean that the structure must be perfectly perpendicular, but may be slightly inclined.
As shown in fig. 1, the present application discloses a method for image measurement calibration, comprising the following steps: s1, placing a calibration plate with a plurality of calibration points distributed in rows and columns on the object carrying table surface of the image measuring instrument, coinciding the central symmetry points of the calibration points with the original point of the coordinate system of the image measuring instrument, and collecting coordinate data of the calibration points on the calibration plate, wherein the row spacing of the row-column distribution is equal to the column spacing; the line spacing or the row spacing is 10mm +/-1.1 mm. The calibration plate is provided with a coordinate system, the calibration points are uniformly distributed in a first quadrant, a second quadrant, a third quadrant and a fourth quadrant of the coordinate, and the origin of the coordinate system of the calibration plate is the central symmetry point of the calibration points. However, since the central symmetry point of the calibration point cannot be made to be absolutely coincident with the origin of the coordinate system of the image measuring instrument, and the lateral and longitudinal directions of the calibration points distributed in a determinant form a certain angle with the coordinate system of the image measuring instrument, that is, the lateral and longitudinal directions form an angle with the coordinate axes of the coordinate system of the image measuring instrument, the error of the above state should be considered in the calculation. The value of the calibration point is 6 x 6, so that data of a plurality of points can be collected, the error range is reduced, and the calculation pressure caused by excessive calibration points can be avoided. S2, rotating the calibration board and translating the calibration board according to the multiple times of taking the central symmetry point of the calibration points as a fixed point, and collecting the multiple calibration boards after rotating the calibration board and translating the calibration board each timeAnd the coordinate data of the calibration points are acquired after the calibration plate is rotated and translated every time, the brightness and the exposure value of the light source are changed while the coordinate data of the calibration points are acquired, and data are acquired simultaneously. The rotating the calibration plate and translating the calibration plate according to the multiple times of taking the central symmetry point of the calibration points as a fixed point, and collecting the coordinate data of the calibration points after rotating the calibration plate and translating the calibration plate each time specifically comprises: s21, rotating the calibration board by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 1 And secondly collecting coordinate data of the plurality of calibration points on the calibration plate, wherein theta is 1 Specifically 90 degrees plus or minus 0.6 degrees; s22, taking the rotation angle as theta 1 The degree-rotation scale plate returns to the position state before step S21, θ 1 Specifically 90 degrees plus or minus 0.6 degrees; translating the calibration board by a preset distance along one coordinate axis direction of a coordinate system of the image measuring instrument, wherein the preset distance is equal to the line spacing or the row spacing of the calibration points, the line spacing or the row spacing is 10mm +/-1.1 mm, and collecting coordinate data of the calibration points on the calibration board for the third time, the preset distance is equal to the line spacing or the row spacing of the calibration points, and the line spacing or the row spacing is 10mm +/-1.1 mm; s23, rotating the calibration board by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 2 Acquiring coordinate data of the calibration points on the calibration plate for the fourth time; theta 2 Specifically 180 degrees plus or minus 0.6 degrees; s24, translating the image measuring instrument along the direction opposite to the translation distance of one coordinate axis of the coordinate system of the image measuring instrument, wherein the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, the line spacing or the row spacing is 10mm +/-1.1 mm, and collecting the coordinate data of the plurality of calibration points on the calibration plate for the fifth time; and S25, translating the image measuring instrument along the other coordinate axis direction of the coordinate system of the image measuring instrument by the preset distance, wherein the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, the line spacing or the row spacing is 10mm +/-1.1 mm, and collecting coordinate data of the calibration points on the calibration plate for the sixth time.
At this timeError E General assembly Can be divided into four types of error factors which act together and are respectively: first type error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the third type: noise error factor E 3 (ii) a Error factors of the fourth type: the central symmetry point of the calibration point and the origin of the coordinate system of the image measuring instrument can not be in absolute coincidence, and the transverse direction and the longitudinal direction of the calibration point distributed in a determinant form have certain angular rotation with the coordinate system of the image measuring instrument, namely, an error factor E of an included angle exists between the transverse direction and the longitudinal direction and the coordinate axis of the coordinate system of the image measuring instrument 4 (ii) a Wherein E Total = E 1 +E 2 +E 3 +E 4 . S3, determining error composition, and calculating error factors of the image measuring instrument based on a least square method according to the coordinate data of the plurality of calibration points in the steps S1 and S2; the method comprises the following specific steps: s31, error E General assembly Can be divided into four types of error factors which act together and are respectively: first type error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the third type: noise error factor E 3 (ii) a Error factors of the fourth type: the central symmetry point of the calibration point and the origin of the coordinate system of the image measuring instrument can not be in absolute coincidence, and the transverse direction and the longitudinal direction of the calibration point distributed in a determinant form have certain angular rotation with the coordinate system of the image measuring instrument, namely, an error factor E of an included angle exists between the transverse direction and the longitudinal direction and the coordinate axis of the coordinate system of the image measuring instrument 4 (ii) a Wherein E Total = E 1 +E 2 +E 3 +E 4 . For error factor E in error 4 Performing equivalent replacement, wherein a deviation exists between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate, the deviation comprises a distance deviation and an angle deviation of the origins of the coordinate systems of the image measuring instrument and the calibration plate, the distance deviation is the distance between the coordinate origins, the angle deviation is the included angle theta between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate, and the distance deviation comprises a transverse deviation and a longitudinal deviation; representing the error factor E by the amount of deviation 4 Said error factor E 4 Is the center of the index pointThe symmetrical point and the origin of the coordinate system of the image measuring instrument can not be absolutely coincident, and the transverse direction and the longitudinal direction of the calibration points distributed in a determinant form have certain angles with the coordinate system of the image measuring instrument, namely, the transverse direction and the longitudinal direction have errors of included angles with the coordinate axes of the coordinate system of the image measuring instrument.
S32, constructing a calculation model based on a least square method, establishing model calculation error factors according to the coordinate data of the plurality of calibration points, where the error factors include an error factor along the X-axis direction of the coordinate system of the image measuring instrument and an error factor along the Y-axis direction of the coordinate system of the image measuring instrument, and calculating the error factor along the X-axis direction of the coordinate system of the image measuring instrument and the error factor along the Y-axis direction of the coordinate system of the image measuring instrument according to the least square method, where the calculation model of the least square method is:
wherein:
representing the X coordinate value in the coordinate data;
representing the Y coordinate value in the coordinate data;
representing the error of the system error factor in the X-axis direction;
representing the error of the Y-axis direction of the system error factor;
representing the error of the measurement error factor in the X-axis direction;
representing a measurement error factor YErrors in the axial direction;
a lateral deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate;
a longitudinal deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate; theta represents an included angle between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; i is
An order identity matrix;
is a single column matrix with elements all of 1,
is a matrix
Transposing; o is a zero matrix;
is composed of
A column of a single-row matrix is formed,
is a matrix
Transposing;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing; the other elements are the nominal scale of the index points,
the number of the index points.
The data measured by the calculation model through the steps S22-S25 can be calculated without considering the noise error factor E 3 . To enable the solution model to have a low dispersion, θ is described 1 Specifically 90 degrees plus or minus 0.6 degrees; and translating the calibration board along one coordinate axis direction of a coordinate system of the image measuring instrument for a preset distance, wherein the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, the line spacing or the row spacing is 10mm +/-1.1 mm, and collecting coordinate data of the plurality of calibration points on the calibration board for the third time, the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, and the line spacing or the row spacing is 10mm +/-1.1 mm. The value of the calibration point is 6 x 6, so that data of a plurality of points can be collected, the error range is reduced, and the calculation pressure caused by excessive calibration points can be avoided.
The data of the calibration point under different states are obtained through multiple times of rotation measurement, and the error of the part belonging to the image measuring instrument is separated from the error through the data processing of the calibration point, so that the subsequent measurement is more accurate.
S4, collecting error factors calculated each time to establish an error factor library, carrying out distance measurement on the collected error factors, establishing distance distribution, establishing an effective distance threshold value through analyzing the distance distribution, searching K adjacent points of the newly collected once error factors through the error factor library based on the distance threshold value when the once error factors are newly collected, obtaining the newly collected once error factor near-sight value as a new error factor through weighting the K adjacent points, and endowing the new error factor to the image measuring instrument. For each extractionThe error factors of the set are subjected to K nearest neighbor processing, so that the system error factor E can be further reduced 1 . Whether the newly acquired system error factors are errors caused by misoperation, such as the fact that angles and translations do not meet the range value requirements in the steps S1-S3 can be determined through the distance threshold, and when errors caused by the fact that the misoperation, such as the fact that angles and translations do not meet the range value requirements in the steps S1-S3 exist, judgment can be carried out and fed back to a corrector, specifically: when the newly acquired error factors cannot find the adjacent points within the range of the distance threshold value, the judgment is changed into the correction of the error with misoperation, or other problems of system hardware occur, such as the error caused by hardware aging.
As a second aspect of the present application, there is also provided a system for image measurement correction, including a calibration board 1 having a plurality of calibration points 11, a calibration point data acquisition module 2, and an error factor calculation module 3. The plurality of calibration points are distributed in a central symmetry manner, the plurality of calibration points are distributed in rows and columns, the row spacing of the row and column distribution is equal to the column spacing, and the row spacing of the row and column distribution is equal to the column spacing; the line spacing or the row spacing is 10mm +/-1.1 mm. The calibration plate is provided with a coordinate system, the calibration points are uniformly distributed in a first quadrant, a second quadrant, a third quadrant and a fourth quadrant of the coordinate, and the origin of the coordinate system of the calibration plate is the central symmetry point of the calibration points. However, since the central symmetry point of the calibration point cannot be absolutely coincident with the origin of the coordinate system of the image measuring instrument, and the lateral and longitudinal directions of the calibration points distributed in a determinant form a certain angular rotation with the coordinate system of the image measuring instrument, that is, the lateral and longitudinal directions form an angle with the coordinate axes of the coordinate system of the image measuring instrument, the error of the above state should be considered in the calculation. The value of the calibration point is 6 x 6, so that data of a plurality of points can be collected, the error range can be reduced, and the calculation pressure caused by excessive calibration points can be avoided.
The calibration point data acquisition module is used for acquiring coordinate data of the calibration points of the calibration plate in different states; rotating the calibration plate and translating the calibration plate for multiple times by taking the central symmetry point of the calibration points as a fixed pointThe calibration plate, the calibration point data acquisition module acquires the coordinate data of the plurality of calibration points after rotating and translating the calibration plate each time, and changes the brightness and exposure value of the light source while acquiring the coordinate data of the plurality of calibration points after rotating and translating the calibration plate each time, and acquires data: rotating the calibration plate and translating the calibration plate according to the fact that the central symmetry point of the calibration points is used as a fixed point for multiple times, and collecting coordinate data of the calibration points after the calibration plate is rotated and translated every time, specifically: the central symmetry point of the calibration points is a fixed point for rotating the calibration plate and translating the calibration plate, and the calibration point data acquisition module acquires the coordinate data of the calibration points after rotating the calibration plate and translating the calibration plate at each time specifically comprises the following steps: the calibration plate is placed on the object carrying table surface of the image measuring instrument, the central symmetry point of the calibration points is superposed with the origin of the coordinate system of the image measuring instrument, and the coordinate data of the calibration points on the calibration plate are collected; rotating the calibration plate by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 1 Degree, said theta 1 Specifically 90 degrees plus or minus 0.6 degrees; secondly, collecting coordinate data of the calibration points on the calibration plate; the rotation angle is theta 1 Rotating the calibration plate to return to the state that the calibration plate is placed on the object carrying table of the image measuring instrument, translating the calibration plate along one coordinate axis direction of a coordinate system of the image measuring instrument for a preset distance, and collecting coordinate data of the calibration points on the calibration plate for the third time, wherein the theta is 1 Specifically 90 degrees +/-0.6 degrees, wherein the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, and the line spacing or the row spacing is 10mm +/-1.1 mm; rotating the calibration plate with the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 2 And collecting coordinate data theta of the calibration points on the calibration plate for the fourth time 2 Specifically 180 degrees plus or minus 0.6 degrees; translating the preset distance along the direction opposite to the translation distance of one coordinate axis of the coordinate system of the image measuring instrument, wherein the preset distance is equal to more thanThe line spacing or the column spacing of each calibration point is 10mm +/-1.1 mm, and the coordinate data of the calibration points on the calibration plate are collected for the fifth time; and translating the preset distance along the other coordinate axis direction of the coordinate system of the image measuring instrument, wherein the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, the line spacing or the row spacing is 10mm +/-1.1 mm, and collecting coordinate data of the calibration points on the calibration plate for the sixth time.
And error E at this time General assembly Can be divided into four types of error factors which are respectively: first type error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the third type: noise error factor E 3 (ii) a Error factors of the fourth type: the central symmetry point of the calibration point and the origin of the coordinate system of the image measuring instrument can not be in absolute coincidence, and the transverse direction and the longitudinal direction of the calibration point distributed in a determinant form have certain angular rotation with the coordinate system of the image measuring instrument, namely, an error factor E of an included angle exists between the transverse direction and the longitudinal direction and the coordinate axis of the coordinate system of the image measuring instrument 4 (ii) a Wherein E Total = E 1 +E 2 +E 3 +E 4 。
The error factor calculation module is used for calculating the error factor of the image measuring instrument based on a least square method according to the collected coordinate data of the calibration points. The error factor calculation module establishes a calculation model of a least square method according to the constitution of errors, wherein the errors can be divided into four types of error factors which are respectively: first type error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the third type: noise error factor E 3 (ii) a Error factors of the fourth type: the central symmetry point of the calibration point and the origin of the coordinate system of the image measuring instrument can not be in absolute coincidence, and the transverse direction and the longitudinal direction of the calibration point distributed in a determinant form have certain angular rotation with the coordinate system of the image measuring instrument, namely, an error factor E of an included angle exists between the transverse direction and the longitudinal direction and the coordinate axis of the coordinate system of the image measuring instrument 4 (ii) a Wherein E Total = E 1 +E 2 +E 3 +E 4 . The error factor calculation module is used for calculating an error factor E 4 Performing equivalent replacement, wherein a deviation amount exists between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate, the deviation amount comprises a distance deviation and an angle deviation of the coordinate system of the image measuring instrument and the origin of the coordinate system of the calibration plate, the distance deviation is the distance between the origin of the coordinates, the angle deviation is the included angle theta between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate, and the distance deviation comprises a transverse deviation and a longitudinal deviation; representing the error factor E by the amount of deviation 4 Said error factor E 4 The central symmetry point of the calibration point and the origin of the coordinate system of the image measuring instrument can not be absolutely coincident, and the transverse direction and the longitudinal direction of the calibration point distributed in a determinant form have certain angular rotation with the coordinate system of the image measuring instrument, namely, the transverse direction and the longitudinal direction have included angle errors with the coordinate axis of the coordinate system of the image measuring instrument. Modeling and calculating the error factors according to the coordinate data of the plurality of calibration points, wherein the error factors comprise an error factor along the X-axis direction of the coordinate system of the image measuring instrument and an error factor along the Y-axis direction of the coordinate system of the image measuring instrument, and the error factor along the X-axis direction of the coordinate system of the image measuring instrument and the error factor along the Y-axis direction of the coordinate system of the image measuring instrument are calculated according to a least square method, and the calculation model of the least square method is as follows:
wherein:
representing the X coordinate value in the coordinate data;
representing the Y coordinate value in the coordinate data;
representing the error of the system error factor in the X-axis direction;
representing the error of the system error factor Y-axis direction;
representing the error of the measurement error factor in the X-axis direction;
representing the error of the measurement error factor Y-axis direction;
a lateral deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate;
a longitudinal deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate; theta represents an included angle between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; i is
An order identity matrix;
is a single column matrix with elements all of 1,
is a matrix
Transposing; o is a zero matrix;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing; the other elements are the nominal scale of the index points,
the number of the index points.
The data measured by the calculation model through the steps S22-S25 can be calculated without considering the noise error factor E 3 . To enable the solution model to have a low dispersion, θ is described 1 Specifically 90 degrees plus or minus 0.6 degrees; and translating the calibration board along one coordinate axis direction of a coordinate system of the image measuring instrument for a preset distance, wherein the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, the line spacing or the row spacing is 10mm +/-1.1 mm, and collecting coordinate data of the plurality of calibration points on the calibration board for the third time, the preset distance is equal to the line spacing or the row spacing of the plurality of calibration points, and the line spacing or the row spacing is 10mm +/-1.1 mm. The value of the calibration point is 6 x 6, so that data of a plurality of points can be collected, the error range is reduced, and the calculation pressure caused by excessive calibration points can be avoided.
The system for correcting the image measuring instrument further comprises an error factor correction module 5, wherein the error factor correction module is used for collecting error factors calculated each time to establish an error factor library, measuring the distance between the collected error factors, establishing the distribution of the distance, and calculating the error factors according to the distance between the collected error factorsAnd (2) establishing an effective distance threshold value through distance distribution analysis, searching K adjacent points of the newly acquired primary error factor based on the distance threshold value through an error factor library when the primary error factor is newly acquired, obtaining a near-looking value of the newly acquired primary error factor as a new error factor through weighting the K adjacent points, and endowing the new error factor to the image measuring instrument. The K nearest neighbor processing is carried out on the error factors collected each time, and the system error factor E can be further reduced 1 . Whether the newly acquired system error factors are errors caused by misoperation, such as the fact that angles and translations do not meet the range value requirements in the steps S1-S3 can be determined through the distance threshold, and when errors caused by the fact that the misoperation, such as the fact that angles and translations do not meet the range value requirements in the steps S1-S3 exist, judgment can be carried out and fed back to a corrector, specifically: when the newly acquired error factors cannot find the adjacent points within the range of the distance threshold value, the judgment is changed to correct the error with misoperation, or other problems of system hardware, such as the error caused by hardware aging, occur.
Finally, the above embodiments are only used for illustrating the technical solutions of the present application and not for limiting, although the present application is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made on the technical solutions of the present application without departing from the spirit and scope of the technical solutions of the present application, and all the technical solutions of the present application should be covered by the claims of the present application.
Claims (8)
1. A method for image measurement calibration, comprising the steps of: s1, placing a calibration plate with a plurality of calibration points distributed in rows and columns on the object carrying table surface of the image measuring instrument, coinciding the central symmetry points of the calibration points with the origin of the coordinate system of the image measuring instrument, and collecting the coordinate data of the calibration points on the calibration plate; s2, rotating the calibration board and translating the calibration board for multiple times by taking the central symmetry point of the calibration points as a fixed point, and collecting the calibration points after rotating the calibration board and translating the calibration board each timeCoordinate data; s3, determining error E General assembly Calculating an error factor of the image measuring instrument based on a least square method from the coordinate data of the plurality of calibration points of step S1 and step S2; the step S3 specifically includes: s31, determining error E General assembly Composition of said error E General assembly Including a first type of error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the fourth type: deviation error factor E between coordinate system of image measuring instrument and coordinate system of calibration plate 4 (ii) a Wherein E Total = E 1 +E 2 +E 4 (ii) a The deviation comprises distance deviation and angle deviation of the origin of the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; and S32, constructing a calculation model based on a least square method, establishing model calculation error factors according to the coordinate data of the calibration points, wherein the error factors comprise error factors along the X-axis direction of the coordinate system of the image measuring instrument and error factors along the Y-axis direction of the coordinate system of the image measuring instrument, and calculating the error factors along the X-axis direction of the coordinate system of the image measuring instrument and the error factors along the Y-axis direction of the coordinate system of the image measuring instrument according to the least square method.
2. The method according to claim 1, wherein the step S2 includes: s21, rotating the calibration board by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 1 Secondly, collecting coordinate data of the calibration points on the calibration plate for the second time; s22, taking the rotation angle as theta 1 The calibration plate is rotated by the degree and returns to the position state before the step S21, the calibration plate is translated by a preset distance along one coordinate axis direction of the coordinate system of the image measuring instrument, and the coordinate data of the calibration points on the calibration plate are collected for the third time; s23, rotating the calibration board by taking the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 2 Acquiring coordinate data of the calibration points on the calibration plate for the fourth time; s24, along the coordinate system of the image measuring instrumentTranslating the preset distance in the direction opposite to the translation distance of the coordinate axis direction, and collecting the coordinate data of the calibration points on the calibration plate for the fifth time; and S25, translating the preset distance along the other coordinate axis direction of the coordinate system of the image measuring instrument, and collecting coordinate data of the calibration point on the calibration plate for the sixth time.
3. The method of claim 2, wherein the least squares calculation model is:
wherein:
representing the X coordinate value in the coordinate data;
representing the Y coordinate value in the coordinate data;
representing the error of the system error factor in the X-axis direction;
representing the error of the system error factor Y-axis direction;
representing the error of the measurement error factor in the X-axis direction;
representing the error of the Y-axis direction of the measurement error factor;
representative imageA lateral deviation of the distance deviation between the coordinate system of the measuring instrument and the coordinate system of the calibration plate;
a longitudinal deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate; theta represents an included angle between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; i is
An order identity matrix;
is a single column matrix with elements all of 1,
is a matrix
Transposing; o is a zero matrix;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing; the other elements are the nominal scale of the index points,
the number of the index points.
4. The method of claim 3, wherein the row-column spacing is equal to the column-column spacing; the preset distance is equal to the row spacing or the column spacing; the line spacing or the row spacing is 10mm +/-1.1 mm; theta is described 1 Specifically 90 degrees plus or minus 0.6 degrees theta 2 Specifically 180 degrees ± 0.6 degrees.
5. A system for image measurement correction is characterized by comprising a calibration plate with a plurality of calibration points, a calibration point data acquisition module and an error factor calculation module; the plurality of calibration points are distributed in a central symmetry manner, the plurality of calibration points are distributed in rows and columns, and the row spacing of the row and column distribution is equal to the column spacing;
the calibration point data acquisition module is used for acquiring coordinate data of the calibration points of the calibration plate in different states; the different states comprise that the central symmetrical points of the plurality of calibration points are coincided with the origin of the coordinate system of the image measuring instrument for the first time, and the calibration plate is rotated and translated for multiple times by taking the central symmetrical points of the plurality of calibration points as fixed points;
the error factor calculation module is used for determining an error E General assembly Forming and calculating an error factor of the image measuring instrument based on a least square method according to the coordinate data of the plurality of calibration points; the method specifically comprises the following steps: determining an error E General assembly Composition of said error E General assembly Including a first type of error factor: systematic error factor E 1 (ii) a Second type error factor: measuring error factor E 2 (ii) a Error factors of the fourth type: the central symmetry point of the calibration point and the origin of the coordinate system of the image measuring instrument can not be in absolute coincidence, and the transverse direction and the longitudinal direction of the calibration point distributed in a determinant form have certain angular rotation with the coordinate system of the image measuring instrument, namely, an error factor E of an included angle exists between the transverse direction and the longitudinal direction and the coordinate axis of the coordinate system of the image measuring instrument 4 (ii) a Wherein E Total = E 1 +E 2 +E 4 (ii) a For error factor E in error 4 Performing equivalent replacement, the error factor E 4 Replacing the deviation between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate, wherein the deviation comprises the distance deviation and the angle deviation of the origin points of the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; and constructing a calculation model based on a least square method, establishing model calculation error factors according to the coordinate data of the calibration points, wherein the error factors comprise error factors along the X-axis direction of the coordinate system of the image measuring instrument and error factors along the Y-axis direction of the coordinate system of the image measuring instrument, and calculating the error factors along the X-axis direction of the coordinate system of the image measuring instrument and the error factors along the Y-axis direction of the coordinate system of the image measuring instrument according to the least square method.
6. The system of claim 5, wherein the plurality of rotations of the calibration plate and the translation of the calibration plate with a point of central symmetry of the plurality of calibration points as a fixed point comprises: rotating the calibration plate with the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 1 Degree; the rotation angle is theta 1 Rotating the calibration plate by a preset angle to return to a state that the calibration plate is placed on the object carrying table of the image measuring instrument, and translating the calibration plate by a preset distance along one coordinate axis direction of a coordinate system of the image measuring instrument; rotating the calibration plate with the central symmetry point of the calibration points as a fixed point, wherein the rotation angle is theta 2 Degree; translating the preset distance along the direction opposite to the translation distance of one coordinate axis of the coordinate system of the image measuring instrument; another along the coordinate system of the image measuring instrumentAnd translating the coordinate axis direction by the preset distance.
7. The system of claim 6, wherein the least squares calculation model is:
wherein:
representing the X coordinate value in the coordinate data;
representing the Y coordinate value in the coordinate data;
representing the error of the system error factor in the X-axis direction;
representing the error of the system error factor Y-axis direction;
representing the error of the measurement error factor in the X-axis direction;
representing the error of the measurement error factor Y-axis direction;
a lateral deviation representing a distance deviation between a coordinate system of the image measuring instrument and a coordinate system of the calibration plate;
representative image measuring instrumentA longitudinal deviation of the distance deviation between the coordinate system of (a) and the coordinate system of the calibration plate; theta represents an included angle between the coordinate system of the image measuring instrument and the coordinate system of the calibration plate; i is
An order identity matrix;
is a single column matrix with elements all of 1,
is a matrix
Transposing; o is a zero matrix;
is composed of
A column of a single-row matrix is provided,
is a matrix
Transposing;
is composed of
A column of a single-row matrix is formed,
is a matrix
Transposing; the other elements are the nominal scale of the index points,
the number of the index points.
8. The system of claim 7, wherein the row-column spacing is equal to the column-column spacing; the preset distance is equal to the row spacing or the column spacing; the line spacing or the row spacing is 10mm +/-1.1 mm; theta is described 1 Specifically 90 degrees plus or minus 0.6 degrees theta 2 Specifically 180 degrees ± 0.6 degrees.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210387332.4A CN114463422B (en) | 2022-04-14 | 2022-04-14 | Method and system for image measurement correction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210387332.4A CN114463422B (en) | 2022-04-14 | 2022-04-14 | Method and system for image measurement correction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114463422A CN114463422A (en) | 2022-05-10 |
| CN114463422B true CN114463422B (en) | 2022-08-16 |
Family
ID=81418417
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210387332.4A Active CN114463422B (en) | 2022-04-14 | 2022-04-14 | Method and system for image measurement correction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463422B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117190919B (en) * | 2023-11-07 | 2024-02-13 | 湖南科天健光电技术有限公司 | Pipeline straightness measuring method and measuring system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107462184A (en) * | 2017-08-15 | 2017-12-12 | 东南大学 | The parameter recalibration method and its equipment of a kind of structured light three-dimensional measurement system |
| CN111174697A (en) * | 2019-12-13 | 2020-05-19 | 中国南方电网有限责任公司超高压输电公司柳州局 | Stereoscopic vision image accurate measurement method based on unmanned aerial vehicle |
| CN113160331A (en) * | 2021-04-20 | 2021-07-23 | 中国科学院空天信息创新研究院 | External parameter calibration method based on vision system imaging |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108489395B (en) * | 2018-04-27 | 2019-03-22 | 中国农业大学 | Vision measurement system structural parameters calibration and affine coordinate system construction method and system |
-
2022
- 2022-04-14 CN CN202210387332.4A patent/CN114463422B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107462184A (en) * | 2017-08-15 | 2017-12-12 | 东南大学 | The parameter recalibration method and its equipment of a kind of structured light three-dimensional measurement system |
| CN111174697A (en) * | 2019-12-13 | 2020-05-19 | 中国南方电网有限责任公司超高压输电公司柳州局 | Stereoscopic vision image accurate measurement method based on unmanned aerial vehicle |
| CN113160331A (en) * | 2021-04-20 | 2021-07-23 | 中国科学院空天信息创新研究院 | External parameter calibration method based on vision system imaging |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114463422A (en) | 2022-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109556515B (en) | A kind of systematic error calibration method, system and equipment based on machine vision | |
| US20130194569A1 (en) | Substrate inspection method | |
| CN114463422B (en) | Method and system for image measurement correction | |
| CN112415493A (en) | Three-dimensional scanning laser radar coordinate error correction method | |
| CN109073351B (en) | Flatness measuring method and pin height adjusting method | |
| CN116045851B (en) | Line laser profiler calibration method and device, electronic equipment and storage medium | |
| CN112059413A (en) | Laser galvanometer correction method and device, computer equipment and storage medium | |
| JPH02165699A (en) | Mounting method of flat package type ic using industrial robot | |
| CN111709999A (en) | Calibration plate, camera calibration method and device, electronic equipment and camera system | |
| CN114235351B (en) | Method, system and related equipment for detecting laser spot offset in laser array | |
| CN115666125A (en) | Method for detecting and compensating positioning error of XY platform of chip mounter based on machine vision | |
| JP5166916B2 (en) | Apparatus for superimposing patterns and device manufacturing method | |
| US20180307134A1 (en) | Method Of Using Equipment Method And System For Manufacturing Mask Or Display Substrate | |
| Qiao et al. | Self-calibration for the 2D stage based on weighted least squares | |
| CN112894154B (en) | Laser marking method and device | |
| Andreazza | Survey of the ATLAS pixel detector components | |
| CN109141385B (en) | Positioning method of total station instrument without leveling | |
| US20240009760A1 (en) | Processing device and method | |
| CN115100131B (en) | High-precision visual alignment method and system | |
| CN110440693B (en) | Quasi-optical feed network system and error testing method | |
| CN109579805B (en) | Baseline self-calibration measuring method | |
| CN115127493B (en) | Coordinate calibration method and device for product measurement | |
| CN113359399B (en) | Exposure method and exposure system | |
| Woźniak et al. | MarQR technology for measuring relative displacements of building structure elements with regard to joints and cracks | |
| CN111833401B (en) | Rapid ranging method and system based on double-sided point cloud information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |