CN112161574B - Three-dimensional measurement system and measurement method based on divergent multi-line laser projection - Google Patents
Three-dimensional measurement system and measurement method based on divergent multi-line laser projection Download PDFInfo
- Publication number
- CN112161574B CN112161574B CN202011086813.9A CN202011086813A CN112161574B CN 112161574 B CN112161574 B CN 112161574B CN 202011086813 A CN202011086813 A CN 202011086813A CN 112161574 B CN112161574 B CN 112161574B
- Authority
- CN
- China
- Prior art keywords
- line
- calibration
- industrial camera
- stripe
- laser
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2504—Calibration devices
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/254—Projection of a pattern, viewing through a pattern, e.g. moiré
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2545—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with one projection direction and several detection directions, e.g. stereo
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a three-dimensional measurement system and a three-dimensional measurement method based on divergent multi-line laser projection, and belongs to the technical field of computer vision measurement. The measuring system comprises a divergent multi-line laser, an industrial camera, a self-made calibration device, a computer, a workbench and a support, wherein the self-made calibration device comprises a circular calibration plate, a white plate, a rubber magnetic shell and a metal flat plate. The divergent multi-line laser and the industrial camera are installed on the support, the divergent multi-line laser projects vertically, and the industrial camera shoots obliquely. The measuring method comprises the following steps: in the off-site system pre-calibration stage, the calibration of the industrial camera is accurately finished by self-made calibration equipment and a Zhang's plane calibration method, and the calibration of the optical plane is finished based on a Steger algorithm and a RANSAC algorithm; in the field measurement stage, the extraction of the light stripe center line under the condition of poor measurement environment is quickly realized by adopting a multi-line laser center line extraction method based on system space structure constraint, and the three-dimensional measurement is quickly realized by combining with a light plane parameter equation obtained by system calibration.
Description
Technical Field
The invention relates to a three-dimensional measurement system and a measurement method based on divergent multi-line laser projection, in particular to a non-contact three-dimensional measurement method based on a laser triangulation method, and belongs to the technical field of computer vision measurement.
Background
The traditional two-dimensional machine vision technology is difficult to meet the requirements of modern intelligent manufacturing, and the three-dimensional vision technology becomes a development trend of the manufacturing industry. The three-dimensional vision system utilizes a non-contact optical imaging technology to identify the spatial position and the shape of an object to be measured, has the characteristics of high measurement precision, high speed, low cost, convenient and flexible application and the like, and a line laser scanning method is a non-contact active optical three-dimensional measurement method based on a laser triangulation principle. The three-dimensional reconstruction and measurement of the object surface can be realized by scanning the object surface by the projection line laser, the production efficiency of an enterprise is improved, the product quality can be ensured and even improved, and the method has important significance for industrial automatic production. The system calibration and the extraction of the central line of the light strip are the key of the three-dimensional measurement precision, and the quality of the three-dimensional reconstruction of the surface of an object is directly influenced.
At present, a two-dimensional plane calibration plate is mostly adopted for system calibration, the manufacturing is simple, the use is convenient, the application range is wide, and a three-dimensional calibration plate is difficult to manufacture, expensive, limited by a measurement view field and difficult to apply. The traditional industrial camera calibration and the light plane calibration are mostly completed by one checkerboard calibration plate, and black and white checkered patterns on the calibration plate can interfere laser lines and are not beneficial to subsequent light stripe center line extraction. The invention uses two calibration plates to separate the calibration of the industrial camera from the calibration of the light plane, wherein the circular calibration plate is used for the calibration of the industrial camera, and the calibration of the light plane needs to be combined with the circular calibration plate and the white plate. Compared with a chessboard calibration plate, the circular calibration plate has higher calibration precision, and the surface of the white board has no pattern, thereby being convenient for subsequent extraction of the central line of the light striation.
At present, the extraction methods of the light strip center lines mainly include a geometric center method, a threshold value method, an extreme value method, a Steger method, a gray scale gravity center method and the like, but the extraction of the light strip center lines is greatly influenced by the environment of a measuring field, and the extraction precision, the extraction speed and the anti-interference performance of each method are good and bad.
Disclosure of Invention
The invention relates to a three-dimensional measurement system and a three-dimensional measurement method based on divergent multi-line laser projection, which can realize the rapid, accurate and stable extraction of a light stripe center line under the condition of poor measurement environment by adopting a multi-line laser stripe center line extraction method based on system space structure constraint, and can realize the rapid three-dimensional measurement by combining a light plane parameter equation obtained by system calibration. The method is used for meeting the requirements of the manufacturing industry in the aspect of vision measurement, and effectively avoiding the condition that the detection is unreliable due to the fact that the two-dimensional machine vision is influenced by external factors.
The technical scheme adopted by the invention is as follows: a three-dimensional measuring system based on divergent multi-line laser projection comprises a divergent multi-line laser 1, an industrial camera 2, a self-made calibration device, a computer 3, a workbench 4 and a support 5. Wherein, the self-control equipment of maring includes circular calibration board 6, blank 7, rubber magnetism shell 8 and metal flat board 9, and it fixes on support 5 to disperse type multi-line laser 1 and industry camera 2, and 1 vertical projection of dispersion type multi-line laser, 2 slope shots of industry camera and links to each other with computer 3. In the off-site system pre-calibration stage, a metal flat plate 9 needs to be placed on the workbench 4, the circular calibration plate 6 and the white plate 7 are respectively inserted into the rubber magnetic shell 8, and the rubber magnetic shell 8 is placed on the metal flat plate 9. In the field measurement stage, the object to be measured needs to be placed on the workbench 4.
Specifically, the circular calibration board 6 and the white board 7 are used for system calibration (industrial camera 2 calibration and light plane calibration). The circular calibration plate 6 is used for calibrating the industrial camera 2, and the light plane calibration needs to be combined with the circular calibration plate 6 and the white board 7. The circular calibration plate 6 and the white plate 7 are each an opaque rectangular flat plate made of a hard PVC material. The hard PVC board has the characteristics of high strength, high hardness, flatness and difficult deformation. The surface of the circular calibration plate 6 is attached with a white black circle pattern, the pattern is a 7 multiplied by 7 array of black circles, the radius of the circle is 5mm, and the distance between the centers of the circles is 13 mm. The external dimension of the circular calibration plate 6 is 340 multiplied by 260mm, and the thickness is 3 mm. The white board 7 has a white surface without any pattern. The white board 7 has the external dimension of 340 multiplied by 260mm and the thickness of 3 mm.
Specifically, the flat metal plate 9 is used to ensure that the position where the calibration plate is placed remains unchanged. The metal flat plate 9 is made of martensite type stainless steel, and has magnetic permeability, the external dimension is 400 multiplied by 400mm, and the thickness is 5 mm.
Specifically, the rubber magnet housing 8 is used to ensure that the circular calibration plate 6 and the white plate 7 do not slide when an image is captured. The rubber magnet is a magnet with low hardness and good plasticity, is widely applied to refrigerator door seals, water dispenser door seals, magnetic blackboards and the like, has moderate adsorption force, and can be adsorbed on the surface of the metal flat plate 9. The side of the rubber magnetic shell 8 is provided with a slot for inserting the circular calibration plate 6 and the white plate 7 into the rubber magnetic shell 8, and the cross section of the slot is 261 multiplied by 4 mm. When the off-site system is pre-calibrated, the circular calibration plate 6 and the white plate 7 need to be respectively inserted into the rubber magnetic shell 8 and placed on the metal flat plate 9.
A measuring method of a three-dimensional measuring system based on divergent multi-line laser projection comprises the following steps:
step 1: firstly, the divergent multi-line laser 1 and the industrial camera 2 are kept fixed at proper positions of a bracket 5, then the metal flat plate 9 is placed in a field of view of the industrial camera 2, the number of the positions of the metal flat plate 9 is changed (N is more than or equal to 10), and two images are required to be acquired at each position. The image 1 is an image acquired by placing a rubber magnet housing 8, into which a circular calibration plate 6 is inserted, on a metal flat plate 9 under an illumination condition; the image 2 is an image captured under no-lighting conditions by placing the rubber magnet housing 8 with the whiteboard 7 inserted on the metal plate 9 and projecting a laser line onto the whiteboard.
Step 2: performing edge detection on the feature circle in the image 1 to obtain the center coordinate information of the feature circle, and calibrating the industrial camera 2 based on a Zhangyingyou plane calibration method to obtain internal and external parameters of the industrial camera 2;
and step 3: extracting light stripe center lines of the same laser line on the multiple images 2 based on a Steger multi-line laser center line extraction method;
and 4, step 4: converting the extracted center line pixel coordinates into three-dimensional coordinates under an industrial camera 2 coordinate system;
and 5: based on a random sample consensus (RANSAC) algorithm, performing light plane fitting on the three-dimensional coordinates of the central line, thereby obtaining an equation of a light plane and completing the calibration of the light plane;
and 5: when an object is placed in a severe environment for field measurement, the object is required to be placed on a workbench 4, laser lines are projected to the surface of the object and images are collected, and central line extraction is respectively carried out on light stripes on the surface of the object and light stripes on the surface of the workbench 4 by adopting a multi-line laser stripe central line extraction method based on system space structure constraint;
and 7: connecting each point on the extracted central line of the object surface with the coordinates of the industrial camera 2 into a straight line, and then solving the intersection point of the straight line and the corresponding light plane to obtain the three-dimensional coordinates of the central line of the object surface under the coordinate system of the industrial camera 2;
and 8: performing three-dimensional surface fitting according to the three-dimensional coordinates of the center line of the surface of the object to obtain three-dimensional coordinates of all points on the surface of the object;
and step 9: based on a random sample consensus algorithm (RANSAC algorithm), carrying out platform 4 plane fitting on the three-dimensional coordinates of the central line on the surface of the platform 4 extracted in the step5, and calculating the three-dimensional coordinates of the reference plane of the platform 4;
step 10: and (3) calculating the distance from the three-dimensional coordinate of the central line of the surface of the object to the plane of the workbench 4, and performing precision analysis on the three-dimensional measurement of the object.
When the image is collected in the step1, the circular calibration plate 6 is firstly inserted into the rubber magnetic housing 8, the rubber magnetic housing 8 is placed on the metal flat plate 9, an image for calibrating the industrial camera 2 is shot, then the rubber magnetic housing 8 is taken down, the circular calibration plate 6 is taken out, the white plate 7 is inserted into the rubber magnetic housing 8, then the circular calibration plate is placed on the metal flat plate 9, the multi-line laser is projected to the white plate 7, and an image for calibrating the optical plane is shot. The system can be calibrated by shooting for multiple times according to the process after the pose of the metal flat plate 9 is changed. The position of the flat metal plate 9 is not changed when the circular calibration plate 6 or the white plate 7 is replaced.
In the steps 1 and 2, due to the position change of the circular pattern on the circular calibration plate 6, the obtained pattern becomes an nonstandard ellipse at the view angle of the industrial camera 2. The invention obtains the edge information of the characteristic circle by carrying out edge detection on the characteristic circle (namely, an nonstandard characteristic ellipse), thereby fitting the characteristic circle according to a least square method to obtain the central coordinate information of the characteristic circle, and then completing the calibration of the industrial camera 2 by utilizing a Zhang's plane calibration method.
The invention is based on the laser triangulation vertical incidence type design idea, adopts the divergent multi-line laser 1 to carry out non-contact three-dimensional measurement on an object, the divergent multi-line laser 1 projects 25 laser lines on the white board 7, and the center position of the laser line at the middle position has a light spot.
Specifically, in the step 3 and the step5, according to the characteristics of the light plane, the laser lines irradiated on the white board 7 at different positions by the same light plane are all on the light plane, the light stripe center lines of the same laser line at different positions are extracted, and plane fitting is performed according to the extracted light stripe center line information, so that a light plane equation can be obtained, and the calibration of the light plane is completed. The optical plane equation is as follows:
Ax+By+Cz+D＝0
where A, B, C, D is the coefficients of the light plane equation and x, y, z are the three-dimensional coordinates of the light stripe center line in the industrial camera 2 coordinate system.
Specifically, the step6 adopts a multiline laser centerline extraction method based on system space structure constraint, and mainly includes the following steps:
step 1: graying and filtering the laser stripe image, setting a rectangular region of interest (ROI) and completing the extraction of any single line of the multi-line laser stripe;
step 2: and (3) performing edge extraction on the single line laser stripes obtained in Step1, acquiring edge pixel coordinates of the light stripes and calculating the width d of the light stripe pixels.
Step 3: taking a median value for the edge pixel abscissa of the optical stripe, and roughly extracting the central line of the optical stripe;
step 4: and calculating three-dimensional coordinates of the divergent multi-line laser 1 and the light spot in a coordinate system of the industrial camera 2. The intensity distribution of the middle stripe of the multi-line laser meets the Gaussian distribution and is standard normal distribution. Therefore, the system space structure constraint (including the space positions of the laser and the industrial camera 2 and the optical axis included angle thereof) is drawn, and the offset coefficient k of each stripe is obtained, and the calculation formula is as follows:
in the formula, the offset b is the difference between the abscissa of the central line of the light stripe before offset and the abscissa of the central line of the light stripe after offset, and the offset coefficient k is the ratio of the offset to the width of the same stripe.
Step 5: the width d of the stripe pixel of the roughly extracted light stripeiMultiplying by the corresponding offset coefficient kiGet the corresponding offset bi. As shown in the formula:
bi＝diki,(i＝1,2,...25)
step 6: roughly extracted light stripe central line pixel abscissa added with offset biThe accurate pixel coordinate of the central line of the light stripe can be obtained, and then the corresponding three-dimensional coordinate can be obtained.
Among them, in Step5 and Step6, (u)i,vi) Is the crude extraction pixel coordinate of the light stripe central line with the number of i,
is the exact pixel coordinate of the center line of the light stripe numbered i, biIs the offset of the light stripe numbered i, diStripe pixel width, k, of the light stripe numbered iiIs the shift coefficient of the light stripe numbered i.
Specifically, in step 7, when the object is measured in three dimensions, the acquired laser line image of the object surface is actually a projection of the laser line of the object surface onto the image sensor plane of the industrial camera 2 under the view angle of the industrial camera 2. Therefore, when three-dimensional measurement and reconstruction are carried out, a connecting line is needed between the projection point and the central point of the lens of the industrial camera 2, a plurality of straight lines are obtained and are intersected with the corresponding light plane, and the obtained intersection point is the three-dimensional coordinate of the central line of the surface stripe of the object under the coordinate system of the industrial camera 2.
Specifically, in step 10, the precision analysis of the three-dimensional measurement of the object is realized by calculating the distance from the three-dimensional coordinates of the center line of the object to the plane where the worktable 4 is located.
The invention has the beneficial effects that: the invention provides a three-dimensional measurement system and a measurement method based on divergent multi-line laser projection, which are characterized in that a divergent multi-line laser 1 is adopted to carry out non-contact three-dimensional measurement on an object, and compared with a single-line laser, the three-dimensional measurement system is more convenient and faster, and compared with a multi-line parallel laser, the cost is low. The industrial camera 2 calibration is separated from the light plane calibration by two calibration plates, wherein a circular calibration plate 6 is used for the industrial camera 2 calibration and a white plate 7 is used for the light plane calibration. Compared with a chessboard calibration plate, the circular calibration plate 6 has higher calibration precision, and the surface of the white board 7 has no pattern, so that interference on subsequent extraction of light stripe center lines is avoided. By adopting the multi-line laser center line extraction method based on system space structure constraint, the light stripe center line extraction under the condition of poor measurement environment can be quickly realized, the light plane calibration is completed, and the three-dimensional measurement is further realized.
Drawings
FIG. 1 is a system flow diagram of the present invention;
FIG. 2 is a block diagram of a hardware system of the present invention;
FIG. 3 is a schematic view of the optical plane calibration and coordinate system of the present invention;
FIG. 4 is a schematic view of a circular calibration plate and rubber magnet housing of the present invention;
FIG. 5 is a schematic view of a characteristic circle of the present invention;
FIG. 6 is a flow chart of industrial camera calibration according to the present invention;
FIG. 7 is a flow chart of the optical plane calibration of the present invention;
FIG. 8 is a schematic diagram of the light plane calibration of the present invention;
fig. 9 is a schematic diagram of the three-dimensional reconstruction of the present invention.
The reference numbers in the figures are: the device comprises a divergent multi-line laser-1, an industrial camera-2, a computer-3, a workbench-4, a bracket-5, a circular calibration plate-6, a white plate-7, a rubber magnetic shell-8 and a metal flat plate-9.
Detailed Description
Example 1: as shown in fig. 1-9, a three-dimensional measurement system based on divergent multi-line laser projection includes a divergent multi-line laser 1, an industrial camera 2, a homemade calibration device, a computer 3, a workbench 4, and a support 5. Wherein, the self-control equipment of maring includes circular calibration board 6, blank 7, rubber magnetism shell 8 and metal flat board 9, and it fixes on support 5 to disperse type multi-line laser 1 and industry camera 2, and 1 vertical projection of dispersion type multi-line laser, 2 slope shots of industry camera and links to each other with computer 3. In the off-site system pre-calibration stage, a metal flat plate 9 needs to be placed on the workbench 4, the circular calibration plate 6 and the white plate 7 are respectively inserted into the rubber magnetic shell 8, and the rubber magnetic shell 8 is placed on the metal flat plate 9. In the field measurement stage, the object to be measured needs to be placed on the workbench 4.
Further, the circular calibration board 6 and the white board 7 are used for system calibration (industrial camera 2 calibration and light plane calibration). The circular calibration plate 6 is used for calibrating the industrial camera 2, and the light plane calibration needs to be combined with the circular calibration plate 6 and the white board 7. The circular calibration plate 6 and the white plate 7 are each an opaque rectangular flat plate made of a hard PVC material. The hard PVC board has the characteristics of high strength, high hardness, flatness and difficult deformation. The surface of the circular calibration plate 6 is attached with a white black circle pattern, the pattern is a 7 multiplied by 7 array of black circles, the radius of the circle is 5mm, and the distance between the centers of the circles is 13 mm. The external dimension of the circular calibration plate 6 is 340 multiplied by 260mm, and the thickness is 3 mm. The white board 7 has a white surface without any pattern. The white board 7 has the external dimension of 340 multiplied by 260mm and the thickness of 3 mm.
Further, the flat metal plate 9 is used to ensure that the position where the calibration plate is placed remains unchanged. The metal flat plate 9 is made of martensite type stainless steel, and has magnetic permeability, the external dimension is 400 multiplied by 400mm, and the thickness is 5 mm.
Further, the rubber magnet housing 8 is used to ensure that the circular calibration plate 6 and the white plate 7 do not slide when the image is captured. The rubber magnet is a magnet with low hardness and good plasticity, is widely applied to refrigerator door seals, water dispenser door seals, magnetic blackboards and the like, has moderate adsorption force, and can be adsorbed on the surface of the metal flat plate 9. The side of the rubber magnetic shell 8 is provided with a slot for inserting the circular calibration plate 6 and the white plate 7 into the rubber magnetic shell 8, and the cross section of the slot is 261 multiplied by 4 mm. When the off-site system is pre-calibrated, the circular calibration plate 6 and the white plate 7 need to be respectively inserted into the rubber magnetic shell 8 and placed on the metal flat plate 9.
A measuring method of a three-dimensional measuring system based on divergent multi-line laser projection comprises the following steps:
step 1: firstly, the divergent multi-line laser 1 and the industrial camera 2 are kept fixed at proper positions of a bracket 5, then the metal flat plate 9 is placed in a field of view of the industrial camera 2, the number of the positions of the metal flat plate 9 is changed (N is more than or equal to 10), and two images are required to be acquired at each position. The image 1 is an image acquired by placing a rubber magnet housing 8, into which a circular calibration plate 6 is inserted, on a metal flat plate 9 under an illumination condition; the image 2 is an image captured under no-lighting conditions by placing the rubber magnet housing 8 with the whiteboard 7 inserted on the metal plate 9 and projecting a laser line onto the whiteboard.
Step 2: performing edge detection on the feature circle in the image 1 to obtain the center coordinate information of the feature circle, and calibrating the industrial camera 2 based on a Zhangyingyou plane calibration method to obtain internal and external parameters of the industrial camera 2;
and step 3: extracting light stripe center lines of the same laser line on the multiple images 2 based on a Steger multi-line laser center line extraction method;
and 4, step 4: converting the extracted center line pixel coordinates into three-dimensional coordinates under an industrial camera 2 coordinate system;
and 5: based on a random sample consensus (RANSAC) algorithm, performing light plane fitting on the three-dimensional coordinates of the central line, thereby obtaining an equation of a light plane and completing the calibration of the light plane;
step 6: when an object is placed in a severe environment for field measurement, the object is required to be placed on a workbench 4, laser lines are projected to the surface of the object and images are collected, and central line extraction is respectively carried out on light stripes on the surface of the object and light stripes on the surface of the workbench 4 by adopting a multi-line laser stripe central line extraction method based on system space structure constraint;
and 7: connecting each point on the extracted central line of the object surface with the coordinates of the industrial camera 2 into a straight line, and then solving the intersection point of the straight line and the corresponding light plane to obtain the three-dimensional coordinates of the central line of the object surface under the coordinate system of the industrial camera 2;
and 8: performing three-dimensional surface fitting according to the three-dimensional coordinates of the center line of the surface of the object to obtain three-dimensional coordinates of all points on the surface of the object;
and step 9: based on a random sample consensus algorithm (RANSAC algorithm), carrying out platform 4 plane fitting on the three-dimensional coordinates of the central line on the surface of the platform 4 extracted in the step6, and calculating the three-dimensional coordinates of the reference plane of the platform 4;
step 10: and (3) calculating the distance from the three-dimensional coordinate of the central line of the surface of the object to the plane of the workbench 4, and performing precision analysis on the three-dimensional measurement of the object.
Further, when the image is collected in step1, firstly, the circular calibration plate 6 is inserted into the rubber magnet housing 8, the rubber magnet housing 8 is placed on the metal flat plate 9, an image for calibration of the industrial camera 2 is shot, then the rubber magnet housing 8 is removed, the circular calibration plate 6 is taken out, the white plate 7 is inserted into the rubber magnet housing 8, then the circular calibration plate is placed on the metal flat plate 9, the multi-line laser is projected to the white plate 7, and an image for optical plane calibration is shot. The system can be calibrated by shooting for multiple times according to the process after the pose of the metal flat plate 9 is changed. The position of the flat metal plate 9 is not changed when the circular calibration plate 6 or the white plate 7 is replaced.
Further, in the steps 1 and 2, the circular pattern on the circular calibration plate 6 becomes an irregular ellipse at the view angle of the industrial camera 2 due to the position change. The invention obtains the edge information of the characteristic circle by carrying out edge detection on the characteristic circle (namely, an nonstandard characteristic ellipse), thereby fitting the characteristic circle according to a least square method to obtain the central coordinate information of the characteristic circle, and then completing the calibration of the industrial camera 2 by utilizing a Zhang's plane calibration method.
Further, based on the laser triangulation vertical incidence type design idea, the divergent multi-line laser 1 is adopted to perform non-contact three-dimensional measurement on the object, the divergent multi-line laser 1 projects 25 laser lines on the white board 7, and the center position of the laser line in the middle position has a light spot.
Further, in the step 3 and the step5, according to the characteristics of the light plane, the laser lines irradiated on the white board 7 at different positions by the same light plane are all on the light plane, the light stripe center lines of the same laser line at different positions are extracted, and plane fitting is performed according to the extracted light stripe center line information, so that a light plane equation can be obtained, and the calibration of the light plane is completed. The optical plane equation is as follows:
Ax+By+Cz+D＝0
where A, B, C, D is the coefficients of the light plane equation and x, y, z are the three-dimensional coordinates of the light stripe center line in the industrial camera 2 coordinate system.
Further, the step6 adopts a multiline laser centerline extraction method based on system space structure constraint, and mainly comprises the following steps:
step 1: graying and filtering the laser stripe image, setting a rectangular region of interest (ROI) and completing the extraction of any single line of the multi-line laser stripe;
step 2: and (3) performing edge extraction on the single line laser stripes obtained in Step1, acquiring edge pixel coordinates of the light stripes and calculating the width d of the light stripe pixels.
Step 3: taking a median value for the edge pixel abscissa of the optical stripe, and roughly extracting the central line of the optical stripe;
step 4: and calculating three-dimensional coordinates of the divergent multi-line laser 1 and the light spot in a coordinate system of the industrial camera 2. The intensity distribution of the middle stripe of the multi-line laser meets the Gaussian distribution and is standard normal distribution. Therefore, the system space structure constraint (including the space positions of the laser and the industrial camera 2 and the optical axis included angle thereof) is drawn, and the offset coefficient k of each stripe is obtained, and the calculation formula is as follows:
in the formula, the offset b is the difference between the abscissa of the central line of the light stripe before offset and the abscissa of the central line of the light stripe after offset, and the offset coefficient k is the ratio of the offset to the width of the same stripe.
Step 5: the width d of the stripe pixel of the roughly extracted light stripeiMultiplying by the corresponding offset coefficient kiGet the corresponding offset bi. As shown in the formula:
bi＝diki,(i＝1,2,...25)
step 6: roughly extracted light stripe central line pixel abscissa added with offset biThe accurate pixel coordinate of the central line of the light stripe can be obtained, and then the corresponding three-dimensional coordinate can be obtained.
Among them, in Step5 and Step6, (u)i,vi) Is the center line of the light stripe numbered iThe crude extraction of the pixel coordinates of (a),
is the exact pixel coordinate of the center line of the light stripe numbered i, biIs the offset of the light stripe numbered i, diStripe pixel width, k, of the light stripe numbered iiIs the shift coefficient of the light stripe numbered i.
Further, in the step 7, when the object is measured in three dimensions, the acquired laser line image of the object surface is actually a projection of the laser line of the object surface on the image sensor plane of the industrial camera 2 under the view angle of the industrial camera 2. Therefore, when three-dimensional measurement and reconstruction are carried out, a connecting line is needed between the projection point and the central point of the lens of the industrial camera 2, a plurality of straight lines are obtained and are intersected with the corresponding light plane, and the obtained intersection point is the three-dimensional coordinate of the central line of the surface stripe of the object under the coordinate system of the industrial camera 2.
Further, in the step 10, the precision analysis of the three-dimensional measurement of the object is realized by calculating the distance from the three-dimensional coordinates of the center line of the object to the plane where the worktable 4 is located.
The working principle of the invention is as follows:
a three-dimensional measurement system and a measurement method based on divergent multi-line laser projection, a principle of characteristic point coordinate extraction in industrial camera calibration: the circular pattern on the circular calibration plate is changed into an irregular ellipse due to the change of position under the view angle of the industrial camera, so that the regular ellipse must be rotated.
The standard elliptic equation is
b2(x-xc)2+a2(y-yc)2＝a2b2
Wherein (x)c,yc) And a and b are respectively a major semi-axis and a minor semi-axis of the ellipse. Due to the change of the position of the calibration plate, the shot circular template changes, and the ellipse equation changes accordingly. The characteristic circle equation at this time is
(x-xc)2(b2 cos2θ+a2 sin2θ)+(y-yc)2(b2 sin2θ+a2 cos2θ)
+(x-xc)(y-yc)(b2-a2)sin 2θ＝a2b2
Wherein (x)c,yc) And a and b are respectively a major semi-axis and a minor semi-axis of the characteristic circle, and theta is an elliptic rotation angle.
The invention carries out edge detection on the characteristic circle based on the Laplace operator, and fits the extracted edge information into the most appropriate ellipse according to the least square criterion.
Let Mi(xi,yi) And (i ═ 1,2, ·, N) is N measurement points on the characteristic circular profile, and the objective function G is set as a binary quadratic polynomial about x, y according to an arbitrary elliptic equation and a least square principle. According to the extreme principle, the partial differential of the objective function G for each coefficient is zero. Thereby obtaining a linear equation system related to polynomial coefficients, and then solving each coefficient to obtain the central coordinates (x) of the characteristic circlec,yc)。
A three-dimensional measurement technology based on divergent multi-line laser projection relates to a world coordinate system, an image coordinate system and an industrial camera coordinate system. The information extracted from the centerline of the light bar is the pixel coordinates of the centerline in the image coordinate system, which then needs to be converted to three-dimensional coordinates in the industrial camera coordinate system.
The light plane is calibrated according to the characteristics of the light plane: the laser lines of the white board which irradiate different positions on the same light plane are on the light plane. And extracting light stripe center lines of the same laser line at different positions, and performing plane fitting on the extracted light stripe center line information based on RANSAC algorithm to obtain a light plane equation to finish the calibration of the light plane.
A three-dimensional measurement technology based on divergent multi-line laser projection adopts a multi-line laser center line extraction method based on system space structure constraint. Filtering and edge extraction are carried out on laser lines in the multi-line laser stripe image, a region of interest (ROI) is set, single line extraction of the multi-line laser stripe is completed, the pixel width of the light stripe is calculated, and the pixel center line of the light stripe is roughly extracted. And calculating the width of light striations, the coordinates of the intersection point of light planes emitted by the divergent multi-line laser and the central coordinates of light spots irradiated on an object by the divergent multi-line laser under an industrial camera coordinate system according to the calibration result of the industrial camera, and calculating the central line offset b and the offset coefficient k in the system space structure constraint by combining the Gaussian distribution characteristic of the laser. And combining the pixel width of the light stripe and the center line offset coefficient k to finish the accurate extraction of the center line of the light stripe.
When the object is measured in three dimensions, the acquired laser line of the object surface is actually a projection of the laser line of the object surface onto the sensor plane of the industrial camera under the view angle of the industrial camera. Therefore, when three-dimensional measurement and reconstruction are carried out, the projection point needs to be connected with the industrial camera point to obtain a plurality of straight lines which are intersected with the corresponding light plane, and the obtained intersection point is the three-dimensional coordinate of the central line of the surface stripe of the object under the coordinate system of the industrial camera.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, such as the 25-wire diverging multi-wire laser, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (8)
1. A measuring method of a three-dimensional measuring system based on divergent multi-line laser projection is characterized in that:
a three-dimensional measurement system based on divergent multi-line laser projection comprises a divergent multi-line laser (1), an industrial camera (2), a homemade calibration device, a computer (3), a workbench (4) and a bracket (5), wherein the homemade calibration device comprises a circular calibration plate (6), a white plate (7), a rubber magnetic shell (8) and a metal flat plate (9), the divergent multi-line laser (1) and the industrial camera (2) are fixed on the bracket (5), the divergent multi-line laser (1) is vertically projected on the workbench (4), the industrial camera (2) is obliquely shot and connected with the computer (3), the metal flat plate (9) is placed on the workbench (4), the circular calibration plate (6) and the white plate (7) are respectively inserted into the rubber magnetic shell (8), the rubber magnetic shell (8) is placed on the metal flat plate (9), in the field measurement stage, a measured object needs to be placed on the workbench (4);
the measuring method of the three-dimensional measuring system based on the divergent multi-line laser projection comprises the following steps:
step 1: firstly, fixing a divergent multi-line laser (1) and an industrial camera (2) on a bracket (5), then placing a metal flat plate (9) in a field of view of the industrial camera (2), changing the position of the metal flat plate (9), wherein the number N of the positions is more than or equal to 10, and each position needs to acquire two images: the image 1 is an image collected by placing a rubber magnetic shell (8) inserted with a circular calibration plate (6) on a metal flat plate (9) under the condition of illumination; the image 2 is an image collected by placing a rubber magnetic shell (8) inserted with a white board (7) on a metal flat plate (9) and projecting a laser line onto the white board under the condition of no illumination;
step 2: performing edge detection on the feature circle in the image 1 to obtain the center coordinate information of the feature circle, and calibrating the industrial camera (2) based on a Zhang Zhengyou plane calibration method to obtain internal and external parameters of the industrial camera (2);
and step 3: extracting light stripe center lines of the same laser line on the multiple images 2 based on a Steger multi-line laser center line extraction method;
and 4, step 4: converting the extracted center line pixel coordinates into three-dimensional coordinates under an industrial camera (2) coordinate system;
and 5: based on a random sample consensus (RANSAC) algorithm, performing light plane fitting on the three-dimensional coordinates of the central line, so as to obtain an equation of a light plane and finish the calibration of the light plane;
step 6: when an object is placed in a severe environment for field measurement, the object is placed on a workbench (4), laser lines are projected to the surface of the object and images are collected, and the central line extraction is respectively carried out on light stripes on the surface of the object and the light stripes on the surface of the workbench (4) by adopting a multi-line laser stripe central line extraction method based on system space structure constraint;
and 7: connecting each point on the extracted center line of the object surface with the coordinates of the industrial camera (2) into a straight line, and then solving the intersection point of the straight line and the corresponding light plane to obtain the three-dimensional coordinates of the center line of the object surface under the coordinate system of the industrial camera (2);
and 8: performing three-dimensional surface fitting according to the three-dimensional coordinates of the center line of the surface of the object to obtain three-dimensional coordinates of all points on the surface of the object;
and step 9: based on a random sample consensus (RANSAC) algorithm, carrying out platform (4) plane fitting on the three-dimensional coordinates of the central line on the surface of the platform (4) extracted in the step6, and calculating the three-dimensional coordinates of the reference plane of the platform (4);
step 10: the distance from the three-dimensional coordinate of the center line of the surface of the object to the plane of the workbench (4) is calculated, and the three-dimensional measurement of the object is subjected to precision analysis;
the method for extracting the center line of the multi-line laser based on the system space structure constraint adopted in the step6 comprises the following steps:
step 1: graying and filtering the laser stripe image, setting a rectangular region of interest (ROI), and completing the extraction of any single line of the multi-line laser stripe;
step 2: performing edge extraction on the single line laser stripe image obtained in Step1, obtaining edge pixel coordinates of light stripes and calculating the width d of the light stripe pixels;
step 3: taking a median value for the edge pixel abscissa of the optical stripe, and roughly extracting the central line of the optical stripe;
step 4: calculating three-dimensional coordinates of the divergent multi-line laser (1) and the light spot under a coordinate system of the industrial camera (2), wherein the intensity distribution of the central stripe of the divergent multi-line laser (1) meets Gaussian distribution and is standard normal distribution, drawing system space structure constraint and solving a shift coefficient k of each stripe, wherein the system space structure constraint comprises the space positions of the laser and the industrial camera (2) and the included angle of the optical axes of the laser and the industrial camera, and the calculation formula is as follows:
in the formula, the offset b is the difference between the horizontal coordinate of the central line of the light stripe before offset and the horizontal coordinate of the central line of the light stripe after offset, and the offset coefficient k is the ratio of the offset of the same stripe to the width of the corresponding stripe;
step 5: the width d of the stripe pixel of the roughly extracted light stripeiMultiplying by the corresponding offset coefficient kiGet the corresponding offset biAs shown in formula:
bi＝diki,i＝1,2,...25
step 6: roughly extracted light stripe central line pixel abscissa added with offset biObtaining the accurate pixel coordinate of the center line of the light stripe, and further obtaining the corresponding three-dimensional coordinate;
among them, in Step5 and Step6, (u)i,vi) Is the crude extraction pixel coordinate of the light stripe central line with the number of i,
is the exact pixel coordinate of the center line of the light stripe numbered i, biIs the offset of the light stripe numbered i, diStripe pixel width, k, of the light stripe numbered iiIs the shift coefficient of the light stripe numbered i.
2. The measurement method of the divergent multiline laser projection-based three-dimensional measurement system as claimed in claim 1, wherein: the system calibration comprises industrial camera (2) calibration and light plane calibration, wherein a circular calibration plate (6) is used for industrial camera (2) calibration, the light plane calibration is combined with the circular calibration plate (6) and a white plate (7), the circular calibration plate (6) and the white plate (7) are opaque rectangular flat plates made of hard PVC materials, white-bottom black circle patterns are attached to the surface of the circular calibration plate (6), the patterns are black circles of a 7 x 7 array, and the surface of the white plate (7) is white and does not have any patterns.
3. The measurement method of the divergent multiline laser projection-based three-dimensional measurement system as claimed in claim 1, wherein: the metal flat plate (9) is made of martensite stainless steel and has magnetic permeability.
4. The measurement method of the divergent multiline laser projection-based three-dimensional measurement system as claimed in claim 1, wherein: the rubber magnetism of rubber magnetism shell (8) bottom can adsorb on the surface of metal flat board (9), and there is the slot side of rubber magnetism shell (8) for inside circular calibration board (6) and blank (7) inserted rubber magnetism shell (8), off-the-spot system is when pre-scaling, in circular calibration board (6) and blank (7) inserted rubber magnetism shell (8) respectively.
5. The measurement method of the divergent multiline laser projection-based three-dimensional measurement system as claimed in claim 1, wherein: when the image is collected in the step1, firstly, the circular calibration plate (6) is inserted into the rubber magnetic shell (8), the rubber magnetic shell (8) is placed on the metal flat plate (9), an image for calibrating the industrial camera (2) is shot, then the rubber magnetic shell (8) is taken down, the circular calibration plate (6) is taken out, the white plate (7) is inserted into the rubber magnetic shell (8), then the circular calibration plate is placed on the metal flat plate (9), multi-line laser is projected to the white plate (7), an image for calibrating the optical plane is shot, the position of the metal flat plate (9) is changed, the system can be calibrated by shooting for multiple times according to the process, and when the circular calibration plate (6) or the white plate (7) is replaced, the position of the metal flat plate (9) is not changed.
6. The measurement method of the divergent multiline laser projection-based three-dimensional measurement system as claimed in claim 1, wherein: in the step1 and the step 2, due to the position change of the circular pattern on the circular calibration plate (6), under the visual angle of the industrial camera (2), the obtained pattern is changed into an ellipse, the edge information of the characteristic ellipse is obtained by carrying out edge detection on the characteristic ellipse, the characteristic ellipse is fitted according to a least square method, the central coordinate information of the characteristic ellipse is obtained, and the calibration of the industrial camera (2) is completed by utilizing a Zhang's plane calibration method.
7. The measurement method of the divergent multiline laser projection-based three-dimensional measurement system as claimed in claim 1, wherein: and 3, in the step5, according to the characteristics of the light plane, the laser lines irradiated on the white board at different positions by the same light plane are all on the light plane, the light stripe center lines of the same laser line at different positions are extracted, and plane fitting is performed according to the extracted light stripe center line information, so that a light plane equation can be obtained, and the calibration of the light plane is completed.
8. The measurement method of the divergent multiline laser projection-based three-dimensional measurement system as claimed in claim 1, wherein: in the step 7, when the object is measured three-dimensionally, the acquired laser line image of the object surface is actually the projection of the laser line of the object surface on the image sensor plane of the industrial camera (2) at the viewing angle of the industrial camera (2), so that when the three-dimensional measurement and reconstruction are performed, the projection point needs to be connected with the lens central point of the industrial camera (2) to obtain a plurality of straight lines and intersect with the corresponding light plane, and the intersection point is the three-dimensional coordinate of the central line of the stripe on the object surface in the coordinate system of the industrial camera (2).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011086813.9A CN112161574B (en) | 2020-10-12 | 2020-10-12 | Three-dimensional measurement system and measurement method based on divergent multi-line laser projection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011086813.9A CN112161574B (en) | 2020-10-12 | 2020-10-12 | Three-dimensional measurement system and measurement method based on divergent multi-line laser projection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112161574A CN112161574A (en) | 2021-01-01 |
| CN112161574B true CN112161574B (en) | 2021-07-16 |
Family
ID=73868173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011086813.9A Active CN112161574B (en) | 2020-10-12 | 2020-10-12 | Three-dimensional measurement system and measurement method based on divergent multi-line laser projection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112161574B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113008158B (en) * | 2021-03-25 | 2023-02-24 | 烟台大学 | Multi-line laser tire pattern depth measuring method |
| CN113029041B (en) * | 2021-04-02 | 2022-11-01 | 扬州大学 | Measuring method for dislocation interpolation of multiple laser lines |
| CN113358052A (en) * | 2021-04-09 | 2021-09-07 | 宿迁学院 | Express size measuring device and method |
| CN113028912B (en) * | 2021-04-21 | 2022-09-20 | 湘潭大学 | Bullet primer priming charge filling amount detection method based on 3D vision |
| CN113566706B (en) * | 2021-08-01 | 2022-05-31 | 北京工业大学 | Device and method for composite rapid high-precision visual positioning |
| CN113465548B (en) * | 2021-08-23 | 2022-06-07 | 广东维正科技有限公司 | Calibration and precision evaluation method of fringe projection three-dimensional measurement system |
| CN113758422B (en) * | 2021-09-09 | 2022-10-14 | 昆明理工大学 | Method for numbering and identifying divergent multi-line laser stripes |
| CN117433450B (en) * | 2023-12-20 | 2024-04-19 | 派姆特科技(苏州)有限公司 | Cross line three-dimensional camera and modeling method |
| CN117705004B (en) * | 2023-12-27 | 2024-05-28 | 福建省高速公路科技创新研究院有限公司 | High-precision measurement and calibration method for compaction of filled soil layer |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105225224B (en) * | 2015-08-30 | 2017-12-26 | 大连理工大学 | Improve the arrangements of cameras and scaling method of depth of field measurement accuracy |
| CN106485757A (en) * | 2016-10-13 | 2017-03-08 | 哈尔滨工业大学 | A kind of Camera Calibration of Stereo Vision System platform based on filled circles scaling board and scaling method |
| US11042146B2 (en) * | 2017-11-17 | 2021-06-22 | Kodak Alaris Inc. | Automated 360-degree dense point object inspection |
| CN110455225B (en) * | 2019-08-22 | 2020-12-25 | 吉林大学 | Rectangular spline shaft coaxiality and key position measuring method based on structured light vision |
| CN111006599B (en) * | 2019-10-30 | 2021-07-20 | 东北大学 | Object surface micro-scale measurement method based on pixel grating and computer vision |
| CN111156925B (en) * | 2019-12-19 | 2021-12-28 | 南京理工大学 | Three-dimensional measurement method for large component based on line structured light and industrial robot |
| CN111090103B (en) * | 2019-12-25 | 2021-03-02 | 河海大学 | Three-dimensional imaging device and method for dynamically and finely detecting underwater small target |
| CN111207670A (en) * | 2020-02-27 | 2020-05-29 | 河海大学常州校区 | Line structured light calibration device and method |
-
2020
- 2020-10-12 CN CN202011086813.9A patent/CN112161574B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112161574A (en) | 2021-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112161574B (en) | Three-dimensional measurement system and measurement method based on divergent multi-line laser projection | |
| CN108759714B (en) | Coordinate system fusion and rotating shaft calibration method for multi-line laser profile sensor | |
| CN111486802B (en) | Rotating shaft calibration method based on self-adaptive distance weighting | |
| CN110118528B (en) | Line structure light calibration method based on chessboard target | |
| CN110378969B (en) | Convergent binocular camera calibration method based on 3D geometric constraint | |
| CN108562250B (en) | Keyboard keycap flatness rapid measurement method and device based on structured light imaging | |
| CN105486289B (en) | A kind of laser photography measuring system and camera calibration method | |
| JP2005514606A (en) | Three-dimensional three-dimensional measurement system and method | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN107869954B (en) | Binocular vision volume weight measurement system and implementation method thereof | |
| JP2005514606A5 (en) | ||
| CN102853786A (en) | Apparatus and method for detecting flatness | |
| CN110672037A (en) | Linear light source grating projection three-dimensional measurement system and method based on phase shift method | |
| CN109443214B (en) | Calibration method and device, measurement method and device for structured light three-dimensional vision | |
| CN113074660B (en) | Surface shape measuring method for large-size transparent object | |
| CN106524943B (en) | Three-dimensional reconstruction device and method of double-rotation laser | |
| CN109751964A (en) | A kind of contactless Calibration method and device of high-precision | |
| CN113119129A (en) | Monocular distance measurement positioning method based on standard ball | |
| CN109272542A (en) | A kind of determination method of three-dimension object volume | |
| CN109506629B (en) | Method for calibrating rotation center of underwater nuclear fuel assembly detection device | |
| CN115855955A (en) | Mold surface structure defect detection device and method based on multi-beam laser | |
| CN112747670A (en) | BGA packaging solder ball detection system and method | |
| CN112179292B (en) | Projector-based line structured light vision sensor calibration method | |
| CN115289997B (en) | Binocular camera three-dimensional contour scanner and application method thereof | |
| CN111006599A (en) | Object surface micro-scale measurement method based on pixel grating and computer vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |