CN109579695B - Part measuring method based on heterogeneous stereoscopic vision - Google Patents
Part measuring method based on heterogeneous stereoscopic vision Download PDFInfo
- Publication number
- CN109579695B CN109579695B CN201811194104.5A CN201811194104A CN109579695B CN 109579695 B CN109579695 B CN 109579695B CN 201811194104 A CN201811194104 A CN 201811194104A CN 109579695 B CN109579695 B CN 109579695B
- Authority
- CN
- China
- Prior art keywords
- formula
- image
- telecentric
- industrial camera
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000005259 measurement Methods 0.000 claims abstract description 48
- 238000003384 imaging method Methods 0.000 claims abstract description 41
- 230000008569 process Effects 0.000 claims description 11
- 238000013178 mathematical model Methods 0.000 claims description 8
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 5
- 150000001875 compounds Chemical class 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 5
- 238000004519 manufacturing process Methods 0.000 abstract description 3
- 230000003287 optical effect Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 5
- 238000000691 measurement method Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000002310 reflectometry Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000011664 nicotinic acid Substances 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 229960000744 vinpocetine Drugs 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a part measuring method based on heterogeneous stereoscopic vision, which adopts 1 telecentric industrial camera and 2 ordinary industrial cameras, carries out relevant processing on images shot by the cameras, enables binocular stereoscopic vision and monocular telecentric vision to be effectively combined, compensates principle errors of an ordinary optical system through telecentric imaging, integrates the advantages of the binocular vision and the advantage of high measurement precision of the telecentric vision, solves the problem of low three-dimensional non-contact measurement precision of a production field, and has the technical effects of high data reliability and reliable measurement result.
Description
Technical Field
The invention relates to the field of part size measurement, in particular to a part measurement method based on heterogeneous stereoscopic vision.
Background
The research of part detection based on machine vision is started from 90 years in the 20 th century, and gradually enters various industrial fields, and measuring means and methods are rapidly developed. Machine vision is to use a machine to replace human eyes for measurement and judgment, convert a shot target into an image signal through an image shooting device CMOS or CCD, transmit the image signal to a special image processing system, and convert the image signal into a digital signal according to information such as pixel distribution, brightness, color and the like; the image system performs various calculations on these signals to extract the features of the target, and then controls the operation of the on-site equipment according to the result of the discrimination.
Prior art 1:
inspired by human eyes for sensing the environmental depth by the parallax principle, the binocular stereo vision measurement system simultaneously observes a measured part (shown in figure 1) from two points in space, obtains two images of the measured part under different visual angles, calculates the position deviation between pixels by the triangulation principle according to the registration relation of the pixels between the two images, obtains the depth information of any point in a three-dimensional space, finally reconstructs the three-dimensional shape of the measured part, and performs multi-element measurement of the part.
Aiming at the problem of stepped shaft measurement, Wuxiang, Changchun university of industry establishes a set of binocular vision measurement system, and can measure a plurality of elements of a measured part. Experimental results show that the minimum and maximum measurement errors of the binocular measurement system are 0.1mm and 0.9mm, respectively (wuxiang. critical technology research of a binocular vision-based part dimension measurement system [ D ]. vinpocetine university, 2017.). Zhang Jun Yong of Wuhan science and technology university proposes a part multi-size measurement method and system based on binocular vision, and realizes multi-element three-dimensional measurement of a measured part through a series of processes such as calibration, polar line correction, polar line constraint matching, three-dimensional fitting, three-dimensional reconstruction and the like (Zhang Yong, Wushiqian, Xuzhang. a part multi-size measurement method and system based on binocular vision: China, CN107588721A [ P ]. 2018.01.16.). A binocular vision measuring and positioning device and a method for narrow space (Zhao 21089, Su Qing, Wu Fang Lin, Yang Kui, Zhang Xiao Cheng) are provided for three-dimensional measurement of targets in narrow space such as Zhao \21089ofBeijing university of aerospace, China, ZL201210191014.7[ P ]. 2012.11.05)
The binocular vision measurement is a method based on the bionic principle, has a plurality of advantages, and is ideally very suitable for online non-contact geometric precision inspection and quality control in a manufacturing field. However, binocular vision is not as applicable to other methods, and the most critical limiting factor is the measurement accuracy, and the accuracy of most binocular systems is only in the sub-millimeter level.
Prior art 2:
the theoretical basis of the structured light vision measurement is the laser triangulation distance measurement principle. By projecting a structural light source known by a spatial mathematical model to a measured part, a machine vision system projects and images the part and the structured light (shown in figure 2), and the spatial mathematical model of the structured light and the machine vision system imaging mathematical model are combined, so that the recovery depth information can be obtained by solving. The light projection mode includes four types of light, plane, curved surface and light beam, so that the common structured light vision measuring system also includes four types.
A structured light vision measuring system is constructed by the great bear university of Harbin industry, and the measuring range of the system is not less than 200mm multiplied by 200mm when the object distance is 250 mm. The system was used to make repeated measurements on standard gauge blocks of 8.845mm and 20.000mm thickness and width, respectively. In the thickness measurement experiment, the maximum and minimum measurement errors were 50 μm and 30 μm, respectively; in the width measurement experiment, the maximum and minimum measurement errors were 50 μm and 18 μm, respectively (bear great man. development of three-dimensional structured light vision measuring apparatus [ D ]. harabine: harabine university of industry, 2017.). The method mainly comprises the processes of calibrating a structural light vision system by a plane model method, collecting and preprocessing characteristic images, extracting characteristics, calculating characteristic parameters and the like, and has the application characteristics of high precision, high adaptability and high efficiency (Zhang from Peng, Hou, Cao Wen, Lu Lei, Yu of any world). The invention relates to a structured light measuring method and a device thereof for high-reflectivity parts, which mainly solve the problem that the high-reflectivity parts are difficult to measure (about ten thousand waves, Guo Yanyan, forward, and the like). A surface structured light three-dimensional measuring device and a method for the high-reflectivity parts are ZL201310717211.2[ P ] 2016.09.07 ]. The method comprises the steps of measuring the radial run-out error of a shaft part by using a structured light vision system in Tanchang, university of Jilin, and the like, calibrating a vision sensor and structured light by a two-step calibration method of a Zhangzhou plane and a template matching method respectively on the basis of establishing a radial run-out error structured light vision model, and solving a space coordinate of an intersection point of the structured light and the surface of the part by using the measurement model (Tanchang, Bahaohan, Zhang Yachao, and the like, an on-line measurement method for the radial run-out error of the shaft part based on structured light vision, China, CN107101582A [ P ] 2017.08.29.).
The structured light vision and the improvement method thereof both utilize a light source with a known pose to carry out active illumination, and the image depth information can be recovered only by extracting features from one image, so that the difficult problem of registration of a left image and a right image in binocular vision is solved, and therefore, the measurement accuracy of the structured light vision is obviously improved compared with that of the binocular vision. Nevertheless, the measurement accuracy of structured light vision still remains at the silk level. Moreover, compared with binocular vision, the point cloud density of structured light vision is obviously reduced, which is not beneficial to the three-dimensional reconstruction of the tiny features of the part. In addition, the calibration problem of the structured light, the high performance laser and the cost problem thereof, and the like, are still to be researched.
Prior art 3:
the multi-view vision and the binocular vision have the same theoretical basis and working principle, and are formed by additionally arranging a plurality of imaging sensors on the basis of the original binocular vision. The multi-vision simultaneously observes the measured part from three or more points in the space to obtain a plurality of images of the measured part under different vision, and according to the registration relation between the images and the triangulation distance measurement principle, a mathematical model representing the depth of any point in the three-dimensional space is established, so that the three-dimensional appearance of the measured part can be reconstructed, and the multi-element measurement of the part can be carried out. Compared with binocular vision, the multi-ocular vision introduces more constraint conditions and theoretically has higher measurement accuracy. The simplest multi-ocular vision is the three-ocular vision, and its application in industry presents an increasing trend.
And a third camera is introduced into a binocular system by Ye and the like to form a binocular vision measuring system, so that more constraint conditions are added for stereo matching, uncertainty of binocular vision system matching is reduced, interference information is eliminated, and result precision is effectively improved (Ye Pan, Li, Jin Wei-Qi, Jiang Yu-Tong.research on imaging and mapping based on the constraint information of trigonometric vision [ C ]// Proc.SPIE 9301, IPTA 2014: Image Processing and Pattern registration. Beijing, China, y 13-15,2014.). Lu and Shao establish a set of three-purpose system, design a spherical calibration piece, and obtain a translation and rotation matrix through singular value decomposition. The results show that the relative accuracy of the trinocular system is improved to 0.105%, the root mean square error is 0.026mm, and the accuracy and robustness of the system are proved (Lu Rui, Shao Mingwei. sphere-based calibration method for a trinocular vision sensor [ J ]. Opt. Lasers Eng.,2017,90: 119-127.). Aiming at the problem of low precision of binocular vision measurement of large-amplitude swinging objects, a target tracking method and a target tracking device based on trinocular vision are proposed by the technical company of intelligent-made-to-future (Beijing) robot system (Korea Xin. trinocular vision identification and tracking device and method: China, CN107507231A [ P ]. 2017.12.22.).
Inspired by multi-sensor fusion measurement technology, a three-eye or more-eye multi-vision system is constructed by introducing more imaging sensors on the basis of a binocular system, so that measurement uncertainty is reduced, and measurement precision is improved to a certain extent. However, the precision of the trinocular system still cannot meet the requirement of high-precision measurement of mechanical parts. The reason for this is that many cameras in the multi-view vision system all use pinhole imaging models, and the principle errors such as parallax and distortion caused by the common lens still cannot be eliminated.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a part measuring method based on heterogeneous stereoscopic vision, which effectively combines binocular stereoscopic vision and monocular telecentric vision, compensates the principle error of a common optical system through telecentric imaging, integrates the advantages of the binocular vision and the advantage of high telecentric vision measuring precision, and solves the problem of low three-dimensional non-contact measuring precision in a production field.
The technical scheme adopted by the invention is as follows:
a part measuring method based on heterogeneous stereoscopic vision comprises the following steps:
s1, determining the positions of a No. 1 telecentric industrial camera, a No. 2 ordinary industrial camera and a No. 3 ordinary industrial camera according to the size of the part to be measured and the depth of field of the cameras, keeping the position relation among the cameras, calibrating the cameras, and enabling the cameras to image the part to be measured to obtain a left image, a right image and a telecentric image;
s2, registering the left-right image pair and the left-right-telecentric image pair through processes including image preprocessing, feature extraction and feature matching;
s3, constructing a binocular vision system mathematical model according to the registration relation of the left image and the right image and the internal and external parameters calibrated by the common industrial camera to obtain the depth information of the characteristic points of the left image and the right image;
s4, grouping the feature points of the left-right-telecentric image into a group according to each 2 feature points, calculating the absolute difference of the depths of the 2 feature points in each group according to the depth information obtained in S3, and excluding the group if the absolute difference is greater than a preset threshold;
s5, projecting the feature points reserved in the S4 to a focal plane of a telecentric industrial camera, and calculating the distance of 2 projection points in each group along the horizontal and vertical coordinate directions;
s6, correcting the internal and external parameters marked in S1 according to the distance obtained in S5, reconstructing a binocular vision system mathematical model according to the registration relation of the left image and the right image obtained in S2 and the corrected internal and external parameters of the common industrial camera, and obtaining the depth information of the feature points of the updated left image and the updated right image;
and S7, constructing a three-dimensional model of the measured part according to the updated depth information, performing three-dimensional feature recognition, and extracting outline elements of the measured part, thereby realizing the measurement of the geometric features of the measured part.
Preferably, in S1, the telecentric industrial camera is calibrated as follows: and imaging the calibration piece with the known size by the telecentric camera, and acquiring the ratio of the actual size of the calibration piece to the pixel size to obtain the equivalent pixel of the telecentric camera.
Preferably, in S1, the general industrial camera is calibrated as follows: after the common industrial camera is installed, the common industrial camera is calibrated by adopting a Zhangyingyou plane calibration method, and internal and external parameters of the common industrial camera are determined based on a calibration result.
Preferably, in S4, feature points in the left image, the right image and the telecentric image are extracted, homonymous features of the left image and the right image are found, the euclidean distance between the two feature points is measured, and if the absolute difference is greater than a preset threshold, the group of feature points is rejected.
Preferably, the functional procedure of the part measurement method is as follows:
establishing a world coordinate system ow-xwywzwCamera coordinate system o of No. 1 telecentric industrial camera1c-x1cy1cz1cCamera coordinate system o of general industrial camera No. 22c-x2cy2cz2cCamera coordinate system o of general industrial camera No. 33c-x3cy3cz3c；
A certain point P on the part to be measurediWorld coordinate is (x)i,yi,zi) The coordinates of the point under the coordinate systems of the No. 1, No. 2 and No. 3 cameras are respectively as follows:
in the formula RjAnd TjRepresenting the coordinate transformation relation from a world coordinate system to a j (j is 1,2,3) th imaging system camera coordinate system, which is also called an imaging system external parameter;
by using
Represents PiThe image in the jth imaging system has a coordinate of (u)ij,vij) Where j is 1,2,3, the relationship between the image point and the spatial point is:
in the formula of alphaj、βj、γj、ujAnd vjIs an imaging system intrinsic parameter, (u)ij,vij) Is a picture point PijThe pixel coordinates of (a);
extracting the characteristic points of the images collected by the No. 1, No. 2 and No. 3 imaging systems, and establishing a corresponding characteristic point combination set A1、A2And A3；
From set A2In which an element a is arbitrarily selected2s，
For positive integers, search strategy and measure function are adopted in A3Find out the 'same name' feature a3t，
Respectively a to2sAnd a3tDeposit to New set B2And B3Performing the following steps;
from set B2In any one of the elements b2s，
In B3In which there is its "same name" feature b3sUsing search strategy and measure function in set A1In (b) is found2sCharacteristic of "same name" of (a)1r，
Validating a with a measure function1rAnd b3sWhether it is a pair of "same name" features, if so, a1r、b2s、b3sAre respectively stored in a new set C1、C2And C3From set B if not2In the process of re-choosing an element b2t，
And s is not equal to t, the steps are repeated until the collection B is completely taken2All of the elements in (1);
the equations (1) and (3) are put together and rewritten as follows, and the spatial coordinates to be found are written to the left side of the equations:
in the formula
Let m be | C2|，c2i∈C2，c3i∈C3I 1,2, …, m, characterPoint c2iHas a pixel coordinate of (u)i2,vi2) I.e., j is 2; characteristic point c3iHas a pixel coordinate of (u)i3,vi3) I.e., j is 3; substituting the pixel coordinates into equations (4) and hjkAnd
spatial coordinate (x) in formula (5)i,yi,zi) Unknown, and the remaining parameters are known, and rewriting formula (5) into formula (6):
Gixi＝gi (6)
in the formula xi＝[xi,yi,zi]T，giAnd GiSee, respectively, formula (7) and formula (8):
solving the formula (6) by the least square method to obtain
Defining a new set D by taking the space coordinate obtained by solving as an element; taking two elements from set D
And
p, q ≠ 1,2, …, m, and p ≠ q, calculating the difference in z-coordinates
If the absolute value of z is less than a given non-negative constant τ, go to set C1Find and space point xpAnd xqThe corresponding pixel points have pixel coordinates of (u)p1,vp1) And (u)q1,vq1) And measuring by using a No. 1 imaging system to obtain:
in the formula, beta-And beta⊥Equivalent pixels in the horizontal and vertical directions of the No. 1 imaging system, respectively, in mm/pixel units;
the confidence level is 95 percent, and the expansion uncertainty of the No. 1 imaging system is U95，dkTrue value of
It should satisfy:
in the formula (I), the compound is shown in the specification,
changing the values of p and q, and repeating the process until xiAll (i ═ 1,2, … m) are linked, so in formula (9) and formula (10) the subscript k is used, which ranges from [1, n ═ n]N should be not less than m-1;
equation (6) is equivalent to the nonlinear unconstrained extremum problem shown in equation (11):
the measurement accuracy of the binocular vision system can reach 1/10mm, and in the field depth range of the telecentric vision system, based on the two points, the whole formula (10) is used as a constraint condition, and the formula (11) is changed into a nonlinear constraint extreme value problem:
using the solution of least squares as initial values, i.e.
Solving equation (12) to obtain an updated solution
Will be provided with
And the pixel coordinates are substituted into the formula (1) and the formula (3) to update the internal and external parameters of the camera to be
And Tj(ii) a Substituting the updated camera internal and external parameters into the formula (6), and using the formula (6) to set B2And B3The element in (3) calculates its three-dimensional coordinates.
Compared with the prior art, the invention has the following implementation effects:
the invention organically combines the prior art I and a telecentric vision system, obtains a measurement result by fusing data of a plurality of sensors, increases the utilization rate of system information, enhances the reliability of data and improves the reliability of the system.
The invention does not need to use a high-performance laser light source, and has no problem of high cost of the laser light source and no problem of point cloud density reduction caused by interval motion of the structured light.
The third camera introduced by the invention is a telecentric industrial camera which has the characteristics of constant magnification factor, no parallax, small image distortion and the like, and has higher measurement precision.
Drawings
Fig. 1 is a schematic diagram of the measurement of a binocular stereo vision system in the prior art 1.
Fig. 2 is a schematic diagram of a line structured light vision measurement in prior art 2.
Fig. 3 is a schematic diagram of the measurement of a multi-vision system in prior art 3.
Fig. 4 is a schematic structural diagram of the present invention.
Fig. 5 is a schematic diagram of the principle of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Of course, the described embodiments are only some embodiments of the invention, and not all embodiments.
Referring to fig. 4 and 5, in the drawings:
1 (a): telecentric industrial camera No. 1;
2 (a): no. 2 general industrial camera;
3 (a): no. 3 general industrial camera;
1 (b): 1 (a);
2 (b): 2 (a);
3 (b): 3 (a);
4: a part to be tested;
5: any measured point p on the measured part;
6: any measured point q on the measured part;
7: the image point of the measured point P in 1 (a);
8: the image point of the measured point q in 1 (a);
9: the image point of the measured point q in 2 (b);
10: the image point of the measured point P in the imaging system 3;
11: and (5) installing a foundation for the heterogeneous stereoscopic vision system.
Before measurement, the positions of a No. 1 telecentric industrial camera, a No. 2 common industrial camera and a No. 3 common industrial camera are determined according to the size of a measured part and the depth of field of the cameras, the position relation among the cameras is maintained, a three-dimensional measurement space shown in figure 4 is established, and then the three cameras are calibrated.
ow-xwywzwIs the world coordinate system o1c-x1cy1cz1cIs the camera coordinate system of No. 1 telecentric Industrial Camera, o2c-x2cy2cz2cIs the camera coordinate system of No. 2 common industrial camera, o3c-x3cy3cz3cIs the camera coordinate system of general industrial camera No. 3.
PiAt a certain point on the part to be measured, its world coordinate is (x)i,yi,zi) The coordinates of the point under the coordinate systems of the No. 1, No. 2 and No. 3 cameras are respectively as follows:
in the formula RjAnd TjThe coordinate transformation relationship from the world coordinate system to the j (j ═ 1,2,3) th imaging system camera coordinate system is shown, and is also called as an imaging system external parameter.
By using
Represents PiThe image in the jth imaging system has a coordinate of (u)ij,vij) Where j is 1,2,3, the relationship between the image point and the spatial point is:
in the formula of alphaj、βj、γj、ujAnd vjIs an imaging system intrinsic parameter, (u)ij,vij) Is a picture point
The pixel coordinates of (a).
Generally, the telecentric imaging system No. 1 does not calibrate its equivalent pixels by calibrating its internal and external parameters, i.e. by imaging a calibration of known dimensions to obtain the ratio of the actual dimensions of the calibration to the pixel dimensions. There are various solutions to the Calibration problem of imaging systems 2 and 3, such as Zhang Zheng Young's A Flexible New technology for Camera Calibration [ J ]. IEEE Transactions on Pattern Analysis and Machine Analysis, 2000,22(11):1330-1334 ]. Now, it is assumed that all three imaging systems are calibrated, that is, the equivalent pixels of the imaging system No. 1, and the internal and external parameters of the imaging systems No. 2 and No. 3 are known.
The 3 cameras are used for imaging the tested part to obtain a left image, a right image and a telecentric image, and images collected by No. 1, No. 2 and No. 3 imaging systems are respectively assumed to be image1, image2 and image 3. Now, feature point extraction is performed on the images 1,2 and 3, and corresponding feature descriptors are established, such as the classic feature detection and feature descriptor algorithm SIFT (Lowe D g. distinguishing Image Features from scales-inverse keys [ J)]International Journal of Computer Vision,2004,60(2): 91-110). Set A1、A2And A3Are the feature points of image1, image2, and image3, respectively.
From set A2In which an element a is arbitrarily selected2s 
Using search strategy and measure function at A3Find out the 'same name' feature a3t 
I.e. to perform feature retrieval and registration. A commonly used search strategy is a KD tree and an improved algorithm thereof, such as a BBF (Best-Bin-First, BBF) search strategy (Beis J S, Lowe D G. shape indexing using adaptive search nearest-neighbor search in high-dimensional spaces [ C)]//Conference on Computer Vision and Pattern Recognition,Puerto Rico, USA,17-19June1997: 1000-. The most commonly used measure function is the euclidean distance. Theoretically a2sAnd a3tShould be zero, in practice their euclidean distances are usually not zero. Therefore, a relatively small non-negative number is often taken, when a2sAnd a3tWhen the Euclidean distance is less than (a), it is determined that2sAnd a3tIs a pair of "homonymous" features. Respectively a to2sAnd a3tDeposit to New set B2And B3In (1).
From set B2In any one of the elements b2s 
Is a positive integer, in B3In which the "same name" feature is b3s. According to the characteristic retrieval and registration algorithm of the last step, in the set A1In (b) is found2sCharacteristic of "same name" of (a)1r 
Validating a with a measure function1rAnd b3sWhether it is a pair of "same name" features, if so, a1r、b2s、b3sAre respectively stored in a new set C1、C2And C3From set B if not2In the process of re-choosing an element b2t(
And s ≠ t), repeating the steps until the collection B is taken2All of the elements in (a).
Equations (1) and (3) are put together and rewritten as follows, while writing the spatial coordinates to be found to the left of the equations:
in the formula
Let m be | C2|，c2i∈C2，c3i∈C3I is 1,2, …, m2iHas a pixel coordinate of (u)i2,vi2) I.e., j is 2; characteristic point c3iHas a pixel coordinate of (u)i3,vi3) I.e., j is 3. Substituting the pixel coordinates into equations (4) and hjkAnd
formula (5) is the middle spatial coordinate (x)i,yi,zi) Are unknown and the remaining parameters are known. Rewriting formula (5) to formula (6):
Gixi＝gi (6)
in the formula xi＝[xi,yi,zi]T，giAnd GiSee, respectively, formula (7) and formula (8):
solving the formula (6) by the least square method to obtain
And defining a new set D by taking the solved space coordinates as elements. Taking two elements from set D
And
(p, q ≠ q) 1,2, …, m, and p ≠ q), calculates the difference in z-coordinate
If the absolute value of z is less than a given non-negative constant τ, go to set C1Find and space point xpAnd xqThe corresponding pixel points have pixel coordinates of (u)p1,vp1) And (u)q1,vq1) And measuring by using a No. 1 imaging system to obtain:
in the formula, beta-And beta⊥Equivalent pixels in the horizontal and vertical directions of the No. 1 imaging system, respectively, are in mm/pixel units.
The confidence level is 95 percent, and the expansion uncertainty of the No. 1 imaging system is U95，dkTrue value of
It should satisfy:
in the formula (I), the compound is shown in the specification,
changing the values of p and q, and repeating the process until xiAll (i ═ 1,2, … m) are linked, so in formula (9) and formula (10) the subscript k is used, which ranges from [1, n ═ n]And n should be not less than m-1.
Equation (6) is equivalent to the nonlinear unconstrained extremum problem shown in equation (11):
the telecentric vision system has good optical performance and the measurement precision is far higher than that of a binocular vision system. In addition, the measurement accuracy of the binocular vision system can reach 1/10mm, and the depth of field of the telecentric vision system is within the range. Therefore, based on the above two points, all equation (10) is used as the constraint condition, and equation (11) becomes the nonlinear constraint extremum problem:
using the solution of least squares as initial values, i.e.
Solving equation (12) to obtain an updated solution
Will be provided with
And the pixel coordinates are substituted into the formula (1) and the formula (3) to update the internal and external parameters of the camera to be
And Tj. Substituting the updated camera internal and external parameters into the formula (6), and using the formula (6) to set B2And B3The element in (3) calculates its three-dimensional coordinates.
And finally, reconstructing a three-dimensional model of the part by using the three-dimensional coordinates obtained in the last step, and performing three-dimensional multi-factor measurement.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (5)
1. A part measuring method based on heterogeneous stereoscopic vision is characterized by comprising the following steps:
s1, determining the positions of a No. 1 telecentric industrial camera, a No. 2 ordinary industrial camera and a No. 3 ordinary industrial camera according to the size of the part to be measured and the depth of field of the cameras, keeping the position relation among the cameras, calibrating the cameras, and enabling the cameras to image the part to be measured to obtain a left image, a right image and a telecentric image;
s2, registering the left-right image pair and the left-right-telecentric image pair through processes including image preprocessing, feature extraction and feature matching;
s3, constructing a binocular vision system mathematical model according to the registration relation of the left image and the right image and the internal and external parameters calibrated by the common industrial camera to obtain the depth information of the characteristic points of the left image and the right image;
s4, grouping the feature points of the left-right-telecentric image into a group according to each 2 feature points, calculating the absolute difference of the depths of the 2 feature points in each group according to the depth information obtained in S3, and excluding the group if the absolute difference is greater than a preset threshold;
s5, projecting the feature points reserved in the S4 to a focal plane of a telecentric industrial camera, and calculating the distance of 2 projection points in each group along the horizontal and vertical coordinate directions;
s6, correcting the internal and external parameters marked in S1 according to the distance obtained in S5, reconstructing a binocular vision system mathematical model according to the registration relation of the left image and the right image obtained in S2 and the corrected internal and external parameters of the common industrial camera, and obtaining the depth information of the feature points of the updated left image and the updated right image;
and S7, constructing a three-dimensional model of the measured part according to the updated depth information, performing three-dimensional feature recognition, and extracting outline elements of the measured part, thereby realizing the measurement of the geometric features of the measured part.
2. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S1, the telecentric industrial camera is calibrated as follows: and imaging the calibration piece with the known size by the telecentric camera, and acquiring the ratio of the actual size of the calibration piece to the pixel size to obtain the equivalent pixel of the telecentric camera.
3. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S1, a normal industrial camera is calibrated as follows: after the common industrial camera is installed, the common industrial camera is calibrated by adopting a Zhangyingyou plane calibration method, and internal and external parameters of the common industrial camera are determined based on a calibration result.
4. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S4, feature points in the left image, the right image and the telecentric image are extracted, the homonymous features of the left image and the right image are found, the euclidean distance between the two feature points is measured, and if the absolute difference value is greater than a preset threshold value, the set of feature points is rejected.
5. The part measuring method based on the heterogeneous stereovision according to claim 1, wherein the function process of the part measuring method is as follows:
establishing a world coordinate system ow-xwywzwCamera coordinate system o of No. 1 telecentric industrial camera1c-x1cy1cz1cCamera coordinate system o of general industrial camera No. 22c-x2cy2cz2cCamera coordinate system o of general industrial camera No. 33c-x3cy3cz3c；
A certain point P on the part to be measurediWorld coordinate is (x)i,yi,zi) The coordinates of the point under the coordinate systems of the No. 1, No. 2 and No. 3 cameras are respectively as follows:
in the formula RjAnd TjRepresenting the coordinate transformation relation from a world coordinate system to a j (j is 1,2,3) th imaging system camera coordinate system, which is also called an imaging system external parameter;
by using
Represents PiThe image in the jth imaging system has a coordinate of (u)ij,vij) Where j is 1,2,3, the relationship between the image point and the spatial point is:
in the formula of alphaj、βj、γj、ujAnd vjIs an imaging system intrinsic parameter, (u)ij,vij) Is a picture point
The pixel coordinates of (a);
extracting the characteristic points of the images collected by the No. 1, No. 2 and No. 3 imaging systems, and establishing a corresponding characteristic point combination set A1、A2And A3；
From set A2In which an element a is arbitrarily selected2s，
For positive integers, search strategy and measure function are adopted in A3Find out the 'same name' feature a3t，
Respectively a to2sAnd a3tDeposit to New set B2And B3Performing the following steps;
from set B2In any one of the elements b2s，
In B3In which there is its "same name" feature b3sUsing search strategy and measure function in set A1In (b) is found2sCharacteristic of "same name" of (a)1r，
Validating a with a measure function1rAnd b3sWhether it is a pair of "same name" features, if so, a1r、b2s、b3sAre respectively stored in a new set C1、C2And C3From set B if not2In the process of re-choosing an element b2t，
And s is not equal to t, the steps are repeated until the collection B is completely taken2All of the elements in (1);
the equations (1) and (3) are put together and rewritten as follows, and the spatial coordinates to be found are written to the left side of the equations:
in the formula
k＝1,2,3
Let m be | C2|，c2i∈C2，c3i∈C3I is 1,2, …, m, characteristic point c2iHas a pixel coordinate of (u)i2,vi2) I.e., j is 2; characteristic point c3iHas a pixel coordinate of (u)i3,vi3) I.e., j is 3; substituting the pixel coordinates into equations (4) and hjkAnd
spatial coordinate (x) in formula (5)i,yi,zi) Unknown, and the remaining parameters are known, and rewriting formula (5) into formula (6):
Gixi＝gi (6)
in the formula xi＝[xi,yi,zi]T，giAnd GiSee, respectively, formula (7) and formula (8):
solving the formula (6) by the least square method to obtain
Defining a new set D by taking the space coordinate obtained by solving as an element; taking two elements from set D
And
p, q ≠ 1,2, …, m, and p ≠ q, calculating the difference in z-coordinates
If the absolute value of z is less than a given non-negative constant τ, go to set C1Find and space point xpAnd xqThe corresponding pixel points have pixel coordinates of (u)p1,vp1) And (u)q1,vq1) And measuring by using a No. 1 imaging system to obtain:
in the formula, beta-And beta⊥Equivalent pixels in the horizontal and vertical directions of the No. 1 imaging system, respectively, in mm/pixel units;
the confidence level is 95 percent, and the expansion uncertainty of the No. 1 imaging system is U95，dkTrue value of
It should satisfy:
in the formula (I), the compound is shown in the specification,
changing the values of p and q, and repeating the process until xiAll (i ═ 1,2, … m) are linked, so in formula (9) and formula (10) the subscript k is used, which ranges from [1, n ═ n]N should be not less than m-1;
equation (6) is equivalent to the nonlinear unconstrained extremum problem shown in equation (11):
the measurement accuracy of the binocular vision system can reach 1/10mm, and in the field depth range of the telecentric vision system, based on the two points, the whole formula (10) is used as a constraint condition, and the formula (11) is changed into a nonlinear constraint extreme value problem:
using the solution of least squares as initial values, i.e.
Solving equation (12) to obtain an updated solution
Will be provided with
And the pixel coordinates are substituted into the formula (1) and the formula (3) to update the internal and external parameters of the camera to be
And Tj(ii) a Substituting the updated camera internal and external parameters into the formula (6), and using the formula (6) to set B2And B3The element in (3) calculates its three-dimensional coordinates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811194104.5A CN109579695B (en) | 2018-10-15 | 2018-10-15 | Part measuring method based on heterogeneous stereoscopic vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811194104.5A CN109579695B (en) | 2018-10-15 | 2018-10-15 | Part measuring method based on heterogeneous stereoscopic vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109579695A CN109579695A (en) | 2019-04-05 |
| CN109579695B true CN109579695B (en) | 2020-12-25 |
Family
ID=65920012
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811194104.5A Active CN109579695B (en) | 2018-10-15 | 2018-10-15 | Part measuring method based on heterogeneous stereoscopic vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109579695B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110470216B (en) * | 2019-07-10 | 2022-01-28 | 湖南交工智能技术有限公司 | Three-lens high-precision vision measurement method and device |
| CN111426268B (en) * | 2020-04-20 | 2021-09-24 | 武汉大学中南医院 | Binocular vision camera matching method |
| CN111721197B (en) * | 2020-05-14 | 2022-02-01 | 南京工程学院 | Body model measuring device and method based on binocular stereo |
| CN111914857B (en) * | 2020-08-11 | 2023-05-09 | 上海柏楚电子科技股份有限公司 | Layout method, device and system for plate excess material, electronic equipment and storage medium |
| CN112237416A (en) * | 2020-09-10 | 2021-01-19 | 北京信息科技大学 | Fundus multi-mode imaging system calibration method based on retinal surface blood vessel characteristics |
| CN112489109B (en) * | 2020-11-19 | 2022-10-21 | 广州视源电子科技股份有限公司 | Three-dimensional imaging system method and device and three-dimensional imaging system |
| CN112991460B (en) * | 2021-03-10 | 2021-09-28 | 哈尔滨工业大学 | Binocular measurement system, method and device for obtaining size of automobile part |
| CN114535825B (en) * | 2022-04-28 | 2022-07-12 | 安普森智能科技(江苏)有限公司 | Laser marking vehicle identification code system based on manipulator |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE202010004039U1 (en) * | 2010-03-22 | 2010-06-17 | Modus High-Tech Electronics Gmbh | Device for optical quality control of three-dimensional objects |
| CN204479038U (en) * | 2015-02-27 | 2015-07-15 | 福建浩蓝光电有限公司 | Novel high resolving power vertical 3 D scanner |
| CN106483642A (en) * | 2016-12-14 | 2017-03-08 | 舜宇光学(中山)有限公司 | A kind of doubly telecentric camera lens based on machine vision |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018162979A1 (en) * | 2017-03-06 | 2018-09-13 | Spectrum Optix Inc. | Diamond shaped lens system |
-
2018
- 2018-10-15 CN CN201811194104.5A patent/CN109579695B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE202010004039U1 (en) * | 2010-03-22 | 2010-06-17 | Modus High-Tech Electronics Gmbh | Device for optical quality control of three-dimensional objects |
| CN204479038U (en) * | 2015-02-27 | 2015-07-15 | 福建浩蓝光电有限公司 | Novel high resolving power vertical 3 D scanner |
| CN106483642A (en) * | 2016-12-14 | 2017-03-08 | 舜宇光学(中山)有限公司 | A kind of doubly telecentric camera lens based on machine vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109579695A (en) | 2019-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109579695B (en) | Part measuring method based on heterogeneous stereoscopic vision | |
| CN109035309B (en) | Stereoscopic vision-based pose registration method between binocular camera and laser radar | |
| CN108734744B (en) | Long-distance large-view-field binocular calibration method based on total station | |
| CN102376089B (en) | Target correction method and system | |
| CN108648232B (en) | Binocular stereoscopic vision sensor integrated calibration method based on precise two-axis turntable | |
| Yang et al. | Robust and accurate surface measurement using structured light | |
| CN109242908B (en) | Calibration method for underwater binocular vision measurement system | |
| CN108986070B (en) | Rock crack propagation experiment monitoring method based on high-speed video measurement | |
| CN102221331B (en) | Measuring method based on asymmetric binocular stereovision technology | |
| CN109341668B (en) | Multi-camera measuring method based on refraction projection model and light beam tracking method | |
| CN109859272A (en) | A kind of auto-focusing binocular camera scaling method and device | |
| CN110044374B (en) | Image feature-based monocular vision mileage measurement method and odometer | |
| CN104075688A (en) | Distance measurement method of binocular stereoscopic gazing monitoring system | |
| Zhang et al. | High-precision measurement of binocular telecentric vision system with novel calibration and matching methods | |
| CN111709985B (en) | Underwater target ranging method based on binocular vision | |
| CN110223355B (en) | Feature mark point matching method based on dual epipolar constraint | |
| CN113592721B (en) | Photogrammetry method, apparatus, device and storage medium | |
| CN103743352A (en) | Three-dimensional deformation measuring method based on multi-camera matching | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN106500625B (en) | A kind of telecentricity stereo vision measurement method | |
| CN110879080A (en) | High-precision intelligent measuring instrument and measuring method for high-temperature forge piece | |
| CN107816942A (en) | A kind of planar dimension measurement method based on cross structure light vision system | |
| CN103983186A (en) | Binocular vision system correcting method and device | |
| CN116051659A (en) | Linear array camera and 2D laser scanner combined calibration method | |
| CN112525106B (en) | Three-phase machine cooperative laser-based 3D detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190405 Assignee: Nanjing University of Engineering Science Park Co.,Ltd. Assignor: NANJING INSTITUTE OF TECHNOLOGY Contract record no.: X2024980001093 Denomination of invention: A Part Measurement Method Based on Heterogeneous Stereoscopic Vision Granted publication date: 20201225 License type: Common License Record date: 20240122 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190405 Assignee: Nanjing Jinxi Yunchuang Technology Co.,Ltd. Assignor: NANJING INSTITUTE OF TECHNOLOGY Contract record no.: X2024980001805 Denomination of invention: A Part Measurement Method Based on Heterogeneous Stereoscopic Vision Granted publication date: 20201225 License type: Common License Record date: 20240202 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190405 Assignee: Nanjing Institute of Engineering Technical Service Co.,Ltd. Assignor: NANJING INSTITUTE OF TECHNOLOGY Contract record no.: X2024980002478 Denomination of invention: A Part Measurement Method Based on Heterogeneous Stereoscopic Vision Granted publication date: 20201225 License type: Common License Record date: 20240305 |