CN112985293B - Binocular vision measurement system and measurement method for single-camera double-spherical mirror image - Google Patents
Binocular vision measurement system and measurement method for single-camera double-spherical mirror image Download PDFInfo
- Publication number
- CN112985293B CN112985293B CN202110049684.4A CN202110049684A CN112985293B CN 112985293 B CN112985293 B CN 112985293B CN 202110049684 A CN202110049684 A CN 202110049684A CN 112985293 B CN112985293 B CN 112985293B
- Authority
- CN
- China
- Prior art keywords
- image
- mirror
- calibration
- spherical mirror
- measurement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention belongs to the technical field of measurement, and provides a binocular vision measurement system and a measurement method for a single-camera double-spherical mirror image. The measuring system of the invention consists of an industrial camera 2, an optical lens 3, a small spherical mirror 6, a large spherical mirror 7, a mechanical bracket 4, a data transmission line 5, a computer 8, a calibration target 11 and a power supply 12. The industrial camera 2 is mirrored via the large spherical mirror 7 and the small spherical mirror 6 as a left virtual camera 13 and a right virtual camera 14, the left virtual camera 13 and the right virtual camera 14 constituting a mirrored binocular. The measuring method of the invention utilizes the calibration target 11 to calibrate imaging model parameters, mirror image dual-purpose structure parameters and an essential matrix of the virtual camera, the measured object 10 is placed in the visual field range 9 of the mirror image dual-purpose sensor 1, is received by the industrial camera 2 after being reflected by the small spherical mirror 6 and the large spherical mirror 7, the formed image is collected into a computer for processing, the image coordinates of the measuring point are extracted, and the three-dimensional coordinates of the measuring point of the measured object are calculated according to the measuring model. The invention enlarges the visual field range of the binocular system of the single camera by spherical mirror reflection imaging, solves the contradiction between the visual field range and the system volume in binocular vision measurement, avoids the synchronous requirement of image acquisition of the binocular vision system formed by two cameras in the prior art, reduces the complexity of binocular vision measurement operation, and can widen the application field of binocular vision measurement.
Description
Field of the art
The invention belongs to the technical field of vision measurement, relates to a large-view-field binocular vision measurement system formed by a single camera, and provides a single-camera double-spherical-mirror-mirrored binocular vision measurement system and a measurement method.
(II) background art
Along with the rapid development of image processing technology and computer technology, a three-dimensional vision precision measurement technology based on machine vision is widely focused by researchers, and binocular stereo vision is widely applied to the fields of intelligent transportation, industrial manufacturing, modern medicine, aerospace and the like as a classical measurement system for non-contact three-dimensional measurement, such as railway train wheel pair measurement, industrial processing part measurement, unmanned aerial vehicle navigation, robot positioning and the like. In recent years, the application requirements of vision measurement are continuously improved, the measurement scene is complicated, and requirements of low cost, miniaturization, light weight, large field of view, rapid measurement and the like are provided for a stereoscopic vision sensor. The mirror image binocular stereoscopic vision measurement system is formed by the single camera and the optical lens group, so that the performance of the binocular sensor can be improved, the system is suitable for complex scenes, and higher measurement requirements are met. In addition, a larger measuring field of view range can be obtained by adopting a visual sensing system of spherical mirror reflection imaging.
The existing binocular stereoscopic vision sensor mainly aims at the three-dimensional measurement requirement under specific conditions of indoor conditions, uniform illumination or less reflection. For different scene conditions, students at home and abroad begin to try to study three-dimensional perception measurement technology under complex scenes, such as using a photometric stereo camera to perceive an indoor scene under weak illumination conditions, and using a binocular stereo vision system to perform target recognition and three-dimensional perception to measure occluded objects in the scene. Microsoft corporation and intel corporation have released Kinect and RealSense cameras in 2010 and 2012, respectively, to be able to acquire RGB and depth images of a scene. However, the existing binocular vision measuring method under different scene conditions has the defects of large sensor volume, high cost, low measuring speed, low efficiency, poor image acquisition synchronism, difficult miniaturization of a measuring system and the like, and cannot meet the requirement of high-precision dynamic three-dimensional precision measurement. The mirror image type binocular vision measurement mode combines the advantages of the traditional binocular stereoscopic vision, has the characteristics of low cost, convenient operation, simple measurement structure, easy miniaturization of the system and the like, meets the strict synchronous acquisition requirements of left and right views in binocular vision measurement, and can be widely applied to high-precision dynamic measurement and various limited space complex scenes. For example, a mirror image binocular vision technology based on plane refraction and reflection is utilized, a single camera is utilized to simultaneously capture images of a measured object from different directions through plane mirror reflection, so that the image acquisition efficiency is improved, the system volume is reduced, but the field of view is halved, and a specific sensor structure is required to be designed for different measured objects and scenes. Mirror image binocular vision based on curved surface refraction and reflection expands the field of view range, but an imaging system of the mirror image binocular vision generally adopts an optical axis collinear structure, so that measurement performance is limited, and an image has inherent curling distortion, is unfavorable for homonymous feature matching in stereoscopic vision, is mostly used for qualitative detection, and is difficult to carry out precise three-dimensional measurement. Therefore, it is necessary to provide a mirror image binocular measuring system, which further miniaturizes the sensor structure, obtains a larger field of view range, establishes a mathematical model to recover the curling image, and realizes quantitative three-dimensional measurement.
(III) summary of the invention
The invention aims to solve the problems that: the system and the method for measuring the binocular vision of the double spherical mirrors of the single camera are provided, so that the mirror binocular vision system obtains a larger field of view range, and the system and the method are suitable for complex scenes and limited spaces. The measuring system consists of a single camera and a pair of double-spherical optical lenses, and the implementation process is divided into a calibration stage and a measuring stage. Is implemented after the design of proper sensor size parameters. And in the calibration stage, an imaging model of the virtual camera and a mirror image dual-purpose three-dimensional measurement model are established, and three-dimensional measurement can be performed after the imaging model of the virtual camera is calibrated and the mirror image dual-purpose structural parameters are calibrated. The measuring method solves the problem that the field of view of the existing single-camera double-plane mirror image binocular system is smaller, enlarges the field of view, solves the contradiction between the mirror image binocular volume and the field of view, and reduces the cost and the operation complexity of the binocular system.
The technical scheme of the invention is as follows: a single-camera double-spherical-mirror image binocular vision measuring system and a measuring method are characterized in that:
1. a binocular vision measuring system of a single camera with a double spherical mirror image is characterized in that,
1.1, the binocular vision measuring system of the mirror image of the double spherical mirrors of the single camera, by mirror image binocular sensor (1), data transmission line (5), computer (8), calibration target (11) and power (12) make up; the mirror image binocular sensor (1) consists of an industrial camera (2), an optical lens (3), a mechanical bracket (4), a small spherical mirror (6) and a large spherical mirror (7); the industrial camera (2) and the optical lens (3) form a single camera, the small spherical mirror (6) and the large spherical mirror (7) form a double spherical mirror, and the axial direction (18) of the double spherical mirror is defined as the intersection point M of the edges of the double spherical mirror to the central axis O of the small spherical mirror (6) 1 E 1 And a central axis O of a large spherical mirror (7) 2 E 2 Is the intersection point J of (2); the data transmission line (5) is connected with the computer (8); the object (10) to be measured is arranged in the field of view range (9) of the mirror image binocular sensor (1) and is 200 to 60 away from the intersection point of the edges of the double-spherical mirror in the axial direction of the double-spherical mirrorIn the range of 0 mm; the mirror image binocular sensor (1) transmits the shot image into a computer (8) for storage and processing;
the geometrical parameters of the mirror image binocular sensor (1) comprise: the distance between the industrial camera (2) and the intersection point M of the edges of the small spherical mirror (6) and the large spherical mirror (7) is 80-150 mm, the mirror surface diameter of the small spherical mirror (6) is 20-50 mm, the curvature radius is 40-500 mm, the edge thickness is 1-10 mm, and the placement angle is defined as the included angle between the bottom plate (16) of the small spherical mirror (6) and the optical axis OM of the lens, and the value is 15-45 degrees; the mirror surface diameter of the large spherical mirror (7) is 30-60 mm, the curvature radius is 40-500 mm, the edge thickness is 1-10 mm, the placement angle is defined as the included angle between the bottom plate (17) of the large spherical mirror (7) and the optical axis OM of the lens, and the value is 45-80 degrees;
1.3, the calibration target (11) is a two-dimensional plane, the target is provided with preset characteristic points, black solid circles which are arranged in a matrix arrangement are arranged on the target plane, the number of the circles is 16-100, the diameter is 2-20 mm, the precision is 0.01mm, the center distance of the circles is 8-20 mm, the precision is 0.01mm, the center of the circles on the target surface is selected as the characteristic points, and the number of the characteristic points is 16-100.
2. The method for three-dimensional measurement by using the single-camera double-spherical mirror image binocular vision measurement system according to claim 1, wherein the measurement process is divided into a calibration stage and a measurement stage, and the measurement can be continuously performed after one calibration, and the specific steps are as follows:
2.1, calibration stage:
2.1.1, fastening the industrial camera (2); the focal length of the optical lens (3) is adjusted, so that the image formed by the measured object (10) which is in the range of 200-600 mm away from the edge intersection point of the double-spherical mirror in the axial direction of the double-spherical mirror is clear; after the adjustment, the optical lens (3) is fastened;
2.1.2, the industrial camera (2) is mirrored into two virtual cameras through a double-spherical mirror, which are respectively called a left virtual camera (13) and a right virtual camera (14); the left virtual camera (13) and the right virtual camera (14) form a mirror image binocular; establishing an imaging model of the virtual camera, wherein main parameters comprise an equivalent focal length and a principal point of the virtual camera and a distortion coefficient of a spherical mirror; calibrating imaging model parameters of a virtual camera, wherein the method comprises the following specific steps of:
the method comprises the steps that at least three positions of a calibration target (11) are freely moved in the field of view of a mirror image binocular sensor (1), an image is shot every time one position is moved, the image is called a double-spherical mirror calibration image, and all characteristic points on the target are contained in the shot image;
secondly, disassembling a left part and a right part in the double-spherical-mirror calibration image from an image middle row into two calibration images with the same size, which are respectively called a left calibration image and a right calibration image; the left calibration image corresponds to the left virtual camera (13), and the right calibration image corresponds to the right virtual camera (14);
thirdly, characteristic points in the left calibration image are called left calibration characteristic points, and image coordinates of the left calibration characteristic points are extracted and correspond to world coordinates of the characteristic points; calibrating a distortion coefficient of the large spherical mirror (7) and an equivalent focal length and a principal point of the left virtual camera (13) by using the left calibration characteristic points;
fourthly, characteristic points in the right calibration image are called right calibration characteristic points, and image coordinates of the right calibration characteristic points are extracted and correspond to world coordinates of the characteristic points; calibrating a distortion coefficient of the small spherical mirror (6) and an equivalent focal length and a principal point of the right virtual camera (14) by using the right calibration characteristic points;
fifthly, spherical aberration correction is carried out on the left calibration image by using the distortion coefficient of the large spherical mirror (7) to obtain a left calibration image without spherical aberration; carrying out spherical distortion correction on the right calibration image by using the distortion coefficient of the small spherical mirror (6) to obtain a right calibration image without spherical distortion;
2.1.3, calibrating the parameters of the mirror image dual-purpose structure and the essential matrix, wherein the specific steps are as follows;
firstly, establishing a mirror image dual-purpose polar line geometric constraint relation, and describing the geometric constraint relation by using an essential matrix; establishing a mirror image dual-purpose three-dimensional measurement model, wherein parameters of the model are mirror image dual-purpose structural parameters, and the mirror image dual-purpose three-dimensional measurement model comprises a three-dimensional coordinate system O of a right virtual camera r -x r y r z r Three-dimensional coordinate system O to left virtual camera l -x l y l z l Is shifted by a rotation matrix and a translation of (a)A vector;
secondly, extracting image coordinates of feature points of the undistorted left calibration image and the undistorted right calibration image respectively, and corresponding to world coordinates of the feature points;
thirdly, calibrating the mirror image dual-purpose structural parameters by using undistorted image coordinates and corresponding world coordinates of all the characteristic points, and calculating a mirror image dual-purpose essential matrix;
2.2, measuring stage:
2.2.1, placing the measured object (10) in a view field range (9) of the mirror image binocular sensor (1), adjusting the measured object (10) within a range of 200-600 mm away from the edge intersection point of the double-spherical mirror in the axial direction of the double-spherical mirror, and ensuring that the measured object is imaged on the left half part and the right half part of an image shot by a camera at the same time, and shooting an image, namely a measurement image;
2.2.2, disconnecting the left and right parts in the measured image from the middle row of the image into two calibration images with the same size, which are respectively called a left measured image and a right measured image; the left measurement image corresponds to a left virtual camera (13), and the right measurement image corresponds to a right virtual camera (14);
2.2.3, performing distortion correction on the left measurement image and the right measurement image obtained in the step 2.2.2 by using the imaging model parameters of the virtual camera calibrated in the step 2.1.2 to obtain an undistorted left measurement image and an undistorted right measurement image;
2.2.4 setting measurement points on the measured object (10), and calculating the measurement points at O l -x l y l z l The three-dimensional coordinates in the coordinate system comprise the following specific steps:
firstly, respectively corresponding image points of the measuring points in an undistorted left measuring image and an undistorted right measuring image are called as homonymous corresponding point pairs, and the undistorted left measuring image coordinates and the undistorted right measuring image coordinates of the homonymous corresponding point pairs are determined through image processing and three-dimensional matching according to mirror image dual-purpose polar line geometric constraint;
second step, at O l -x l y l z l In the coordinate system, according to the mirror image dual-purpose three-dimensional measurement model established in the step 2.1.3,calculating the position of the measuring point in O by using the undistorted image coordinates of the corresponding point pairs with the same name of the measuring point l -x l y l z l Storing three-dimensional coordinates in a data file according to the three-dimensional coordinates in the coordinate system;
2.2.5, repeating the steps 2.2.1-2.2.4, and carrying out three-dimensional measurement of a new measuring point of the measured object.
The invention has the advantages that:
1. the binocular vision measuring system and the measuring method for the double spherical mirror image of the single camera are provided, the double spherical mirror image pair is obtained by the single camera, synchronous acquisition of left and right views of the double spherical mirror image is realized, the cost of the double spherical mirror image measuring system is reduced, and the volume of the measuring system is reduced.
2. The existing double-plane mirror is replaced by the double-spherical mirror to carry out mirror image binocular imaging, the field of view range is enlarged, a virtual camera imaging model is established to solve the inherent curling distortion problem of spherical reflection imaging, and a mirror image double-purpose three-dimensional measuring model is established to realize three-dimensional measurement of a measured object.
3. The single-camera double-sphere mirror image binocular vision sensor is easy to miniaturize and lighten, expands the application range of a binocular system, and can be used for measuring limited space and complex scenes.
(IV) description of the drawings
FIG. 1 is a flow chart of a method for measuring binocular vision of a single camera with a double spherical mirror image;
FIG. 2 is a schematic diagram of a single-camera dual-sphere mirror image binocular vision measurement system;
FIG. 3 is a two-dimensional schematic diagram of the mirror image binocular sensor field of view range, the axial direction of the dual spherical mirror and the optical path;
FIG. 4 is a schematic representation of a target;
FIG. 5 is a schematic diagram of a virtual camera mirror binocular three-dimensional measurement model;
FIG. 6 is a captured dual sphere mirror calibration image;
(fifth) detailed description of the invention
The present invention will be described in further detail below. The invention is based on computer vision and image processing technology, according to the paper 'Three-Dimensional Measurement Approach in Small FOV and Confined Space Using an Electronic Endoscope [ J ]. IEEE Sensors Journal,2014,14 (9): 3274-3282', a two-plane mirror single-camera mirror binocular stereoscopic vision system is proposed, improvement is carried out on the basis, a mirror image binocular vision sensor consisting of a single camera and a double-spherical mirror is designed, an imaging model of a virtual camera is established, a spherical curling distortion image captured by the mirror image binocular sensor is unfolded, namely distortion correction is carried out, a mirror image double-purpose Three-dimensional measurement model is established, and Three-dimensional measurement is realized after parameter calibration of each model is completed.
The structure of the mirror image binocular sensor and the system light path design are shown in fig. 3, the measured scene is reflected by the two spherical mirrors and then enters a single camera for imaging, and the optical axis direction and the view field direction of an optical lens in the single camera are not co-oriented. According to each component and layout, the design method comprises two parts of spherical mirror parameter design and mechanical structure parameter design. In the design of spherical mirror parameters, the influence of the placement angles, the lens diameters and the curvature radiuses of the small spherical mirror (6) and the large spherical mirror (7) on the field of view range of the sensor is analyzed, mathematical relations are deduced, and the optical structural parameters of the small spherical mirror (6) and the large spherical mirror (7) are determined. In the design of mechanical structure parameters, a mechanical support (4) is designed according to the actual sizes of the selected industrial camera (2) and the optical lens (3) by combining spherical mirror parameters. And finally, a single-camera double-spherical mirror image binocular vision measuring system is formed by a small spherical mirror (6), a large spherical mirror (7), a mechanical support (4), an industrial camera (2) and an optical lens (3).
The invention aims at a designed single-camera double-spherical mirror image binocular vision measurement system, and establishes an imaging model of a virtual camera and an image double-purpose three-dimensional measurement model, as shown in fig. 5. The imaging model main parameters of the virtual camera comprise the equivalent focal length and principal point of the virtual camera and the distortion coefficient of the spherical mirror. The method comprises the steps of taking a calibration target image shot by a mirror image binocular sensor as a double-spherical-lens calibration image, disassembling a left part and a right part in the double-spherical-lens calibration image from an image middle row into two calibration images with the same size, namely a left calibration image and a right calibration image, wherein the left calibration image corresponds to a left virtual camera (13), and the right calibration image corresponds to a right virtual camera (14). The perspective projection models of the left and right virtual cameras are as follows:
wherein, (x) vl ,y vl ) For the image coordinates of the feature points in the left calibration image, A vl Is an internal reference matrix of the left virtual camera (13), f vxl ,f vyl For the effective focal length of the left virtual camera (13) in the x, y direction, (u) 0vl ,v 0vl ) Is the principal point coordinates of the left virtual camera (13), (R) vl |T vl ) To target at O l -x l y l z l A rotation matrix and a translation vector in a coordinate system; (x) vr ,y vr ) For the image coordinates of the feature points in the left calibration image, A vr Is an internal reference matrix of the right virtual camera (14), f vxr ，f vyr For the effective focal length of the right virtual camera (14) in the x, y direction, (u) 0vr ,v 0vr ) Is the principal point coordinates of the right virtual camera (14); (R) vr |T vr ) To target at O r -x r y r z r Rotation matrix and translation vector in the coordinate system.
Considering lens distortion of the left virtual camera (13) and the right virtual camera (14), a lens distortion model is:
wherein, (x) l ,y l ) Ideal image coordinates imaged on the left virtual camera for the feature point, (x) dl ,y dl ) For the actual image coordinates of the feature points imaged by the left virtual camera, r is the distance between the actual image coordinates and the virtual camera principal point without spherical reflection distortion, (k) l1 ,k l2 ) First and second radial distortion coefficients for left virtual camera lens, (p) l1 ,p l2 ) The first and second tangential distortion coefficients are the left virtual camera lens; (x) r ,y r ) Ideal image coordinates for imaging feature points in left virtual camera，(x dr ,y dr ) For the actual image coordinates of the feature points imaged by the left virtual camera, r is the distance between the actual image coordinates and the virtual camera principal point without spherical reflection distortion, (k) r1 ,k r2 ) First and second radial distortion coefficients for left virtual camera lens, (p) r1 ,p r2 ) Is the first and second tangential distortion coefficients of the left virtual camera lens.
The distortion of the spherical mirror can be regarded as being formed by combining radial distortion and tangential distortion, and then the model is as follows:
wherein, (x) dl ,y dl ) Left calibration image coordinates without spherical distortion, (x) 0l ,y 0l ) For actually calibrating the image coordinates to the left, r 0l For the distance between the actual left calibration image coordinates and the left virtual camera principal point, (k) sl1 ,k sl2 ,k sl3 ) Is the radial distortion coefficient of the large spherical mirror (7), (p) sl1 ,p sl2 ) Is the tangential distortion coefficient of the large spherical mirror (7); (x) dr ,y dr ) Right calibration image coordinates (x) without spherical reflection distortion 0r ,y 0r ) For actually calibrating the image coordinates to the left, r 0r For the distance between the actual right calibration image coordinates and the right virtual camera principal point, (k) sr1 ,k sr2 ,k sr3 ) Is the radial distortion coefficient of the small spherical mirror (6), (p) sr1 ,p sr2 ) Is the tangential distortion coefficient of the small spherical mirror (6).
Establishing mirror image dual-purpose polar line geometric constraint relation, expressing by using an essential matrix, wherein the description method is described in paper 'Precise calibration of binocular vision system used for vision measurement' of Cui Yi [ Optics Express, vol.20, no.8,2014 ]]The method comprises the steps of carrying out a first treatment on the surface of the Establishing a mirror image dual-purpose three-dimensional measurement model, wherein main parameters comprise O r -x r y r z r Coordinate system to O l -x l y l z l The rotation matrix R and translation vector T of the coordinate system.
Failure of a single virtual cameraAnd recovering the three-dimensional coordinates of the circle centers of the characteristic points through the image coordinates. The mirror image binocular stereoscopic vision system can calculate three-dimensional coordinates of the feature points through coordinates of the feature points in the two virtual camera images according to a triangulation principle. Selecting O l -x l y l z l The coordinate system is used as a measurement coordinate system, and the mirror image dual-purpose three-dimensional measurement model is as follows:
wherein lambda is vl ,λ vr Is a scale factor; a is that vl ,A vr Is the parameter matrix in the left and right virtual cameras, x l (u l ,v l ),x vr (u vr ,v vr ) And the pixel coordinates of the corresponding points of the images of the left virtual camera (13) and the right virtual camera (14) are respectively. R, T are virtual binocular structural parameters, representing O r -x r y r z r Coordinate system and O l -x l y l z l Conversion relation between coordinate systems.
And O is l -x l y l z l Coordinate system and O r -x r y r z r The mutual pose relationship between the coordinate systems can be represented by matrix rotation:
according to the measuring system and mathematical model of the single-camera double-spherical mirror image binocular vision sensor, the measuring flow of the single-camera double-spherical mirror image binocular vision measuring system is shown in figure 1, and the specific steps are as follows:
1. -fastening an industrial camera (2); the focal length of the optical lens (3) is adjusted, so that the image formed by the measured object (10) which is in the range of 200-600 mm away from the edge intersection point of the double-spherical mirror in the axial direction of the double-spherical mirror is clear; after the adjustment, the optical lens (3) is fastened;
2. freely moving at least three positions of a calibration target (11) within the field of view of the mirror image binocular sensor (1), shooting an image every time one position is moved, namely a double-spherical mirror calibration image, wherein all characteristic points on the target are contained in the shot image;
3. the left part and the right part in the double-spherical mirror calibration image are disassembled from the middle row of the image into two calibration images with the same size; the left calibration image corresponds to the left virtual camera (13), and the right calibration image corresponds to the right virtual camera (14);
4. according to formulas [1,2], the distortion coefficient of the large spherical mirror (7) and the equivalent focal length and principal point of the left virtual camera (13) are calibrated by using the left calibration characteristic point, the distortion coefficient of the small spherical mirror (6) and the equivalent focal length and principal point of the right virtual camera (14) are calibrated by using the right calibration characteristic point, and the calibration method is shown in Zhang Zhengyou paper [ A flexible new technique for camera calibration ] [ IEEE Transctions on Pattern Analysis and Machine Intelligence, vol.22, no.11,2000];
5. according to the formula [3], spherical aberration correction is carried out on the left calibration image by using the distortion coefficient of the large spherical mirror (7) to obtain a left calibration image without spherical aberration; carrying out spherical distortion correction on the right calibration image by using the distortion coefficient of the small spherical mirror (6) to obtain a right calibration image without spherical distortion;
6. respectively extracting image coordinates of characteristic points of the left calibration image without spherical distortion and the right calibration image without spherical distortion, and corresponding to world coordinates of the characteristic points;
7. according to the proposed mirrored dual-purpose three-dimensional measurement model (equations [4,5 ]]) Calibrating mirror image dual-purpose structural parameters by using the image coordinates of all the characteristic points extracted in the step (10) and the corresponding world coordinates, and calculating an essential matrix to obtain O r -x r y r z r Coordinate system to O l -x l y l z l A rotation matrix and a translation vector of the coordinate system;
8. according to one embodiment of the invention, a measurement is performed on an object (10);
9. placing a measured object (10) in a view field range (9) of a mirror image binocular sensor (1), displaying a real-time image captured by the sensor in a computer (8) within a range of 200-600 mm from an intersection point of the edges of the double spherical mirrors in the axial direction of the double spherical mirrors, adjusting the measured object (10) to ensure that the left part and the right part of the displayed image are clear and complete, and shooting an image as a measurement image; the left and right parts in the measurement image are disassembled from the middle row of the image to form two measurement images, and the two measurement images are used as a left measurement image and a right measurement image; extracting points on the measurement image by using a SIFT algorithm, and setting the points as measurement points on the measured object (10), wherein the extraction method is shown in David G.Lowe 'Distinctive image features from scale invariant keypoints' [ International Journal of Computer Vision,2004] ";
10. sequentially carrying out spherical aberration correction on the left and right measurement images obtained in the step 9 by using calibrated imaging model parameters of the virtual camera to obtain a left measurement image without spherical aberration and a right measurement image without spherical aberration;
11. in the left and right measurement images without spherical distortion obtained in the step 10, extracting the image coordinates of the measurement points of the measured object, carrying out stereo matching according to the geometric constraint of the virtual binocular stereo vision polar lines, determining the left measurement image coordinates without distortion and the right measurement image coordinates without distortion of the corresponding point pairs with the same name, and obtaining the left and right measurement images with the same name through formulas [4,5 ]]Built mirror image dual-purpose three-dimensional measurement model, O is calculated r -x r y r z r Straight line and O determined by origin of coordinate system and projection point of measuring point l -x l y l z l The least squares intersection of the determined straight lines of the origin of the coordinate system and the measurement point is calculated by the method described in paper "Precise calibration of binocular vision system used for vision measurement" by Cui Yi [ Optics Express, vol.22, no.8,2014 ]]Obtaining the measuring point at O l -x l y l z l And (3) completing the measurement of the measuring point of the measured object (10) by three-dimensional coordinates under the coordinate system.
(sixth) example
A large-constant-water-star series MER-301-125U3C type camera and a computer V1228-MPY model 12mm fixed focus lens are adopted. The industrial camera resolution is 2048×1236 pixels. The method comprises the steps of selecting a spherical mirror reflection surface material to be aluminized, enabling the mirror surface size diameter of a spherical mirror 6 to be 35.18mm, enabling the curvature radius to be 200mm, enabling the edge thickness to be 1.60mm, enabling the placement angle to be 37 degrees, enabling the mirror surface size diameter of the spherical mirror 7 to be 47.37mm, enabling the curvature radius to be 200mm, enabling the edge thickness to be 1.60mm, enabling the placement angle to be 53 degrees, enabling the distance d between a single camera and the edge of the bottom surface of the double-spherical mirror to be 100mm, defining the angle of the x-axis direction in FIG. 3 to be 0 degree, enabling the field of view range of the mirror image binocular sensor to be [ -10.46 degrees, 13.91 degrees ] and enabling the field angle to be 24.37 degrees. The size of the sensor base plate is selected to be 70mm multiplied by 220mm multiplied by 10mm, the size of the spherical mirror base plate is selected to be 50mm multiplied by 5mm and 38mm multiplied by 5mm, and the size phi of the camera fixing hole is selected to be 6.6mm multiplied by 6mm.
Visual measurement verification is carried out on the designed and processed mirror image binocular sensor, a two-dimensional plane target is adopted for calibration, the target is placed in the range of 600mm away from the intersection point of the bottom surface edge of the double-spherical mirror in the axial direction of the double-spherical mirror, 20 positions of the target are freely moved and calibrated, and an image is shot when one position is moved. The number of the circular feature points on the target is 49, the distance between the centers of circles is 12.50mm, the diameter of the dots is 6.25mm, and the precision is 0.01mm.
Fig. 6 is a mirror image binocular image acquired by a mirror image binocular sensor, and according to an established imaging model of a virtual camera, left and right parts in the acquired double-sphere calibration image are separated from an image middle column into two calibration images with the same size. The equivalent focal length and principal point of the virtual camera and the distortion coefficient of the lens of the virtual camera are obtained through the imaging model calibration of the virtual camera, and the internal parameter results of the calibrated left virtual camera are as follows:
f vxl ＝1412.50pixels，f vyl ＝1528.92pixels
u 0vl ＝897.39pixels，v 0vl ＝740.31pixels
k l1 ＝-0.25，k l2 ＝0.22，p l1 ＝-0.00，p l2 ＝-0.10
the right virtual camera internal parameter results are as follows:
f vxr ＝1625.01pixels，f vyr ＝2115.20pixels
u 0vr ＝450.27pixels，v 0vr ＝841.79pixels
k r1 ＝0.32，k r2 ＝-16.09，p r1 ＝-0.06，p r2 ＝214.90
the distortion coefficient of the small spherical mirror (6) is as follows:
k sl1 ＝-2.06，k sl2 ＝27.93，k sl3 ＝-139.34，p sl1 ＝-0.02，p sl2 ＝-0.05
the distortion coefficient of the large spherical mirror (7) is as follows:
k sl1 ＝0.14，k sl2 ＝-0.41，k sl3 ＝0.41，p sl1 ＝0.01，p sl2 ＝-0.13
and carrying out distortion correction on the left calibration image and the right calibration image according to the obtained imaging model parameters of the virtual camera to obtain left and right calibration images without spherical distortion. The mirror image dual-purpose three-dimensional measurement model calibration is carried out by utilizing left and right calibration images without spherical distortion, which comprises O r -x r y r z r Coordinate system to O l -x l y l z l The rotation matrix and translation vector structure parameters of the coordinate system result as follows:
the calibration and measurement results show that the image error without spherical reflection distortion correction is larger, the image calibration accuracy after spherical distortion correction is improved, the left virtual camera calibration error is about 0.10 pixel, the right virtual camera calibration error is about 0.12 pixel, the mirror image double-target calibration error is 2.48 pixels, the feasibility of the imaging model and the mirror image double-purpose three-dimensional measurement model of the mirror image double-purpose sensor and the proposed virtual camera is verified through experiments, and the proposed spherical distortion correction has the effect of improving the inherent curling distortion of the spherical mirror reflection imaging. The binocular vision sensor solves the contradiction between the volume of the mirror binocular system and the field of view, so that the sensor is further miniaturized, a larger field of view range is obtained, a mathematical model is built to recover a curling distortion image, and three-dimensional measurement is realized.
Claims (2)
1. A binocular vision measuring system of a single camera with a double spherical mirror image is characterized in that,
1.1, the binocular vision measuring system of the mirror image of the double spherical mirrors of the single camera, by mirror image binocular sensor (1), data transmission line (5), computer (8), calibration target (11) and power (12) make up; the mirror image binocular sensor (1) consists of an industrial camera (2), an optical lens (3), a mechanical bracket (4), a small spherical mirror (6) and a large spherical mirror (7); the industrial camera (2) and the optical lens (3) form a single camera, the small spherical mirror (6) and the large spherical mirror (7) form a double spherical mirror, and the axial direction (18) of the double spherical mirror is defined as the intersection point M of the edges of the double spherical mirror to the central axis O of the small spherical mirror (6) 1 E 1 And a central axis O of a large spherical mirror (7) 2 E 2 Is the intersection point J of (2); the data transmission line (5) is connected with the computer (8); the measured object (10) is arranged in the view field range (9) of the mirror image binocular sensor (1) and is 200-600 mm away from the intersection point of the edges of the double spherical mirrors in the axial direction of the double spherical mirrors; the mirror image binocular sensor (1) transmits the shot image into a computer (8) for storage and processing;
the geometrical parameters of the mirror image binocular sensor (1) comprise: the distance between the industrial camera (2) and the intersection point M of the edges of the small spherical mirror (6) and the large spherical mirror (7) is 80-150 mm, the mirror surface diameter of the small spherical mirror (6) is 20-50 mm, the curvature radius is 40-500 mm, the edge thickness is 1-10 mm, and the placement angle is defined as the included angle between the bottom plate (16) of the small spherical mirror (6) and the optical axis OM of the lens, and the value is 15-45 degrees; the diameter of the mirror surface of the large spherical mirror 7 is 30-60 mm, the radius of curvature is 40-500 mm, the edge thickness is 1-10 mm, the placement angle is defined as the included angle between the bottom plate (17) of the large spherical mirror (7) and the optical axis OM of the lens, and the value is 45-80 degrees;
1.3, the calibration target (11) is a two-dimensional plane, the target is provided with preset characteristic points, black solid circles which are arranged in a matrix arrangement are arranged on the target plane, the number of the circles is 16-100, the diameter is 2-20 mm, the precision is 0.01mm, the center distance of the circles is 8-20 mm, the precision is 0.01mm, the center of the circles on the target surface is selected as the characteristic points, and the number of the characteristic points is 16-100.
2. The method for three-dimensional measurement by using the single-camera double-spherical mirror image binocular vision measurement system according to claim 1, wherein the measurement process is divided into a calibration stage and a measurement stage, and the measurement can be continuously performed after one calibration, and the specific steps are as follows:
2.1, calibration stage:
2.1.1, fastening the industrial camera (2); the focal length of the optical lens (3) is adjusted, so that the image formed by the measured object (10) which is in the range of 200-600 mm away from the intersection point of the edges of the double spherical mirrors in the axial direction of the double spherical mirrors is clear; after the adjustment, the optical lens (3) is fastened;
2.1.2, the industrial camera (2) is mirrored into two virtual cameras through a double-spherical mirror, which are respectively called a left virtual camera (13) and a right virtual camera (14); the left virtual camera (13) and the right virtual camera (14) form a mirror image binocular; establishing an imaging model of the virtual camera, wherein main parameters comprise an equivalent focal length and a principal point of the virtual camera and a distortion coefficient of a spherical mirror; calibrating imaging model parameters of a virtual camera, wherein the method comprises the following specific steps of:
the method comprises the steps that at least three positions of a calibration target (11) are freely moved in the field of view of a mirror image binocular sensor (1), an image is shot every time one position is moved, the image is called a double-spherical mirror calibration image, and all characteristic points on the target are contained in the shot image;
secondly, disassembling a left part and a right part in the double-spherical-mirror calibration image from an image middle row into two calibration images with the same size, which are respectively called a left calibration image and a right calibration image; the left calibration image corresponds to the left virtual camera (13), and the right calibration image corresponds to the right virtual camera (14);
thirdly, characteristic points in the left calibration image are called left calibration characteristic points, and image coordinates of the left calibration characteristic points are extracted and correspond to world coordinates of the characteristic points; calibrating a distortion coefficient of the large spherical mirror (7) and an equivalent focal length and a principal point of the left virtual camera (13) by using the left calibration characteristic points;
fourthly, characteristic points in the right calibration image are called right calibration characteristic points, and image coordinates of the right calibration characteristic points are extracted and correspond to world coordinates of the characteristic points; calibrating a distortion coefficient of the small spherical mirror (6) and an equivalent focal length and a principal point of the right virtual camera (14) by using the right calibration characteristic points;
fifthly, carrying out distortion correction on the left calibration image by using a distortion coefficient of the large spherical mirror (7) to obtain a distortion-free left calibration image; carrying out distortion correction on the right calibration image by using the distortion coefficient of the small spherical mirror (6) to obtain a distortion-free right calibration image;
2.1.3, calibrating the parameters and the essential matrix of the mirror image dual-purpose structure, and specifically comprises the following steps:
firstly, establishing a mirror image dual-purpose polar line geometric constraint relation, and describing the geometric constraint relation by using an essential matrix; establishing a mirror image dual-purpose three-dimensional measurement model, wherein parameters of the model are mirror image dual-purpose structural parameters, and the mirror image dual-purpose three-dimensional measurement model comprises a three-dimensional coordinate system O of a right virtual camera r -x r y r z r Three-dimensional coordinate system O to left virtual camera l -x l y l z l A rotation matrix and a translation vector of (a);
secondly, extracting image coordinates of feature points of the undistorted left calibration image and the undistorted right calibration image respectively, and corresponding to world coordinates of the feature points;
thirdly, calibrating the mirror image dual-purpose structural parameters by using undistorted image coordinates and corresponding world coordinates of all the characteristic points, and calculating a mirror image dual-purpose essential matrix;
2.2, measuring stage:
2.2.1, placing the measured object (10) in a view field range (9) of the mirror image binocular sensor (1), adjusting the measured object (10) within a range of 200-600 mm away from the edge intersection point of the double-spherical mirror in the axial direction of the double-spherical mirror, and ensuring that the measured object is imaged on the left half part and the right half part of an image shot by a camera at the same time, and shooting an image, namely a measurement image;
2.2.2, the left and right parts in the measured image are separated from the middle row of the image into two measured images with the same size, which are respectively called a left measured image and a right measured image; the left measurement image corresponds to a left virtual camera (13), and the right measurement image corresponds to a right virtual camera (14);
2.2.3, performing distortion correction on the left measurement image and the right measurement image obtained in the step 2.2.2 by using the imaging model parameters of the virtual camera calibrated in the step 2.1.2 to obtain an undistorted left measurement image and an undistorted right measurement image;
2.2.4 setting measurement points on the measured object (10), and calculating the measurement points at O l -x l y l z l The three-dimensional coordinates in the coordinate system comprise the following specific steps:
firstly, respectively corresponding image points of the measuring points in an undistorted left measuring image and an undistorted right measuring image are called as homonymous corresponding point pairs, and the undistorted left measuring image coordinates and the undistorted right measuring image coordinates of the homonymous corresponding point pairs are determined through image processing and three-dimensional matching according to mirror image dual-purpose polar line geometric constraint;
second step, at O l -x l y l z l In the coordinate system, according to the mirror image dual-purpose three-dimensional measurement model established in the step 2.1.3, calculating the position of the measurement point in O by using the undistorted image coordinates of the corresponding point pairs with the same name of the measurement point l -x l y l z l Storing three-dimensional coordinates in a data file according to the three-dimensional coordinates in the coordinate system;
2.2.5, repeating the steps 2.2.1-2.2.4, and carrying out three-dimensional measurement of a new measuring point of the measured object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110049684.4A CN112985293B (en) | 2021-01-14 | 2021-01-14 | Binocular vision measurement system and measurement method for single-camera double-spherical mirror image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110049684.4A CN112985293B (en) | 2021-01-14 | 2021-01-14 | Binocular vision measurement system and measurement method for single-camera double-spherical mirror image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112985293A CN112985293A (en) | 2021-06-18 |
| CN112985293B true CN112985293B (en) | 2023-05-30 |
Family
ID=76344366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110049684.4A Active CN112985293B (en) | 2021-01-14 | 2021-01-14 | Binocular vision measurement system and measurement method for single-camera double-spherical mirror image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112985293B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112595237A (en) * | 2020-12-01 | 2021-04-02 | 中国石油天然气股份有限公司 | Oil pipe length measuring device and method based on machine vision |
| CN113888651B (en) * | 2021-10-21 | 2024-06-18 | 天津市计量监督检测科学研究院电子仪表实验所 | Dynamic and static visual detection system |
| CN113989105B (en) * | 2021-10-26 | 2022-08-16 | 季华实验室 | Single-camera spherical mirror reflection imaging projection device |
| CN113902910B (en) * | 2021-12-10 | 2022-04-08 | 中国科学院自动化研究所 | Vision measurement method and system |
| CN114964051B (en) * | 2022-05-23 | 2024-09-20 | 合肥工业大学 | Structured light 3D scanning system based on virtual binocular vision |
| CN115797459B (en) * | 2022-08-29 | 2024-02-13 | 南京航空航天大学 | Binocular vision system ranging method with arbitrary focal length combination |
| CN115711589B (en) * | 2022-11-22 | 2023-12-22 | 哈尔滨工业大学 | Method for measuring spherical profile of rotor of large-sized high-speed rotary equipment based on multi-dimensional large-circle projection center integration |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011160491A1 (en) * | 2010-06-23 | 2011-12-29 | 北京航空航天大学 | Self-adaptive micro vision measuring method based on camera coordinate positions |
| CN102788559A (en) * | 2012-07-19 | 2012-11-21 | 北京航空航天大学 | Optical vision measuring system with wide-field structure and measuring method thereof |
| CN106969723A (en) * | 2017-04-21 | 2017-07-21 | 华中科技大学 | High speed dynamic object key point method for three-dimensional measurement based on low speed camera array |
| CN109357632A (en) * | 2018-12-26 | 2019-02-19 | 河北工业大学 | A kind of mirror article 3 D measuring method and device |
-
2021
- 2021-01-14 CN CN202110049684.4A patent/CN112985293B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011160491A1 (en) * | 2010-06-23 | 2011-12-29 | 北京航空航天大学 | Self-adaptive micro vision measuring method based on camera coordinate positions |
| CN102788559A (en) * | 2012-07-19 | 2012-11-21 | 北京航空航天大学 | Optical vision measuring system with wide-field structure and measuring method thereof |
| CN106969723A (en) * | 2017-04-21 | 2017-07-21 | 华中科技大学 | High speed dynamic object key point method for three-dimensional measurement based on low speed camera array |
| CN109357632A (en) * | 2018-12-26 | 2019-02-19 | 河北工业大学 | A kind of mirror article 3 D measuring method and device |
Non-Patent Citations (1)
| Title |
|---|
| 镜像式单摄像机双目视觉传感器的结构设计;杨;周富强;;机械工程学报(第22期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112985293A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112985293B (en) | Binocular vision measurement system and measurement method for single-camera double-spherical mirror image | |
| CN103759670B (en) | A kind of object dimensional information getting method based on numeral up short | |
| CN105698699B (en) | A kind of Binocular vision photogrammetry method based on time rotating shaft constraint | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| CN108986070B (en) | Rock crack propagation experiment monitoring method based on high-speed video measurement | |
| CN110728715A (en) | Camera angle self-adaptive adjusting method of intelligent inspection robot | |
| CN107084671B (en) | A kind of recessed bulb diameter measuring system and measurement method based on three wire configuration light | |
| CN102810205A (en) | Method for calibrating camera shooting or photographing device | |
| CN110378969A (en) | A kind of convergence type binocular camera scaling method based on 3D geometrical constraint | |
| CN110827360B (en) | Photometric stereo measurement system and method for calibrating light source direction thereof | |
| CN104165598B (en) | Automatic reflection light spot positioning method for large-caliber mirror interferometer vertical type detection | |
| CN113724337B (en) | Camera dynamic external parameter calibration method and device without depending on tripod head angle | |
| CN112229323B (en) | Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method | |
| CN111879354A (en) | Unmanned aerial vehicle measurement system that becomes more meticulous | |
| CN108230401A (en) | 3D four-wheel position finder automatic camera calibration method and system | |
| CN109084959B (en) | Optical axis parallelism correction method based on binocular distance measurement algorithm | |
| CN105374067A (en) | Three-dimensional reconstruction method based on PAL cameras and reconstruction system thereof | |
| CN116740187A (en) | Multi-camera combined calibration method without overlapping view fields | |
| CN114396894A (en) | Pipeline straightness detection device and method based on laser ranging and monocular vision | |
| CN117197241B (en) | Robot tail end absolute pose high-precision tracking method based on multi-eye vision | |
| CN111998823B (en) | Target ranging method based on binocular different-light-source ranging device | |
| CN111754584A (en) | Remote large-field-of-view camera parameter calibration system and method | |
| CN117419640A (en) | Space point three-dimensional coordinate measuring method based on binocular stereoscopic vision | |
| CN111998834A (en) | Crack monitoring method and system | |
| CN109712200B (en) | Binocular positioning method and system based on least square principle and side length reckoning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |