CN112985293B - A single-camera double-spherical mirror image binocular vision measurement system and measurement method - Google Patents

Abstract

The invention belongs to the technical field of measurement, and provides a binocular vision measurement system and a measurement method for a single-camera double-spherical mirror image. The measuring system of the invention consists of an industrial camera 2, an optical lens 3, a small spherical mirror 6, a large spherical mirror 7, a mechanical bracket 4, a data transmission line 5, a computer 8, a calibration target 11 and a power supply 12. The industrial camera 2 is mirrored via the large spherical mirror 7 and the small spherical mirror 6 as a left virtual camera 13 and a right virtual camera 14, the left virtual camera 13 and the right virtual camera 14 constituting a mirrored binocular. The measuring method of the invention utilizes the calibration target 11 to calibrate imaging model parameters, mirror image dual-purpose structure parameters and an essential matrix of the virtual camera, the measured object 10 is placed in the visual field range 9 of the mirror image dual-purpose sensor 1, is received by the industrial camera 2 after being reflected by the small spherical mirror 6 and the large spherical mirror 7, the formed image is collected into a computer for processing, the image coordinates of the measuring point are extracted, and the three-dimensional coordinates of the measuring point of the measured object are calculated according to the measuring model. The invention enlarges the visual field range of the binocular system of the single camera by spherical mirror reflection imaging, solves the contradiction between the visual field range and the system volume in binocular vision measurement, avoids the synchronous requirement of image acquisition of the binocular vision system formed by two cameras in the prior art, reduces the complexity of binocular vision measurement operation, and can widen the application field of binocular vision measurement.

Description

A single-camera double-spherical mirror image binocular vision measurement system and measurement method

(1) Technical field

The invention belongs to the technical field of visual measurement and relates to a large field of view binocular vision measurement system composed of a single camera, and provides a single camera double spherical mirror image binocular vision measurement system and a measurement method.

(2) Background technology

With the rapid development of image processing technology and computer technology, the three-dimensional visual precision measurement technology based on machine vision has attracted extensive attention from researchers. As a classic measurement system of non-contact three-dimensional measurement, binocular stereo vision has been widely used in intelligent transportation, Industrial manufacturing, modern medicine, aerospace and other fields, such as railway train wheel set measurement, industrial processing parts measurement, drone navigation, robot positioning, etc. In recent years, the application requirements of visual measurement have been continuously improved, and the measurement scenarios have become more complicated. The stereo vision sensor has put forward requirements such as low cost, miniaturization, light weight, large field of view, and fast measurement. Among them, the mirror binocular stereo vision measurement system is composed of a single camera and an optical mirror group, which can improve the performance of the binocular sensor, is suitable for complex scenes, and meets higher measurement requirements. In addition, the visual sensing system using spherical mirror reflection imaging can obtain a larger measurement field of view.

The current binocular stereo vision sensor is mainly aimed at the three-dimensional measurement requirements under specific conditions of indoor conditions, uniform illumination or less reflection. For different scene conditions, scholars at home and abroad have begun to try to study 3D perception measurement technology in complex scenes, such as using photometric stereo cameras to perceive indoor scenes under low light conditions, and using binocular stereo vision systems for target recognition and 3D perception measurement. Occluded objects in the scene. Microsoft and Intel released the Kinect and RealSense cameras in 2010 and 2012 respectively, capable of capturing RGB and depth images of a scene. However, the existing binocular vision measurement methods under different scene conditions have the disadvantages of large sensor volume, high cost, slow measurement speed, low efficiency, poor synchronization of image acquisition, and difficulty in miniaturization of the measurement system, which cannot meet high-precision dynamic Three-dimensional precision measurement requirements. The mirror-type binocular vision measurement mode combines the advantages of traditional binocular stereo vision, and at the same time has the characteristics of low cost, convenient operation, simple measurement structure, and easy miniaturization of the system, which satisfies the strict synchronous acquisition of left and right views in binocular vision measurement It can be widely used in high-precision dynamic measurement and various complex scenes with limited space. For example, the mirror binocular vision technology based on plane catadioptric reflection uses a single camera to simultaneously capture images of the measured object from different directions through plane mirror reflection, which improves the image acquisition efficiency and reduces the system volume, but the field of view is halved. For different measurements Objects and scenes require the design of specific sensor structures. Mirrored binocular vision based on surface catadioptric reflection will expand the field of view, but its imaging system usually adopts a collinear optical axis structure, which limits the measurement performance, and the image has inherent curl distortion, which is not conducive to the matching of features with the same name in stereo vision. It is difficult to carry out precise three-dimensional measurement because of qualitative detection. Therefore, it is necessary to propose a mirror binocular measurement system, which can further miniaturize the sensor structure and obtain a larger field of view, and establish a mathematical model to restore the warped image and realize quantitative three-dimensional measurement.

(3) Contents of the invention

The problem to be solved by the present invention is to provide a single camera double spherical mirror mirror binocular vision measurement system and measurement method, so that the mirror binocular measurement system can obtain a larger field of view range, which is suitable for complex scenes and limited spaces. The measurement system consists of a single camera and a pair of double-spherical optical lenses, and the implementation process is divided into a calibration stage and a measurement stage. Implemented after proper sensor size parameter design. In the calibration stage, the imaging model of the virtual camera and the three-dimensional measurement model of the mirrored binoculars are established, and the three-dimensional measurement can be carried out after the calibration of the imaging model of the virtual camera and the calibration of the structural parameters of the mirrored binoculars. This measurement method solves the problem that the field of view of the existing single-camera dual-plane mirrored binocular system is small, expands the field of view, solves the contradiction between the volume of the mirrored binocular and the range of the field of view, and reduces the size of the binocular system. cost and operational complexity.

The technical solution of the present invention is: a single camera double spherical mirror image binocular vision measurement system and measurement method, characterized in that:

1, a kind of binocular visual measurement system of double-spherical mirror mirror image of single camera, it is characterized in that,

1.1, the described binocular visual measuring system of double-spherical mirror mirror image of single camera, is made up of mirror binocular sensor (1), data transmission line (5), computer (8), calibration target (11) and power supply (12); mirror image Binocular sensor (1) consists of industrial camera (2), optical lens (3), mechanical support (4), small spherical mirror (6) and large spherical mirror (7); industrial camera (2) and optical lens (3) Single camera, small spherical mirror (6) and large spherical mirror (7) constitute double spherical mirror, double spherical mirror axial direction (18) is defined as double spherical mirror edge intersection point M to small spherical mirror (6) central axis O ₁ E ₁ and large spherical mirror ( 7) Intersection J of the central axis O ₂ E ₂ ; the data transmission line (5) is connected to the computer (8); the measured object (10) is placed in the field of view (9) of the mirror binocular sensor (1), In the axial direction of the double spherical surface, it is within the range of 200-600 mm from the intersection point of the edge of the double spherical mirror; the mirror image binocular sensor (1) transmits the captured image to the computer (8) for storage and processing;

1.2. The geometric parameters of the mirror image binocular sensor (1) include: the distance between the industrial camera (2) and the edge intersection M of the small spherical mirror (6) and the large spherical mirror (7) is 80-150mm, and the small spherical mirror (6) The diameter of the mirror surface is 20-50mm, the radius of curvature is 40-500mm, and the edge thickness is 1-10mm. The placement angle is defined as the angle between the bottom plate (16) of the small spherical mirror (6) and the optical axis OM of the lens, and the value is 15°～45°; the mirror surface diameter of the large spherical mirror (7) is 30～60mm, the radius of curvature is 40～500mm, and the edge thickness is 1～10mm. The angle between the optical axes OM is 45°～80°;

1.3. The calibration target (11) is a two-dimensional plane with preset feature points arranged on the target plane as black solid circles arranged in a matrix. The number of circles is 16 to 100, and the diameter is 2 ～20mm, the accuracy is 0.01mm, the center distance is 8～20mm, the precision is 0.01mm, the center of the circle on the target surface is selected as the feature point, and the number of feature points is 16～100.

2. The method for carrying out three-dimensional measurement using the double-spherical mirror image binocular vision measurement system of a single-camera as claimed in claim 1 is characterized in that the measurement process is divided into a calibration stage and a measurement stage, and can be continuously measured after one calibration, specifically Proceed as follows:

2.1. Calibration stage:

2.1.1. Fasten the industrial camera (2); adjust the focal length of the optical lens (3) to ensure that the image formed by the measured object (10) within the range of 200-600mm from the intersection point of the edge of the double spherical mirror in the axial direction of the double spherical mirror relatively clear; after adjustment, fasten the optical lens (3);

2.1.2. The industrial camera (2) is mirrored by a double spherical mirror into two virtual cameras, which are respectively called the left virtual camera (13) and the right virtual camera (14); the left virtual camera (13) and the right virtual camera (14) constitute Mirror binoculars; establish the imaging model of the virtual camera, the main parameters include the equivalent focal length and principal point of the virtual camera, and the distortion coefficient of the spherical mirror; calibrate the imaging model parameters of the virtual camera, the specific steps are as follows:

The first step is to freely move the calibration target (11) to at least three positions within the field of view of the mirrored binocular sensor (1), and take an image for each position moved, which is called a dual spherical mirror calibration image. The feature points of should be included in the captured image;

In the second step, the left and right parts of the double-spherical mirror calibration image are disassembled from the middle column of the image into two calibration images of the same size, which are called the left calibration image and the right calibration image respectively; the left calibration image and the left virtual camera ( 13) Corresponding, the right calibration image corresponds to the right virtual camera (14);

In the third step, the feature points in the left calibration image are called left calibration feature points, and the image coordinates of the left calibration feature points are extracted, corresponding to the world coordinates of the feature points; the distortion coefficient and Equivalent focal length and principal point of left virtual camera (13);

The fourth step, the feature point in the right calibration image is called the right calibration feature point, extract the image coordinates of the right calibration feature point, corresponding to the world coordinates of the feature point; utilize the right calibration feature point to calibrate the distortion coefficient and Equivalent focal length and principal point of right virtual camera (14);

The fifth step is to use the distortion coefficient of the large spherical mirror (7) to perform spherical distortion correction on the left calibration image to obtain the left calibration image without spherical distortion; to use the distortion coefficient of the small spherical mirror (6) to perform spherical distortion correction to the right calibration image to obtain Spherical distortion right calibration image;

2.1.3. To calibrate the structural parameters and essential matrix of the mirrored binoculars, the specific steps are as follows;

The first step is to establish the epipolar geometric constraint relationship of the mirrored binoculars, which is described by the essential matrix; the 3D measurement model of the mirrored binoculars is established, and the parameters of the model are the structural parameters of the mirrored binoculars, including the 3D coordinate system O _r of the right virtual camera The rotation matrix and translation vector of -x _r y _r z _r to the three-dimensional coordinate system O _l of the left virtual camera -x _l y _l z _l ;

The second step is to extract the image coordinates of the feature points of the undistorted left calibration image and the undistorted right calibration image respectively, and correspond to the world coordinates of the feature points;

The third step is to use the undistorted image coordinates of all feature points and the corresponding world coordinates to calibrate the structural parameters of the mirrored binoculars, and calculate the essential matrix of the mirrored binoculars;

2.2. Measurement stage:

2.2.1. Place the object to be measured (10) within the field of view (9) of the mirrored binocular sensor (1), and adjust the object to be measured within the range of 200-600mm from the intersection point of the edge of the double spherical mirror in the axial direction of the double spherical mirror (10), to ensure that the measured object is simultaneously imaged on the left half and right half of the image captured by the camera, and an image is taken, which is called a measurement image;

2.2.2. Separate the left and right parts of the measurement image from the middle column of the image into two calibration images of the same size, which are called the left measurement image and the right measurement image respectively; the left measurement image and the left virtual camera (13) Correspondingly, the right measurement image corresponds to the right virtual camera (14);

2.2.3. Using the imaging model parameters of the calibrated virtual camera in step 2.1.2, perform distortion correction on the left measurement image and right measurement image obtained in step 2.2.2 to obtain the undistorted left measurement image and the undistorted right measurement image ;

2.2.4, setting the measuring point on the measured object (10), and calculating the three-dimensional coordinates of the measuring point under the O _l -x _{ly l} z _l coordinate system, the specific steps are as follows:

In the first step, the image points corresponding to the measurement points in the undistorted left measurement image and the undistorted right measurement image are called the corresponding point pairs with the same name. Through image processing and stereo matching according to the geometric constraints of the mirror binocular epipolar line, determine Undistorted left measurement image coordinates and undistorted right measurement image coordinates of corresponding point pairs with the same name;

In the second step, in the O _l -x _{ly l} z _l coordinate system, according to the mirrored binocular three-dimensional measurement model established in step 2.1.3, the measurement point is calculated from the undistorted image coordinates of the corresponding point pairs with the same name of the measurement point The three-dimensional coordinates of the point in the O _l -x _{ly l z l} coordinate system, and save the three-dimensional coordinates to the data file;

2.2.5. Repeat steps 2.2.1 to 2.2.4 to perform three-dimensional measurement of new measurement points of the measured object.

The advantages of the present invention are:

1. A binocular vision measurement system and measurement method based on a single-camera double-spherical mirror mirror image is proposed. The binocular image pair is obtained by using a single-camera, which realizes the synchronous acquisition of the left and right views of the binocular measurement, reduces the cost of the binocular system, and reduces the size of the binocular system. Measure the volume of the system.

2. Using a double-spherical mirror to replace the existing double-plane mirror for mirrored binocular imaging, expanding the field of view, establishing a virtual camera imaging model to solve the inherent curl distortion problem of spherical reflection imaging, and establishing a mirrored binocular 3D measurement model to realize the measured object 3D measurement.

3. The proposed single camera dual spherical mirror binocular vision sensor is easy to miniaturize and lightweight, expands the application range of the binocular system, and can be used for the measurement of restricted spaces and complex scenes.

(4) Description of drawings

Fig. 1 is a flow chart of the binocular vision measurement method for the double-spherical mirror mirror image of a single camera;

Figure 2 is a schematic diagram of the composition of the binocular vision measurement system for the mirror image of the double spherical mirror of the single camera;

Fig. 3 is a two-dimensional schematic diagram of the field of view of the mirrored binocular sensor, the axial direction of the double spherical mirror and the optical path;

Figure 4 is a schematic diagram of the target;

Fig. 5 is a schematic diagram of a binocular three-dimensional measurement model mirrored by a virtual camera;

Fig. 6 is the calibration image of the double spherical mirror collected;

(5) Specific implementation methods

The present invention will be described in further detail below. The present invention is based on computer vision and image processing technology, according to the paper "Three-Dimensional Measurement Approach in SmallFOV and Confined Space Using an Electronic Endoscope [J]. IEEE Sensors Journal, 2014, 14 (9): 3274-3282." The two-plane mirror single-camera mirroring binocular stereo vision system was proposed. Based on this improvement, a mirror binocular vision sensor composed of a single-camera and a double-spherical mirror was designed, and the imaging model of the virtual camera was established. The spherical warping and distorted image captured by the mirror binocular sensor is unfolded, that is, the distortion is corrected, and the mirror binocular 3D measurement model is established. After the calibration of each model parameter is completed, the 3D measurement is realized.

The structure and system optical path design of the mirror binocular sensor are shown in Figure 3. The measured scene is reflected by two spherical mirrors and then enters the single camera for imaging. The optical axis direction of the optical lens in the single camera is not in the same direction as the field of view. According to each component and layout, it includes two parts: spherical mirror parameter design and mechanical structure parameter design. In the parameter design of the spherical mirror, analyze the impact of the placement angle of the small spherical mirror (6) and the large spherical mirror (7), the diameter of the lens and the radius of curvature on the field of view of the sensor, deduce the mathematical relationship, and determine the small spherical mirror (6) and the large spherical mirror ( 7) The optical structure parameters. In the mechanical structure parameter design, the mechanical support (4) is designed according to the actual size of the selected industrial camera (2) and the optical lens (3), combined with the parameters of the spherical mirror. Finally, the small spherical mirror (6), the large spherical mirror (7), the mechanical support (4), the industrial camera (2), and the optical lens (3) form a single camera double spherical mirror mirror image binocular vision measurement system.

The present invention aims at the designed single-camera double-spherical mirror mirror binocular vision measurement system, and establishes the imaging model of the virtual camera and the mirror binocular three-dimensional measurement model, as shown in FIG. 5 . The main parameters of the imaging model of the virtual camera include the equivalent focal length and principal point of the virtual camera, and the distortion coefficient of the spherical mirror. The calibration target image captured by the mirror binocular sensor is called the double-spherical mirror calibration image, and the left and right parts of the double-spherical mirror calibration image are separated from the middle column of the image into two calibration images of the same size, which are called the left calibration image. and the right calibration image, the left calibration image corresponds to the left virtual camera (13), and the right calibration image corresponds to the right virtual camera (14). The perspective projection model of the left and right virtual cameras is:

Among them, (x _vl , y _vl ) is the image coordinate of the feature point in the left calibration image, A _vl is the internal reference matrix of the left virtual camera (13), f _vxl , f _vyl is the left virtual camera (13) in x, y The effective focal length in the direction, (u _0vl , v _0vl ) is the principal point coordinates of the left virtual camera (13), (R _vl |T _vl ) is the rotation of the calibration target in the O _l -x _{ly l} z _l coordinate system Matrix and translation vector; (x _vr , y _vr ) is the image coordinate of the feature point in the left calibration image, A _vr is the internal reference matrix of the right virtual camera (14), f _vxr , f _vyr is the right virtual camera (14) in The effective focal length in the x and y directions, (u _0vr , v _0vr ) is the principal point coordinates of the right virtual camera (14); (R _vr |T _vr ) is the calibration target in the O _r -x _r y _r z _r coordinate system The rotation matrix and translation vector below.

Consider the lens distortion of the left virtual camera (13) and the right virtual camera (14), the lens distortion model is:

Among them, (x _l , y _l ) is the ideal image coordinates of feature points imaged on the left virtual camera, (x _dl , y _dl ) is the actual image coordinates of feature points imaged on the left virtual camera, r is the actual image coordinates and the asphere The distance between the virtual camera principal points of reflection distortion, (k _l1 ,k _l2 ) is the primary and secondary radial distortion coefficient of the left virtual camera lens, (p _l1 ,p _l2 ) is the primary and secondary radial distortion coefficient of the left virtual camera lens Secondary tangential distortion coefficient; (x _r , y _r ) is the ideal image coordinates of feature points imaged on the left virtual camera, (x _dr , y _dr ) is the actual image coordinates of feature points imaged on the left virtual camera, and r is the actual image The distance between the coordinates and the principal point of the virtual camera without spherical reflection distortion, (k _r1 , k _r2 ) is the primary and secondary radial distortion coefficients of the left virtual camera lens, (p _r1 , p _r2 ) is the left virtual camera lens The primary and secondary tangential distortion coefficients of .

The distortion of a spherical mirror can be regarded as a combination of radial distortion and tangential distortion, and its model is:

Among them, (x _dl , y _dl ) are the coordinates of the left calibration image without spherical distortion, (x _0l , y _0l ) are the coordinates of the actual left calibration image, r _0l is the distance between the actual left calibration image coordinates and the principal point of the left virtual camera, (k _sl1 , k _sl2 , k _sl3 ) are the radial distortion coefficients of the large spherical mirror (7), (p _sl1 , p _sl2 ) are the tangential distortion coefficients of the large spherical mirror (7); (x _dr , y _dr ) are none Spherical reflection distortion right calibration image coordinates, (x _0r , y _0r ) is the actual left calibration image coordinates, r _0r is the distance between the actual right calibration image coordinates and the principal point of the right virtual camera, (k _sr1 , k _sr2 , k _sr3 ) is the radial distortion coefficient of the small spherical mirror (6), and (p _sr1 , p _sr2 ) is the tangential distortion coefficient of the small spherical mirror (6).

Establish the mirror binocular epipolar geometric constraint relationship, which is represented by an essential matrix. For the description method, refer to Cui Yi’s paper "Precise calibration of binocular vision system used for visionmeasurement" [Optics Express, Vol.20, No.8, 2014]; Mirror binocular three-dimensional measurement model, the main parameters include the rotation matrix R and translation vector T from the O _r -x _{ry r} z _r coordinate system to the O _l -x _{ly l} z _l coordinate system.

A single virtual camera cannot restore the three-dimensional coordinates of the center of the feature point circle through the image coordinates. The mirror binocular stereo vision system can calculate the three-dimensional coordinates of the feature points through the coordinates of the feature points in the two virtual camera images according to the principle of triangulation. Select the O _l -x _{ly l z l} coordinate system as the measurement coordinate system, and the 3D measurement model of the mirrored binoculars is:

Among them, λ _vl , λ _vr are scaling factors; A _vl , A _vr are the internal parameter matrices of the left and right virtual cameras, x _l (u _l ,v _l ), x _vr (u _vr ,v _vr ) are the left virtual camera (13) and the corresponding point pixel coordinates of the right virtual camera (14) image. R, T are virtual binocular structure parameters, which represent the conversion relationship between the O _r -x _{ry r} z _r coordinate system and the O _l -x _{ly l} z _l coordinate system.

The mutual pose relationship between the O _l -x _{ly l z l} coordinate system and the O _r -x _ry y _r z _r coordinate system can be expressed by matrix rotation:

According to the measurement system and the mathematical model of the binocular vision sensor of the double-spherical mirror mirror image of the single-camera camera, the measurement process of the binocular vision measurement system of the double-spherical mirror mirror image of the single-camera camera of the present invention is as shown in Figure 1, and the specific steps are as follows:

1. Fasten the industrial camera (2); adjust the focal length of the optical lens (3) to ensure that the image formed by the measured object (10) within the range of 200-600mm from the intersection point of the edge of the double spherical mirror in the axial direction of the double spherical mirror is relatively clear ; After adjustment, fasten the optical lens (3);

2. Within the field of view of the mirrored binocular sensor (1), freely move the calibration target (11) to at least three positions, each time a position is moved, an image is taken, which is called a double-spherical mirror calibration image, and all the features on the target The point should be included in the captured image;

3. Separate the left and right parts of the double-spherical mirror calibration image from the middle column of the image into two calibration images of the same size; the left calibration image corresponds to the left virtual camera (13), and the right calibration image corresponds to the right virtual camera (14 )correspond;

4. According to the formula [1,2], use the left calibration feature point to calibrate the distortion coefficient of the large spherical mirror (7) and the equivalent focal length and principal point of the left virtual camera (13), and use the right calibration feature point to calibrate the small spherical mirror (6) distortion coefficient and the equivalent focal length and principal point of the right virtual camera (14), the calibration method can be found in Zhang Zhengyou's paper "A flexible new technique for camera calibration" [IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22, No.11, 2000];

5. According to the formula [3], use the distortion coefficient of the large spherical mirror (7) to perform spherical distortion correction on the left calibration image to obtain the left calibration image without spherical distortion; use the distortion coefficient of the small spherical mirror (6) to perform spherical distortion on the right calibration image Correction to obtain the right calibration image without spherical distortion;

6. Extract the image coordinates of the feature points of the left calibration image without spherical distortion and the right calibration image without spherical distortion respectively, and correspond to the world coordinates of the feature points;

7. According to the proposed three-dimensional measurement model for mirror image binoculars (formula [4,5]), use the image coordinates of all feature points extracted in step (10) and the corresponding world coordinates to calibrate the mirror image binocular structure parameters, and calculate The essential matrix, obtain the rotation matrix and translation vector from the O _r -x _ry y _r z _r coordinate system to the O _l -x _ly y _l z _l coordinate system;

8. According to an embodiment of the present invention, the measured object (10) is measured;

9. Place the object to be measured (10) within the field of view (9) of the mirrored binocular sensor (1), within the range of 200-600mm from the intersection point of the edge of the double spherical mirror in the axial direction of the double spherical mirror, the sensor captured The real-time image is displayed in the computer (8), and the measured object (10) is adjusted to ensure that the left and right parts of the image presented are clear and complete, and an image is taken as a measurement image; the left and right parts of the measurement image are The part is disassembled from the middle column of the image into two measurement images, as the left measurement image and the right measurement image; the point on the measurement image is extracted by using the SIFT algorithm, and it is set as the measurement point on the measured object (10). For the extraction method, see David G.Lowe's paper David G.Lowe. "Distinctive image features from scaleinvariant keypoints" [International Journal of Computer Vision,2004]";

10. Using the calibrated imaging model parameters of the virtual camera, perform spherical distortion correction on the left and right measurement images obtained in step 9 in turn, to obtain a left measurement image without spherical distortion and a right measurement image without spherical distortion;

11. From the left and right measurement images without spherical distortion obtained in step 10, extract the image coordinates of the measurement points of the measured object, perform stereo matching according to the epipolar geometric constraints of the virtual binocular stereo vision, and determine the distortion-free left Measure the coordinates of the image and the undistorted right measurement image coordinates, and calculate the straight line determined by the origin of the O _r -x _r y _r z _r coordinate system and the projection point of the measurement point through the mirror binocular three-dimensional measurement model established by the formula [4,5] The least squares intersection point of the straight line with the origin of the O _l -x _l y _l z _l coordinate system and the measurement point. For the calculation method, refer to Cui Yi’s paper "Precise calibration of binocular vision system used for vision measurement" [Optics Express, Vol. 22, No.8, 2014], obtain the three-dimensional coordinates of the measuring point in the O _l -x _{ly l} z _l coordinate system, and complete the measurement of the measuring point of the measured object (10).

(6) Embodiment

Daheng Mercury series MER-301-125U3C camera and computar V1228-MPY 12mm fixed-focus lens are used. The resolution of the industrial camera is 2048×1236 pixels. The reflective surface material of the spherical mirror is selected as aluminum plating. The diameter of the mirror surface of the spherical mirror 6 is 35.18mm, the radius of curvature is 200mm, the thickness of the edge is 1.60mm, and the placement angle is 37°. The diameter of the mirror surface of the spherical mirror 7 is 47.37mm, and the radius of curvature is 200mm. The edge thickness is 1.60mm, the placement angle is 53°, the distance d between the intersection point of the single camera and the bottom edge of the double spherical mirror is 100mm, and the angle of the x-axis direction in Figure 3 is defined as 0°, then the field of view of the mirrored binocular sensor is [-10.46°,13.91°], the field of view is 24.37°. The size of the sensor base plate is selected as 70mm×220mm×10mm, the size of the spherical mirror base plate is 50mm×50mm×5mm and 38mm×38mm×5mm, and the size of the camera fixing hole is φ6.6mm×6mm.

Carry out visual measurement verification on the designed and processed mirror binocular sensor, use a two-dimensional plane target for calibration, place the target within 600mm from the intersection point of the bottom edge of the double spherical mirror in the axial direction of the double spherical mirror, and freely move the calibration target to 20 positions, Every time a position is moved, an image is taken. The number of circular feature points on the target is 49, the distance between the centers of the circles is 12.50mm, the diameter of the dots is 6.25mm, and the accuracy is 0.01mm.

Figure 6 shows the binocular mirror image of the target collected by the mirror binocular sensor. According to the established virtual camera imaging model, the left and right parts of the collected dual spherical calibration image are disassembled from the middle column of the image into two parts of the same size. calibration image. The equivalent focal length and principal point of the virtual camera, as well as the distortion coefficient of the virtual camera lens are obtained through the calibration of the imaging model of the virtual camera. The internal parameters of the left virtual camera after calibration are as follows:

f _vxl = 1412.50pixels, f _vyl = 1528.92pixels

u _0vl =897.39pixels, v _0vl =740.31pixels

k _l1 =-0.25, k _l2 =0.22, p _l1 =-0.00, p _l2 =-0.10

The internal parameters of the right virtual camera are as follows:

f _vxr = 1625.01pixels, f _vyr = 2115.20pixels

u _0vr = 450.27pixels, v _0vr = 841.79pixels

k _r1 =0.32, k _r2 =-16.09, p _r1 =-0.06, p _r2 =214.90

The distortion coefficient of small spherical mirror (6) is:

k _sl1 =-2.06, k _sl2 =27.93, k _sl3 =-139.34, p _sl1 =-0.02, p _sl2 =-0.05

The distortion coefficient of large spherical mirror (7) is:

k _sl1 =0.14, k _sl2 =-0.41, k _sl3 =0.41, p _sl1 =0.01, p _sl2 =-0.13

According to the obtained imaging model parameters of the virtual camera, distortion correction is performed on the left calibration image and the right calibration image, and the left and right calibration images without spherical distortion are obtained. Use the left and right calibration images without spherical distortion to perform mirrored binocular 3D measurement model calibration, including the rotation matrix and translation vector from the O _r -x _{ry r} z _r coordinate system to the O _l -x _ly l _{z l} coordinate system structure parameters, the result is as follows:

The calibration and measurement results show that the image error without spherical reflection distortion correction is relatively large, and the calibration accuracy of the image after spherical distortion correction is improved. The calibration error of the left virtual camera is about 0.10 pixels, and the calibration error of the right virtual camera is about 0.12 pixels. , the error of the mirror binocular target is 2.48 pixels, the experiment verifies the feasibility of the mirror binocular sensor and the proposed virtual camera imaging model and the mirror binocular three-dimensional measurement model, and verifies that the proposed spherical distortion correction can improve the reflection of the spherical mirror The effect of curl distortion inherent in imaging. The single-camera dual-spherical mirror binocular vision sensor solves the contradiction between the volume and field of view of the mirrored binocular system, further miniaturizes the sensor, obtains a larger field of view, and establishes a mathematical model to restore curled and distorted images. Realize three-dimensional measurement.

Claims (2)

1. A binocular vision measuring system of a single camera with a double spherical mirror image is characterized in that,

1.1, the binocular vision measuring system of the mirror image of the double spherical mirrors of the single camera, by mirror image binocular sensor (1), data transmission line (5), computer (8), calibration target (11) and power (12) make up; the mirror image binocular sensor (1) consists of an industrial camera (2), an optical lens (3), a mechanical bracket (4), a small spherical mirror (6) and a large spherical mirror (7); the industrial camera (2) and the optical lens (3) form a single camera, the small spherical mirror (6) and the large spherical mirror (7) form a double spherical mirror, and the axial direction (18) of the double spherical mirror is defined as the intersection point M of the edges of the double spherical mirror to the central axis O of the small spherical mirror (6) ₁ E ₁ And a central axis O of a large spherical mirror (7) ₂ E ₂ Is the intersection point J of (2); the data transmission line (5) is connected with the computer (8); the measured object (10) is arranged in the view field range (9) of the mirror image binocular sensor (1) and is 200-600 mm away from the intersection point of the edges of the double spherical mirrors in the axial direction of the double spherical mirrors; the mirror image binocular sensor (1) transmits the shot image into a computer (8) for storage and processing;

the geometrical parameters of the mirror image binocular sensor (1) comprise: the distance between the industrial camera (2) and the intersection point M of the edges of the small spherical mirror (6) and the large spherical mirror (7) is 80-150 mm, the mirror surface diameter of the small spherical mirror (6) is 20-50 mm, the curvature radius is 40-500 mm, the edge thickness is 1-10 mm, and the placement angle is defined as the included angle between the bottom plate (16) of the small spherical mirror (6) and the optical axis OM of the lens, and the value is 15-45 degrees; the diameter of the mirror surface of the large spherical mirror 7 is 30-60 mm, the radius of curvature is 40-500 mm, the edge thickness is 1-10 mm, the placement angle is defined as the included angle between the bottom plate (17) of the large spherical mirror (7) and the optical axis OM of the lens, and the value is 45-80 degrees;

1.3, the calibration target (11) is a two-dimensional plane, the target is provided with preset characteristic points, black solid circles which are arranged in a matrix arrangement are arranged on the target plane, the number of the circles is 16-100, the diameter is 2-20 mm, the precision is 0.01mm, the center distance of the circles is 8-20 mm, the precision is 0.01mm, the center of the circles on the target surface is selected as the characteristic points, and the number of the characteristic points is 16-100.

2. The method for three-dimensional measurement by using the single-camera double-spherical mirror image binocular vision measurement system according to claim 1, wherein the measurement process is divided into a calibration stage and a measurement stage, and the measurement can be continuously performed after one calibration, and the specific steps are as follows:

2.1, calibration stage:

2.1.1, fastening the industrial camera (2); the focal length of the optical lens (3) is adjusted, so that the image formed by the measured object (10) which is in the range of 200-600 mm away from the intersection point of the edges of the double spherical mirrors in the axial direction of the double spherical mirrors is clear; after the adjustment, the optical lens (3) is fastened;

2.1.2, the industrial camera (2) is mirrored into two virtual cameras through a double-spherical mirror, which are respectively called a left virtual camera (13) and a right virtual camera (14); the left virtual camera (13) and the right virtual camera (14) form a mirror image binocular; establishing an imaging model of the virtual camera, wherein main parameters comprise an equivalent focal length and a principal point of the virtual camera and a distortion coefficient of a spherical mirror; calibrating imaging model parameters of a virtual camera, wherein the method comprises the following specific steps of:

the method comprises the steps that at least three positions of a calibration target (11) are freely moved in the field of view of a mirror image binocular sensor (1), an image is shot every time one position is moved, the image is called a double-spherical mirror calibration image, and all characteristic points on the target are contained in the shot image;

secondly, disassembling a left part and a right part in the double-spherical-mirror calibration image from an image middle row into two calibration images with the same size, which are respectively called a left calibration image and a right calibration image; the left calibration image corresponds to the left virtual camera (13), and the right calibration image corresponds to the right virtual camera (14);

thirdly, characteristic points in the left calibration image are called left calibration characteristic points, and image coordinates of the left calibration characteristic points are extracted and correspond to world coordinates of the characteristic points; calibrating a distortion coefficient of the large spherical mirror (7) and an equivalent focal length and a principal point of the left virtual camera (13) by using the left calibration characteristic points;

fourthly, characteristic points in the right calibration image are called right calibration characteristic points, and image coordinates of the right calibration characteristic points are extracted and correspond to world coordinates of the characteristic points; calibrating a distortion coefficient of the small spherical mirror (6) and an equivalent focal length and a principal point of the right virtual camera (14) by using the right calibration characteristic points;

fifthly, carrying out distortion correction on the left calibration image by using a distortion coefficient of the large spherical mirror (7) to obtain a distortion-free left calibration image; carrying out distortion correction on the right calibration image by using the distortion coefficient of the small spherical mirror (6) to obtain a distortion-free right calibration image;

2.1.3, calibrating the parameters and the essential matrix of the mirror image dual-purpose structure, and specifically comprises the following steps:

firstly, establishing a mirror image dual-purpose polar line geometric constraint relation, and describing the geometric constraint relation by using an essential matrix; establishing a mirror image dual-purpose three-dimensional measurement model, wherein parameters of the model are mirror image dual-purpose structural parameters, and the mirror image dual-purpose three-dimensional measurement model comprises a three-dimensional coordinate system O of a right virtual camera _r -x _r y _r z _r Three-dimensional coordinate system O to left virtual camera _l -x _l y _l z _l A rotation matrix and a translation vector of (a);

secondly, extracting image coordinates of feature points of the undistorted left calibration image and the undistorted right calibration image respectively, and corresponding to world coordinates of the feature points;

thirdly, calibrating the mirror image dual-purpose structural parameters by using undistorted image coordinates and corresponding world coordinates of all the characteristic points, and calculating a mirror image dual-purpose essential matrix;

2.2, measuring stage:

2.2.1, placing the measured object (10) in a view field range (9) of the mirror image binocular sensor (1), adjusting the measured object (10) within a range of 200-600 mm away from the edge intersection point of the double-spherical mirror in the axial direction of the double-spherical mirror, and ensuring that the measured object is imaged on the left half part and the right half part of an image shot by a camera at the same time, and shooting an image, namely a measurement image;

2.2.2, the left and right parts in the measured image are separated from the middle row of the image into two measured images with the same size, which are respectively called a left measured image and a right measured image; the left measurement image corresponds to a left virtual camera (13), and the right measurement image corresponds to a right virtual camera (14);

2.2.3, performing distortion correction on the left measurement image and the right measurement image obtained in the step 2.2.2 by using the imaging model parameters of the virtual camera calibrated in the step 2.1.2 to obtain an undistorted left measurement image and an undistorted right measurement image;

2.2.4 setting measurement points on the measured object (10), and calculating the measurement points at O _l -x _l y _l z _l The three-dimensional coordinates in the coordinate system comprise the following specific steps:

firstly, respectively corresponding image points of the measuring points in an undistorted left measuring image and an undistorted right measuring image are called as homonymous corresponding point pairs, and the undistorted left measuring image coordinates and the undistorted right measuring image coordinates of the homonymous corresponding point pairs are determined through image processing and three-dimensional matching according to mirror image dual-purpose polar line geometric constraint;

second step, at O _l -x _l y _l z _l In the coordinate system, according to the mirror image dual-purpose three-dimensional measurement model established in the step 2.1.3, calculating the position of the measurement point in O by using the undistorted image coordinates of the corresponding point pairs with the same name of the measurement point _l -x _l y _l z _l Storing three-dimensional coordinates in a data file according to the three-dimensional coordinates in the coordinate system;

2.2.5, repeating the steps 2.2.1-2.2.4, and carrying out three-dimensional measurement of a new measuring point of the measured object.

Images