WO2019041349A1

WO2019041349A1 - Three-dimensional visual information measuring method based on rotating lens

Info

Publication number: WO2019041349A1
Application number: PCT/CN2017/100407
Authority: WO
Inventors: 贾振元; 刘巍; 马鑫; 梁冰; 李肖; 刘维肖
Original assignee: 大连理工大学
Priority date: 2017-09-04
Filing date: 2017-09-04
Publication date: 2019-03-07

Abstract

A three-dimensional visual information measuring method based on a rotating lens (2). A beveled straight cylindrical lens (2) is placed in front of a lens of a monocular camera (1) and is coaxial with an optical axis thereof and rotates around the optical axis. First, Zhang's calibration is performed on the monocular camera (1) to obtain an internal parameter, and then the rotating lens (2) is held still in positions corresponding to two limit time points to construct, according to differences in normal direction and thickness of a sloped lens surface (3) through which an optical path between an object point (P) and an image point passes, a model of the relationship between three-dimensional coordinates of the object point (P) and three-dimensional coordinates of the image point on the basis of a parameter of the rotating lens (2) and the internal parameter of the monocular camera (1). Three-dimensional spatial coordinates of the object point (P) are calculated by combining the relationship models obtained at the two limit time points. The method reduces a traditional binocular camera system to a combination device having the monocular camera (1) and the rotating lens (2), thereby greatly reducing cost, increasing measuring speed and improving measuring accuracy, while also resolving issues such as inaccuracy of three-dimensional information obtained via monocular vision measurements and the high costs of binocular vision measurement.

Description

Three-dimensional information visual measurement method based on rotating lens

Technical field

The invention belongs to the technical field of computer vision measurement, and relates to a method for rapidly measuring three-dimensional information of objects based on vision.

Background technique

Three-dimensional non-contact vision measurement technology has been widely used in various fields such as automobile, shipbuilding, aviation, aerospace, and military. Especially in the processing and assembly process of typical complex shape components in aviation and aerospace, it is necessary to accurately measure the three-dimensional information to ensure the quality of processing and assembly. In addition, in the field of robotics and artificial intelligence, it is also necessary to quickly measure and sense the three-dimensional information of surrounding scenes. Therefore, the vision-based three-dimensional information measurement method with the advantages of fast measurement speed, high efficiency and non-contact type has important significance. In order to improve the accuracy of three-dimensional information acquisition, binocular or even multi-camera cameras are often used, which not only greatly increases the measurement cost, but also has the problems of poor repeatability and difficulty in matching images acquired by multi-camera due to the influence of on-site experimental conditions. These all put forward higher requirements for the measurement equipment and technical means of three-dimensional information.

The invention patent CN105571518A applied by Liu Wei and Ma Xin et al., "Three-dimensional information visual measurement method based on refractive image deviation" proposes a monocular vision measurement method assisted only by parallel optical glass, which improves measurement efficiency and reduces cost. Achieve a fast measurement of the monocular camera for the full field of view. The method adopts a monocular camera to first take an image A1, then puts a glass plate with a known refractive index into the camera at an arbitrary angle, and then takes an image A2, and uses the deviation amount of the two images to complete the space of the measured object. Measurement of three-dimensional information. However, this method requires artificial placement of the glass plate, which not only introduces human error, but also reduces measurement efficiency.

Song Zhendong et al. Design and implement a monocular multi-view stereo image based on wide-angle camera and plane mirror in "Multi-view multi-view stereo image extraction and application" published in the 32th issue of the Journal of Optics. The camera system gives the design index and optimization method of the hardware device. At the same time, based on the calibration method of the hardware system, its application in 3D ranging is realized. However, the hardware device occupies a large space and is difficult to use in a narrow space, and installation and positioning are relatively difficult.

Lee and Kweon proposed a novel and practical stereo camera system in "A novel stereo camera system by a biprism" published in IEEE Transactions on Robotics and Automation, Vol. 16, No. 5, placing a pair of prisms in front of a monocular camera. The reflected light path generated by the two inclined surfaces of the double prism is different, and the left and right sides of the CCD are simultaneously imaged, and the three-dimensional structure is reconstructed by using the corresponding gap. The system effectively reduces measurement costs and simplifies the calibration process, but the double prism reflection angle is difficult to control.

Gao and Ahuja use the flat panel that can rotate around the center of the optical axis in the "Single camera stereo using planar parallel plate" published in Volume 4 of "International Conference on Pattern Recognition", placed in front of the monocular camera at a certain oblique angle, and Calibration to determine the parameters within the board and its external attitude. Rotate the plane plate to capture the image of the object in different poses, apply the position difference in different images, and calculate the object depth information. The device and algorithm are only used to obtain depth information, and cannot be used as three-dimensional reconstruction of objects, and the precision of parallel plate installation, positioning and rotation process is difficult to guarantee.

Shimizu and Okutomi, in "Reflection stereo-novel monocular stereo using a transparent plate" published in The 2006 Canadian Conference on Computer & Robot Vision, proposed a depth of object based on triangulation based on the difference of light paths when light passes through reflective and refractive media. Measurement methods. The operation steps of the method are simplified to a certain extent, and the measurement efficiency is high. However, due to mutual interference between the reflected and refracted images, the method needs to be improved in measurement accuracy, and calibration is relatively difficult.

Atsushi Yamashita et al. presented in "Monocular Underwater Stereo-3D Measurement Using Difference of Appearance Depending on Optical Paths" by The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. A new method for underwater three-dimensional information measurement. The method is based on the monocular vision principle, placing two angled refraction faces in front of the lens to block water and air. The light is refracted at the two refractive surfaces and the light path is different, and is simultaneously imaged in the camera. The three-dimensional information of underwater objects is measured by optimizing the angles of the two refractive surfaces. However, the optimization process is more complicated and the accuracy is not easy to guarantee.

Teoh and Zhang proposed a measurement method using a monocular camera instead of a binocular camera in "An inexpensive stereoscopic vision system for robots" published by IEEE International Conference on Robotics and Automation. The method places two plane mirrors at an angle of 45° with respect to the optical axis in front of both sides of the camera. The angle between the two plane mirrors is the same as the optical axis, and a third plane mirror is placed in front of the lens. When the third plane mirror is parallel with the other two sheets, the object image is collected to achieve the same effect as the binocular camera, thereby effectively reducing the cost. However, the device is relatively complicated and has many lenses, and its positioning accuracy directly affects the measurement accuracy.

Kim and Lin proposed a new depth measurement method in "Distance Measurement Using a Single Camera with a Rotating Mirror" published in International Journal of Control Automation & Systems, Vol. 3, No. 4. The method uses a monocular camera and a rotating planar mirror. During the rotation of the plane mirror, the monocular camera collects a series of reflected images, extracts the distance information from the image, and calculates the depth information of the object. This method simplifies the calibration process, and the measurement efficiency is high, but the measurement accuracy needs to be improved.

The above invention mainly studies the non-contact rapid measurement method of three-dimensional information, and has achieved many important results, but the efficiency and robustness of the measurement method cannot be guaranteed. The invention provides a three-dimensional information visual measurement method based on a rotating lens, which not only improves the accuracy of the single-camera three-dimensional information measurement method, but also improves the measurement efficiency.

Summary of the invention

The invention overcomes the defects of the prior art and provides a three-dimensional information visual measurement based on a rotating lens method.

The technical solution of the invention:

A three-dimensional information visual measuring method based on a rotating lens, characterized in that the three-dimensional information visual measuring method is based on a three-dimensional information visual measuring device of a rotating lens, comprising a monocular camera and a rotating lens, wherein the rotating lens adopts light transmission 80% or more of the glass material is chamfered straight cylinder, placed in front of the monocular camera lens, its axis coincides with the optical axis, and the parallel surface is opposite to the lens;

During the measurement, the rotating lens is rotated. At different times, the normal direction and the thickness of the optical path between the object point and the image point through the rotating lens bevel are different, and the lens parameters at two moments and the parameters in the monocular camera are constructed. The relationship model between the point and the three-dimensional coordinates of the image point, the measurement of the three-dimensional information of the object by binocular vision is converted into the measurement of the three-dimensional information of the object at different times of the monocular camera and the rotating lens;

Proceed as follows:

(1) Calibration camera with precision machined target board

The camera calibration method based on 2D planar target calibrates the internal parameters of the camera, and obtains the conversion relationship between the coordinates (u, v) ^{T in the} pixel coordinate system and the coordinates (x, y) ^T in the image coordinate system as follows:

Where: (dx, dy) is the physical size corresponding to each pixel on the x-axis and the y-axis, and (u ₀ , v ₀ ) is the coordinate of the origin of the image coordinate system in the pixel coordinate system;

(2) Establishing the relationship between the object point and the image point at two times T1 and T2

1) Establish the relationship between the object point and the image point at T1

The inclined surface of the rotating lens is the surface 3, and the plane of the rotating lens is the surface 4; the time T1 is an arbitrary moment of the rotation of the lens, and at the time T1, the rotating lens is in a stationary state, [n ₁ , n ₂ ] are respectively the surface 4 and the surface 3 The normal vector; [v ₁ , v ₂ , v ₃ ] is the optical path direction vector of the optical center to the surface 4, the surface 4 to the surface 3, and the surface 3 to the object point P; [q ₁ , q ₂ ] are respectively the optical path 4 and the reflection point on the surface 3; [d ₁ , d ₂ , d ₃ ] are the projections of the optical paths from the optical center to the surface 4, the surface 4 to the surface 3, and the surface 3 to the object point P in the positive direction of the Z axis. Distance; (X, Y, Z) ^T is the spatial three-dimensional coordinate of the object point P;

From Snell's law: μ _i sin θ _i = μ _i+1 sin θ _i+1 (2)

Where: μ _i and μ _i+1 are the refractive indices of the two media respectively; let v _i and v _i+1 be the incident and outgoing optical path vectors respectively, then θ ₁ is the angle between v ₁ and n ₁ , θ ₂ is the angle between v ₂ and n ₁ , θ ₃ is the angle between v ₂ and n ₂ , and θ ₄ is the angle between v ₃ and n ₂ ; the formula (2) is written in the form of a vector,

v _i+1 =a _i v _i +b _i n _i (3)

Where: a _i =μ _i /μ _i+1 ,

The initial camera ray v ₁ is obtained by calibration, and the direction vector of each segment of the optical path is derived by the equation (3);

Therefore, the object point P is regarded as the direction vector offset of the optical path of each segment by the camera origin O, and the offset vector is calculated in segments; the offset vector of each segment of the optical path is calculated as

Ez is the Z-axis direction vector, then the cumulative offset from point O to point P is

This gives the P point coordinates:

Where: d ₃ = Zd _{1 -} d ₂ , Z is the coordinate value of the P-axis Z-axis direction; then, the formula (4) is recorded as:

2) Establish the relationship between the object point and the image point at T2

The time T2 is any time different from the time of T1 during the rotation of the lens. According to the above principle, the same The direction vector of each section of the optical path is as follows:

v' _i+1 =a _i v' _i +b _i n' _i (6)

Where: a _i =μ _i /μ _i+1 ,

The initial camera ray v′ ₁ is obtained by calibration, and the direction vector of each segment of the optical path is derived by the equation (6); further, a relationship model between the object point and the image point is obtained, namely:

Where: d' ₃ = Zd' _{1 -} d' ₂ , Z is the coordinate value of the P-point Z-axis direction. Then, equation (7) is recorded as:

(3) Solve three-dimensional information

Equation (5) (8) is connected in series, and the relationship between the pixel coordinates of the image point and the three-dimensional coordinates of the object point under the camera coordinates is as follows:

Solving the three-dimensional coordinates of the object point P space, the measurement of the three-dimensional information of the space object point is completed.

The invention has the beneficial effects of measuring the three-dimensional information of the object based on the rotating lens, selecting the lens position at two different times, performing the monocular camera measurement on the three-dimensional information of the object point, and establishing the relationship model between the object point and the image point at two moments respectively. , in parallel, solve the three-dimensional information of the object point. This design reduces the traditional binocular camera system to a combination of a monocular camera and a rotating lens, which greatly reduces the economic cost of the original measurement system, speeds up the measurement, improves the accuracy of the measurement, and solves the monocular. Visual measurement of inaccurate three-dimensional information and high cost of binocular vision measurement.

DRAWINGS

FIG. 1 is a model diagram of a three-dimensional information visual measurement method based on a rotating lens.

2 is a flow chart of a three-dimensional information visual measurement method based on a rotating lens. The three-dimensional coordinates of the object point space are solved by the difference between the object point and the optical path between the image points by the rotation of the lens to T1 and T2.

Figure 3 is the optical path diagram and geometric parameters between the object point and the image point at time T1.

Figure 4 is a light path diagram between the object point and the image point at time T2.

In the picture: 1 high speed camera;

2 chamfered straight cylindrical lens, which can rotate around the center of the optical axis, and the geometric parameters and optical parameters are known;

3 obliquely cut straight cylindrical lens oblique refractive surface; 4 obliquely cut straight cylindrical lens parallel refractive surface;

5T1 time light path; 6T2 time light path.

Detailed ways

Specific embodiments of the present invention will be described in detail below with reference to the technical drawings and the accompanying drawings.

1 is a model diagram of a three-dimensional information visual measurement method based on a rotating lens. Through the positional transformation of the rotating lens in front of the camera, the optical path relationship model between the object point and the image point at the position of the rotating lens at T1 and T2 is established.

2 is a flow chart of a three-dimensional information visual measurement method based on a rotating lens. The main steps of the measurement method are to establish a relationship between object point points at T1 and T2 times, and then jointly solve the three-dimensional coordinates of the object point.

(1) Calibration camera with precision machined target board

The invention adopts a method in which the camera is calibrated with a precisely processed target plate in a relatively fixed manner. Simulation shooting conditions: camera CCD area is 2cm × 2cm, picture pixels are 1280 × 1024, lens focal length is 20mm. The internal parameters of the camera are calibrated according to the 2D planar target camera calibration method proposed by Zhang Zhengyou et al., and the internal parameters are as follows:

Set the shape parameters of the chamfered straight cylindrical lens as follows: diameter 50mm, lowest height 10mm, highest height 40mm.

According to the plane geometry principle, a mathematical model of the optical path between the object point and the image point is established.

Let the refractive index of air and rotating lens be μ ₁ =1, μ ₂ =1.6, respectively. At T1, the normal vectors of the parallel and oblique refractive surfaces of the rotating lens are n ₁ =[0,0,-1] ^{T, respectively.} , n ₂ = [0, -0.5145, -0.8575] ^T . [v ₁ , v ₂ , v ₃ ] are the optical path direction vectors from the optical center to the parallel refractive surface, the parallel refractive surface to the oblique refractive surface, and the oblique refractive surface to the object point P, respectively, where v ₁ =[0,0.0082,1] ^T ; [d ₁ , d ₂ , d ₃ ] are the projection distances of the optical paths from the optical center to the parallel refractive surface, the parallel refractive surface to the oblique refractive surface, and the oblique refractive surface to the object point P in the positive direction of the Z axis, respectively. d ₁ =150 mm; (X, Y, Z) ^T is the spatial three-dimensional coordinate of the object point P.

The object point P can be regarded as the direction vector offset of the optical path of each segment by the camera origin O, and the offset vector can be calculated in stages. Since v _{1 is} known, v ₂ =[0,0.0051,1] ^T and v ₃ =[0,-0.4026,0.9154] ^T can be calculated according to formula (3). Further, the reflection points of the optical path on the parallel refractive surface 4 and the oblique refractive surface 3 are respectively q ₁ = [0 mm, 1.2274 mm, 150 mm] ^T , q ₂ = [0 mm, 1.3511 mm, 174.1893 mm] ^T , q ₁ , The projection distance d ₂ in the positive direction of the Z axis is d ₂ = 24.1 m8, m9d ₃ = Zd _{1 -} d ₂ = Z-174.1893 (mm). Then, the P point coordinates can be obtained from the formula (4):

P=[0,77.9566-0.4398Z,Z] ^T (11)

According to the above principle, the parameters of the rotating lens at time T2 are as follows: the refractive indices of air and rotating lens are μ ₁ =1, μ ₂ = 1.6, respectively; the normal vectors of the parallel and oblique refractive surfaces of the rotating lens are n' ₁ respectively =[0,0,-1] ^T ,n' ₂ =[0,0.5145,-0.8575] ^T ; optical path from the optical center to the parallel refractive surface, the parallel refractive surface to the oblique refractive surface, and the oblique refractive surface to the object point P The vectors are respectively v' ₁ =[0,-0.1232,0.9924] ^T , v' ₂ =[0,-0.0770,0.9970] ^T ,v′ ₃ =[0,0.2536,0.9673] ^T ; the optical path is on the parallel refractive surface 4 And the reflection points on the oblique refractive surface 3 are q' ₁ = [0 mm, -18.6218 mm, 150 mm] ^T , q' ₂ = [0 mm, -19.6424 mm, 163.2146 mm] ^T ; the optical center to the parallel refractive surface, parallel The projection distances of the optical paths from the refractive surface to the oblique refractive surface and the oblique refractive surface to the object point P in the positive direction of the Z-axis are d' ₁ =150, d' ₂ = 13.2146 mm, d' ₃ = Zd' ₁ -d ' ₂ = Z-163.2146 (mm); the spatial three-dimensional coordinates of the object point P are (X, Y, Z) ^T .

Then, the coordinates of point P can be obtained from formula (7): P = [0, 0.2621Z-62.4249, Z] ^T (12)

(3) Solve three-dimensional information

Equation (11) (12) two sets of equations, can be obtained

Solving the three-dimensional coordinates of the object point P space, the measurement of the three-dimensional information of the space object point is completed. The obtained three-dimensional coordinates of the object point space are: P = [0 mm, -10.0043 mm, 200.0021 mm].

Claims

A three-dimensional information visual measuring method based on a rotating lens, characterized in that the three-dimensional information visual measuring method is based on a three-dimensional information visual measuring device of a rotating lens, comprising a monocular camera and a rotating lens, wherein the rotating lens adopts light transmission 80% or more of the glass material is chamfered straight cylinder, placed in front of the monocular camera lens, its axis coincides with the optical axis, and the parallel surface is opposite to the lens;

During the measurement, the rotating lens is rotated. At different times, the normal direction and the thickness of the optical path between the object point and the image point through the rotating lens bevel are different, and the lens parameters at two moments and the parameters in the monocular camera are constructed. The relationship model between the point and the three-dimensional coordinates of the image point, the measurement of the three-dimensional information of the object by binocular vision is converted into the measurement of the three-dimensional information of the object at different times of the monocular camera and the rotating lens;

Proceed as follows:

(1) Calibration camera with precision machined target board

The camera calibration method based on 2D planar target calibrates the internal parameters of the camera, and obtains the conversion relationship between the coordinates (u, v) T in the pixel coordinate system and the coordinates (x, y) T in the image coordinate system as follows:

Where: (dx, dy) is the physical size corresponding to each pixel on the x-axis and the y-axis, and (u 0 , v 0 ) is the coordinate of the origin of the image coordinate system in the pixel coordinate system;

(2) Establishing the relationship between the object point and the image point at two times T1 and T2

1) Establish the relationship between the object point and the image point at T1

The inclined surface of the rotating lens is the surface 3, the plane of the rotating lens is the surface 4; the time T1 is the arbitrary moment of the rotation of the lens, and at the time T1, the rotating lens is in a stationary state, [n 1 , n 2 ] are respectively the surface 4 and the surface 3 The normal vector; [v 1 , v 2 , v 3 ] is the optical path direction vector of the optical center to the surface 4, the surface 4 to the surface 3, and the surface 3 to the object point P; [q 1 , q 2 ] are respectively the optical path 4 and the reflection point on the surface 3; [d 1 , d 2 , d 3 ] are the projections of the optical paths from the optical center to the surface 4, the surface 4 to the surface 3, and the surface 3 to the object point P in the positive direction of the Z axis. Distance; (X, Y, Z) T is the spatial three-dimensional coordinate of the object point P;

From Snell's law: μ i sin θ i = μ i+1 sin θ i+1 (2)

Where: μ i and μ i+1 are the refractive indices of the two media respectively; let v i and v i+1 be the incident and outgoing optical path vectors respectively, then θ 1 is the angle between v 1 and n 1 , θ 2 is the angle between v 2 and n 1 , θ 3 is the angle between v 2 and n 2 , and θ 4 is the angle between v 3 and n 2 ; the formula (2) is written in the form of a vector,

v i+1 =a i v i +b i n i (3)

Where: a i =μ i /μ i+1 ,
The initial camera ray v 1 is obtained by calibration, and the direction vector of each segment of the optical path is derived by the equation (3);

Therefore, the object point P is regarded as the direction vector offset of the optical path of each segment by the camera origin O, and the offset vector is calculated in segments; the offset vector of each segment of the optical path is calculated as
e z is the Z-axis direction vector, then the cumulative offset from point O to point P is
This gives the P point coordinates:

Where: d 3 = Zd 1 - d 2 , Z is the coordinate value of the P-axis Z-axis direction; then, the formula (4) is recorded as:

2) Establish the relationship between the object point and the image point at T2

The T2 time is any time different from the time T1 during the rotation of the lens. According to the above principle, the direction vector of each optical path is also obtained as follows:

v' i+1 =a i v' i +b i n' i (6)

Where: a i =μ i /μ i+1 ,
The initial camera ray v′ 1 is obtained by calibration, and the direction vector of each segment of the optical path is derived by the equation (6); further, a relationship model between the object point and the image point is obtained, namely:

Where: d' 3 = Zd' 1 -d' 2 , Z is the coordinate value of the Z-axis direction of point P; then, equation (7) is recorded as:

(3) Solve three-dimensional information

Equation (5) (8) is connected in series, and the relationship between the pixel coordinates of the image point and the three-dimensional coordinates of the object point under the camera coordinates is as follows:

Solving the three-dimensional coordinates of the object point P space, the measurement of the three-dimensional information of the space object point is completed.